Sep 07 10:08:02.187120 ip-10-0-141-85 systemd[1]: Starting Kubernetes Kubelet... Sep 07 10:08:02.591509 ip-10-0-141-85 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 07 10:08:02.591509 ip-10-0-141-85 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 07 10:08:02.591509 ip-10-0-141-85 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 07 10:08:02.591509 ip-10-0-141-85 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 07 10:08:02.591509 ip-10-0-141-85 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 07 10:08:02.593696 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.593594 2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 07 10:08:02.597230 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597206 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 10:08:02.597230 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597224 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 10:08:02.597230 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597229 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 10:08:02.597230 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597233 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 10:08:02.597476 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597239 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 10:08:02.597476 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597243 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 10:08:02.597476 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597247 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 10:08:02.597476 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597251 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 10:08:02.597476 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597255 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 10:08:02.597476 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597258 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 10:08:02.597476 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597262 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 10:08:02.597476 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597266 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 10:08:02.597476 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597270 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 10:08:02.597476 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597274 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 10:08:02.597476 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597277 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 10:08:02.597476 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597281 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 10:08:02.597476 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597285 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 10:08:02.597476 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597289 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 10:08:02.597476 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597293 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 10:08:02.597476 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597297 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 10:08:02.597476 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597301 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 10:08:02.597476 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597312 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 10:08:02.597476 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597318 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 10:08:02.597476 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597322 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 10:08:02.598317 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597325 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 10:08:02.598317 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597330 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 10:08:02.598317 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597334 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 10:08:02.598317 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597338 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 10:08:02.598317 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597342 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 10:08:02.598317 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597346 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 10:08:02.598317 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597353 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 10:08:02.598317 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597357 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 10:08:02.598317 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597362 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 10:08:02.598317 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597366 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 10:08:02.598317 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597370 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 10:08:02.598317 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597375 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 10:08:02.598317 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597379 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 10:08:02.598317 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597384 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 10:08:02.598317 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597388 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 10:08:02.598317 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597392 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 10:08:02.598317 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597396 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 10:08:02.598317 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597400 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 10:08:02.598317 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597405 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 10:08:02.598317 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597409 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 10:08:02.599215 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597413 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 10:08:02.599215 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597418 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 10:08:02.599215 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597422 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 10:08:02.599215 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597426 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 10:08:02.599215 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597430 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 10:08:02.599215 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597434 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 10:08:02.599215 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597438 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 10:08:02.599215 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597443 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 10:08:02.599215 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597448 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 10:08:02.599215 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597455 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 10:08:02.599215 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597461 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 10:08:02.599215 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597466 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 10:08:02.599215 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597471 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 10:08:02.599215 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597475 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 10:08:02.599215 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597480 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 10:08:02.599215 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597484 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 10:08:02.599215 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597488 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 10:08:02.599215 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597492 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 10:08:02.599215 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597497 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 10:08:02.599215 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597501 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 10:08:02.600036 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597505 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 10:08:02.600036 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597509 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 10:08:02.600036 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597514 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 10:08:02.600036 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597520 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 10:08:02.600036 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597524 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 10:08:02.600036 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597529 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 10:08:02.600036 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597535 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 10:08:02.600036 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597543 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 10:08:02.600036 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597548 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 10:08:02.600036 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597552 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 10:08:02.600036 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597558 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 10:08:02.600036 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597562 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 10:08:02.600036 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597566 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 10:08:02.600036 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597570 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 10:08:02.600036 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597576 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 10:08:02.600036 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597582 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 10:08:02.600036 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597586 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 10:08:02.600036 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597591 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 10:08:02.600818 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597598 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 10:08:02.600818 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597603 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 10:08:02.600818 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597607 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 10:08:02.600818 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597612 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 10:08:02.600818 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597616 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 10:08:02.600818 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597620 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 10:08:02.600818 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597625 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 10:08:02.600818 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597630 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 10:08:02.600818 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597634 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 10:08:02.600818 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597638 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 10:08:02.600818 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597643 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 10:08:02.600818 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597647 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 10:08:02.600818 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597652 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 10:08:02.600818 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597657 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 10:08:02.600818 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597661 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 07 10:08:02.600818 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597665 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 10:08:02.600818 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.597669 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 10:08:02.600818 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.598868 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 10:08:02.600818 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.598880 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 10:08:02.600818 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.598886 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 10:08:02.601659 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.598890 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 10:08:02.601659 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.598894 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 10:08:02.601659 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.598923 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 10:08:02.601659 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.598929 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 10:08:02.601659 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.598933 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 10:08:02.601659 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.598937 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 10:08:02.601659 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.598944 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 10:08:02.601659 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.598950 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 10:08:02.601659 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.598955 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 10:08:02.601659 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.598960 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 10:08:02.601659 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.598967 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 10:08:02.601659 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.598972 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 10:08:02.601659 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.598978 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 10:08:02.601659 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.598982 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 10:08:02.601659 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.598989 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 10:08:02.601659 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.598995 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 10:08:02.601659 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599000 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 10:08:02.601659 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599004 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 10:08:02.602403 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599008 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 10:08:02.602403 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599013 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 10:08:02.602403 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599018 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 10:08:02.602403 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599022 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 10:08:02.602403 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599027 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 10:08:02.602403 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599032 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 10:08:02.602403 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599036 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 10:08:02.602403 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599040 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 10:08:02.602403 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599045 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 10:08:02.602403 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599049 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 10:08:02.602403 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599053 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 10:08:02.602403 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599058 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 10:08:02.602403 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599062 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 10:08:02.602403 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599067 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 10:08:02.602403 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599071 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 10:08:02.602403 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599075 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 10:08:02.602403 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599079 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 10:08:02.602403 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599084 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 10:08:02.602403 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599088 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 10:08:02.602403 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599093 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 10:08:02.602403 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599097 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 10:08:02.602979 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599101 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 10:08:02.602979 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599106 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 10:08:02.602979 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599110 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 10:08:02.602979 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599115 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 10:08:02.602979 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599119 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 10:08:02.602979 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599123 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 10:08:02.602979 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599128 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 10:08:02.602979 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599133 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 10:08:02.602979 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599138 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 10:08:02.602979 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599142 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 10:08:02.602979 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599147 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 10:08:02.602979 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599151 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 10:08:02.602979 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599155 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 10:08:02.602979 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599160 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 10:08:02.602979 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599164 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 10:08:02.602979 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599169 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 10:08:02.602979 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599174 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 10:08:02.602979 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599178 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 10:08:02.602979 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599182 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 10:08:02.603450 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599187 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 10:08:02.603450 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599192 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 10:08:02.603450 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599197 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 10:08:02.603450 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599201 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 10:08:02.603450 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599205 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 10:08:02.603450 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599211 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 10:08:02.603450 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599216 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 10:08:02.603450 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599221 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 10:08:02.603450 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599226 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 10:08:02.603450 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599229 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 10:08:02.603450 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599234 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 10:08:02.603450 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599239 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 10:08:02.603450 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599243 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 10:08:02.603450 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599247 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 10:08:02.603450 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599251 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 10:08:02.603450 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599255 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 10:08:02.603450 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599260 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 10:08:02.603450 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599264 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 10:08:02.603450 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599268 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 10:08:02.604016 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599272 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 10:08:02.604016 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599276 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 10:08:02.604016 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599281 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 10:08:02.604016 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599285 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 10:08:02.604016 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599289 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 10:08:02.604016 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599294 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 07 10:08:02.604016 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599298 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 10:08:02.604016 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599302 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 10:08:02.604016 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599307 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 10:08:02.604016 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599311 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 10:08:02.604016 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599316 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 10:08:02.604016 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599320 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 10:08:02.604016 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599324 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 10:08:02.604016 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599328 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 10:08:02.604016 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599332 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 10:08:02.604016 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599336 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 10:08:02.604016 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599341 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 10:08:02.604016 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599345 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 10:08:02.604016 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.599349 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 10:08:02.604016 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600004 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 10:08:02.604519 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600013 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 10:08:02.604519 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600018 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 10:08:02.604519 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600022 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 10:08:02.604519 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600027 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 10:08:02.604519 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600031 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 10:08:02.604519 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600035 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 10:08:02.604519 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600039 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 10:08:02.604519 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600043 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 10:08:02.604519 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600047 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 10:08:02.604519 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600052 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 10:08:02.604519 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600059 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 10:08:02.604519 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600066 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 10:08:02.604519 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600070 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 10:08:02.604519 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600075 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 10:08:02.604519 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600080 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 10:08:02.604519 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600084 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 10:08:02.604519 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600089 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 10:08:02.604519 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600093 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 10:08:02.604519 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600098 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 10:08:02.605024 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600102 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 10:08:02.605024 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600106 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 10:08:02.605024 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600111 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 10:08:02.605024 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600115 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 10:08:02.605024 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600120 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 10:08:02.605024 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600124 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 10:08:02.605024 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600129 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 10:08:02.605024 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600133 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 10:08:02.605024 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600137 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 10:08:02.605024 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600141 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 10:08:02.605024 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600146 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 10:08:02.605024 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600150 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 10:08:02.605024 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600154 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 10:08:02.605024 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600158 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 10:08:02.605024 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600162 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 10:08:02.605024 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600167 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 10:08:02.605024 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600171 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 10:08:02.605024 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600182 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 10:08:02.605024 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600186 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 10:08:02.605024 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600190 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 07 10:08:02.605512 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600194 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 10:08:02.605512 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600198 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 10:08:02.605512 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600202 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 10:08:02.605512 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600206 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 10:08:02.605512 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600210 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 10:08:02.605512 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600214 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 10:08:02.605512 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600218 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 10:08:02.605512 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600223 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 10:08:02.605512 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600227 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 10:08:02.605512 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600231 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 10:08:02.605512 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600239 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 10:08:02.605512 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600244 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 10:08:02.605512 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600249 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 10:08:02.605512 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600254 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 10:08:02.605512 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600262 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 10:08:02.605512 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600266 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 10:08:02.605512 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600270 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 10:08:02.605512 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600275 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 10:08:02.605512 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600279 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 10:08:02.605512 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600283 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 10:08:02.606060 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600288 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 10:08:02.606060 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600292 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 10:08:02.606060 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600296 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 10:08:02.606060 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600300 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 10:08:02.606060 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600304 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 10:08:02.606060 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600309 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 10:08:02.606060 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600313 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 10:08:02.606060 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600317 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 10:08:02.606060 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600322 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 10:08:02.606060 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600326 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 10:08:02.606060 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600331 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 10:08:02.606060 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600343 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 10:08:02.606060 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600349 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 10:08:02.606060 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600353 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 10:08:02.606060 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600357 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 10:08:02.606060 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600361 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 10:08:02.606060 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600366 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 10:08:02.606060 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600369 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 10:08:02.606060 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600373 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 10:08:02.606060 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600378 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 10:08:02.606541 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600382 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 10:08:02.606541 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600386 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 10:08:02.606541 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600391 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 10:08:02.606541 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600395 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 10:08:02.606541 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600399 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 10:08:02.606541 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600403 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 10:08:02.606541 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600408 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 10:08:02.606541 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600412 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 10:08:02.606541 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600416 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 10:08:02.606541 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600421 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 10:08:02.606541 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600425 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 10:08:02.606541 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600429 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 10:08:02.606541 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600433 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 10:08:02.606541 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600438 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 10:08:02.606541 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600442 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 10:08:02.606541 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600446 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 10:08:02.606541 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600452 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 10:08:02.606541 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600458 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 10:08:02.606541 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.600462 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600573 2575 flags.go:64] FLAG: --address="0.0.0.0" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600626 2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600635 2575 flags.go:64] FLAG: --anonymous-auth="true" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600642 2575 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600652 2575 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600657 2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600663 2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600670 2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600676 2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600681 2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600686 2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600692 2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600697 2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600702 2575 flags.go:64] FLAG: --cgroup-root="" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600707 2575 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600713 2575 flags.go:64] FLAG: --client-ca-file="" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600717 2575 flags.go:64] FLAG: --cloud-provider="external" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600722 2575 flags.go:64] FLAG: --cluster-dns="[]" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600730 2575 flags.go:64] FLAG: --cluster-domain="" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600739 2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600744 2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600749 2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600755 2575 flags.go:64] FLAG: --container-log-max-files="5" Sep 07 10:08:02.607013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600761 2575 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600767 2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600772 2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600777 2575 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600783 2575 flags.go:64] FLAG: --contention-profiling="false" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600788 2575 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600815 2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600823 2575 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600828 2575 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600834 2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600839 2575 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600844 2575 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600849 2575 flags.go:64] FLAG: --enable-load-reader="false" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600854 2575 flags.go:64] FLAG: --enable-server="true" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600865 2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600872 2575 flags.go:64] FLAG: --event-burst="100" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600878 2575 flags.go:64] FLAG: --event-qps="50" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600882 2575 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600888 2575 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600892 2575 flags.go:64] FLAG: --eviction-hard="" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600899 2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600904 2575 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600909 2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600914 2575 flags.go:64] FLAG: --eviction-soft="" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600919 2575 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 07 10:08:02.607614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600924 2575 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600929 2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600934 2575 flags.go:64] FLAG: --experimental-mounter-path="" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600939 2575 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600944 2575 flags.go:64] FLAG: --fail-swap-on="true" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600950 2575 flags.go:64] FLAG: --feature-gates="" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600957 2575 flags.go:64] FLAG: --file-check-frequency="20s" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600962 2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600967 2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600972 2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600977 2575 flags.go:64] FLAG: --healthz-port="10248" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600981 2575 flags.go:64] FLAG: --help="false" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600986 2575 flags.go:64] FLAG: --hostname-override="ip-10-0-141-85.ec2.internal" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600991 2575 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.600998 2575 flags.go:64] FLAG: --http-check-frequency="20s" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601003 2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601008 2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601014 2575 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601019 2575 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601023 2575 flags.go:64] FLAG: --image-service-endpoint="" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601028 2575 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601033 2575 flags.go:64] FLAG: --kube-api-burst="100" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601039 2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601044 2575 flags.go:64] FLAG: --kube-api-qps="50" Sep 07 10:08:02.608249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601049 2575 flags.go:64] FLAG: --kube-reserved="" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601054 2575 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601058 2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601063 2575 flags.go:64] FLAG: --kubelet-cgroups="" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601068 2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601072 2575 flags.go:64] FLAG: --lock-file="" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601077 2575 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601082 2575 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601087 2575 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601095 2575 flags.go:64] FLAG: --log-json-split-stream="false" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601100 2575 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601105 2575 flags.go:64] FLAG: --log-text-split-stream="false" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601110 2575 flags.go:64] FLAG: --logging-format="text" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601115 2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601120 2575 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601126 2575 flags.go:64] FLAG: --manifest-url="" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601130 2575 flags.go:64] FLAG: --manifest-url-header="" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601137 2575 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601142 2575 flags.go:64] FLAG: --max-open-files="1000000" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601150 2575 flags.go:64] FLAG: --max-pods="110" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601155 2575 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601166 2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601173 2575 flags.go:64] FLAG: --memory-manager-policy="None" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601178 2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601183 2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 07 10:08:02.608941 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601188 2575 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601193 2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601205 2575 flags.go:64] FLAG: --node-status-max-images="50" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601210 2575 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601215 2575 flags.go:64] FLAG: --oom-score-adj="-999" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601220 2575 flags.go:64] FLAG: --pod-cidr="" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601226 2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601235 2575 flags.go:64] FLAG: --pod-manifest-path="" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601240 2575 flags.go:64] FLAG: --pod-max-pids="-1" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601245 2575 flags.go:64] FLAG: --pods-per-core="0" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601250 2575 flags.go:64] FLAG: --port="10250" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601255 2575 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601260 2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0665e849daadd5265" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601265 2575 flags.go:64] FLAG: --qos-reserved="" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601270 2575 flags.go:64] FLAG: --read-only-port="10255" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601275 2575 flags.go:64] FLAG: --register-node="true" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601280 2575 flags.go:64] FLAG: --register-with-taints="" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601286 2575 flags.go:64] FLAG: --registry-burst="10" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601290 2575 flags.go:64] FLAG: --registry-qps="5" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601295 2575 flags.go:64] FLAG: --reserved-cpus="" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601300 2575 flags.go:64] FLAG: --reserved-memory="" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601306 2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601311 2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601317 2575 flags.go:64] FLAG: --rotate-certificates="false" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601321 2575 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 07 10:08:02.609635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601326 2575 flags.go:64] FLAG: --runonce="false" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601331 2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601335 2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601340 2575 flags.go:64] FLAG: --seccomp-default="false" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601346 2575 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601352 2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601356 2575 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601362 2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601366 2575 flags.go:64] FLAG: --storage-driver-password="root" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601371 2575 flags.go:64] FLAG: --storage-driver-secure="false" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601376 2575 flags.go:64] FLAG: --storage-driver-table="stats" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601381 2575 flags.go:64] FLAG: --storage-driver-user="root" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601385 2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601390 2575 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601396 2575 flags.go:64] FLAG: --system-cgroups="" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601401 2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601409 2575 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601415 2575 flags.go:64] FLAG: --tls-cert-file="" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601419 2575 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601426 2575 flags.go:64] FLAG: --tls-min-version="" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601431 2575 flags.go:64] FLAG: --tls-private-key-file="" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601436 2575 flags.go:64] FLAG: --topology-manager-policy="none" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601441 2575 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601445 2575 flags.go:64] FLAG: --topology-manager-scope="container" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601450 2575 flags.go:64] FLAG: --v="2" Sep 07 10:08:02.610270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601456 2575 flags.go:64] FLAG: --version="false" Sep 07 10:08:02.610899 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601463 2575 flags.go:64] FLAG: --vmodule="" Sep 07 10:08:02.610899 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601470 2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 07 10:08:02.610899 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.601475 2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 07 10:08:02.610899 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601661 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 10:08:02.610899 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601668 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 10:08:02.610899 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601673 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 10:08:02.610899 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601678 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 10:08:02.610899 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601683 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 10:08:02.610899 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601688 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 10:08:02.610899 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601692 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 10:08:02.610899 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601699 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 10:08:02.610899 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601703 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 10:08:02.610899 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601707 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 10:08:02.610899 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601712 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 10:08:02.610899 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601716 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 10:08:02.610899 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601720 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 10:08:02.610899 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601724 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 10:08:02.610899 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601728 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 10:08:02.610899 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601732 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 10:08:02.610899 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601736 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 10:08:02.610899 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601740 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 10:08:02.611447 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601745 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 10:08:02.611447 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601749 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 10:08:02.611447 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601753 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 10:08:02.611447 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601758 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 10:08:02.611447 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601762 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 10:08:02.611447 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601766 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 10:08:02.611447 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601770 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 10:08:02.611447 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601774 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 10:08:02.611447 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601777 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 10:08:02.611447 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601784 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 10:08:02.611447 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601789 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 10:08:02.611447 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601814 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 10:08:02.611447 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601819 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 10:08:02.611447 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601823 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 10:08:02.611447 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601827 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 10:08:02.611447 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601832 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 10:08:02.611447 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601836 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 10:08:02.611447 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601839 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 10:08:02.611447 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601844 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 10:08:02.611447 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601848 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 10:08:02.611956 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601853 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 10:08:02.611956 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601859 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 07 10:08:02.611956 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601864 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 10:08:02.611956 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601868 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 10:08:02.611956 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601872 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 10:08:02.611956 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601876 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 10:08:02.611956 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601880 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 10:08:02.611956 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601885 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 10:08:02.611956 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601889 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 10:08:02.611956 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601893 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 10:08:02.611956 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601897 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 10:08:02.611956 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601902 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 10:08:02.611956 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601906 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 10:08:02.611956 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601916 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 10:08:02.611956 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601920 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 10:08:02.611956 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601924 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 10:08:02.611956 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601928 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 10:08:02.611956 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601932 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 10:08:02.611956 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601936 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 10:08:02.611956 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601940 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 10:08:02.612458 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601945 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 10:08:02.612458 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601949 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 10:08:02.612458 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601953 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 10:08:02.612458 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601960 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 10:08:02.612458 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601964 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 10:08:02.612458 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601968 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 10:08:02.612458 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601973 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 10:08:02.612458 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601977 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 10:08:02.612458 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601982 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 10:08:02.612458 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601986 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 10:08:02.612458 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601990 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 10:08:02.612458 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601994 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 10:08:02.612458 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.601998 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 10:08:02.612458 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602005 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 10:08:02.612458 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602009 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 10:08:02.612458 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602014 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 10:08:02.612458 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602018 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 10:08:02.612458 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602022 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 10:08:02.612458 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602027 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 10:08:02.612458 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602031 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 10:08:02.613059 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602035 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 10:08:02.613059 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602039 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 10:08:02.613059 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602043 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 10:08:02.613059 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602047 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 10:08:02.613059 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602052 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 10:08:02.613059 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602056 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 10:08:02.613059 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602061 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 10:08:02.613059 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602065 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 10:08:02.613059 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602072 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 10:08:02.613059 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602078 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 10:08:02.613059 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602082 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 10:08:02.613059 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602086 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 10:08:02.613059 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602090 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 10:08:02.613059 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602094 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 10:08:02.613059 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602099 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 10:08:02.613059 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602104 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 10:08:02.613059 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602111 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 10:08:02.613059 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602116 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 10:08:02.613523 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602121 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 10:08:02.613523 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602126 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 10:08:02.613523 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.602131 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 10:08:02.613523 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.602140 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 07 10:08:02.613523 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.610082 2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 07 10:08:02.613523 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.610102 2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 07 10:08:02.613523 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.611002 2575 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 07 10:08:02.613523 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.611010 2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 07 10:08:02.613523 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611067 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 10:08:02.613523 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611073 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 10:08:02.613523 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611076 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 10:08:02.613523 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611080 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 10:08:02.613523 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611083 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 10:08:02.613523 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611086 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 10:08:02.613523 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611089 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 10:08:02.613922 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611092 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 10:08:02.613922 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611095 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 10:08:02.613922 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611098 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 10:08:02.613922 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611101 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 10:08:02.613922 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611103 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 10:08:02.613922 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611106 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 10:08:02.613922 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611109 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 10:08:02.613922 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611113 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 10:08:02.613922 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611116 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 10:08:02.613922 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611119 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 10:08:02.613922 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611122 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 10:08:02.613922 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611124 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 10:08:02.613922 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611127 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 10:08:02.613922 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611129 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 10:08:02.613922 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611132 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 10:08:02.613922 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611135 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 10:08:02.613922 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611137 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 10:08:02.613922 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611140 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 10:08:02.613922 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611142 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 10:08:02.614381 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611145 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 10:08:02.614381 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611148 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 10:08:02.614381 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611150 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 10:08:02.614381 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611155 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 10:08:02.614381 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611158 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 10:08:02.614381 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611161 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 10:08:02.614381 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611164 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 10:08:02.614381 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611167 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 10:08:02.614381 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611169 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 10:08:02.614381 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611172 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 10:08:02.614381 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611174 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 10:08:02.614381 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611177 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 10:08:02.614381 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611180 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 10:08:02.614381 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611182 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 10:08:02.614381 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611185 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 10:08:02.614381 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611187 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 10:08:02.614381 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611190 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 10:08:02.614381 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611192 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 10:08:02.614381 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611195 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 10:08:02.615081 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611198 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 10:08:02.615081 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611200 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 10:08:02.615081 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611203 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 10:08:02.615081 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611206 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 10:08:02.615081 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611208 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 10:08:02.615081 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611211 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 10:08:02.615081 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611213 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 10:08:02.615081 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611216 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 10:08:02.615081 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611218 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 10:08:02.615081 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611221 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 10:08:02.615081 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611223 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 10:08:02.615081 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611226 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 10:08:02.615081 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611228 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 10:08:02.615081 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611231 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 10:08:02.615081 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611233 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 10:08:02.615081 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611236 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 10:08:02.615081 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611241 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 10:08:02.615081 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611245 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 10:08:02.615081 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611248 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 10:08:02.615081 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611251 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 10:08:02.615923 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611254 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 10:08:02.615923 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611256 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 10:08:02.615923 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611260 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 10:08:02.615923 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611263 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 07 10:08:02.615923 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611266 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 10:08:02.615923 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611269 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 10:08:02.615923 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611272 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 10:08:02.615923 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611275 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 10:08:02.615923 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611278 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 10:08:02.615923 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611280 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 10:08:02.615923 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611283 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 10:08:02.615923 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611286 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 10:08:02.615923 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611288 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 10:08:02.615923 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611291 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 10:08:02.615923 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611293 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 10:08:02.615923 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611296 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 10:08:02.615923 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611298 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 10:08:02.615923 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611301 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 10:08:02.615923 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611303 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 10:08:02.616515 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611306 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 10:08:02.616515 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611309 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 10:08:02.616515 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611312 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 10:08:02.616515 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611314 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 10:08:02.616515 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611317 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 10:08:02.616515 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611319 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 10:08:02.616515 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611322 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 10:08:02.616515 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611325 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 10:08:02.616515 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611328 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 10:08:02.616515 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611330 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 10:08:02.616515 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611333 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 10:08:02.616515 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611336 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 10:08:02.616515 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611338 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 10:08:02.616515 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611341 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 10:08:02.616515 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611344 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 10:08:02.616999 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.611349 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 07 10:08:02.616999 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611490 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 10:08:02.616999 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611497 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 10:08:02.616999 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611502 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 10:08:02.616999 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611505 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 10:08:02.616999 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611509 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 10:08:02.616999 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611512 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 10:08:02.616999 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611515 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 10:08:02.616999 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611518 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 10:08:02.616999 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611521 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 10:08:02.616999 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611525 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 10:08:02.616999 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611528 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 10:08:02.616999 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611530 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 10:08:02.616999 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611533 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 10:08:02.617421 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611536 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 10:08:02.617421 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611540 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 10:08:02.617421 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611543 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 10:08:02.617421 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611546 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 10:08:02.617421 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611549 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 10:08:02.617421 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611552 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 10:08:02.617421 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611555 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 10:08:02.617421 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611558 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 10:08:02.617421 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611561 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 10:08:02.617421 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611564 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 10:08:02.617421 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611566 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 10:08:02.617421 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611569 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 10:08:02.617421 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611572 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 10:08:02.617421 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611574 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 10:08:02.617421 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611577 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 10:08:02.617421 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611580 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 10:08:02.617421 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611582 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 10:08:02.617421 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611585 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 10:08:02.617421 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611588 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 10:08:02.617909 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611591 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 10:08:02.617909 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611593 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 10:08:02.617909 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611596 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 10:08:02.617909 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611598 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 10:08:02.617909 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611601 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 10:08:02.617909 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611604 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 10:08:02.617909 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611606 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 10:08:02.617909 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611608 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 10:08:02.617909 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611611 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 10:08:02.617909 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611613 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 10:08:02.617909 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611616 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 10:08:02.617909 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611618 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 10:08:02.617909 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611621 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 10:08:02.617909 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611623 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 10:08:02.617909 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611626 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 10:08:02.617909 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611629 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 10:08:02.617909 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611631 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 10:08:02.617909 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611634 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 10:08:02.617909 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611637 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 10:08:02.617909 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611639 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 10:08:02.618451 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611642 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 10:08:02.618451 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611644 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 10:08:02.618451 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611647 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 10:08:02.618451 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611650 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 10:08:02.618451 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611652 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 10:08:02.618451 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611655 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 10:08:02.618451 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611658 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 10:08:02.618451 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611660 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 10:08:02.618451 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611663 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 10:08:02.618451 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611666 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 10:08:02.618451 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611669 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 10:08:02.618451 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611671 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 10:08:02.618451 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611674 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 10:08:02.618451 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611676 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 10:08:02.618451 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611679 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 10:08:02.618451 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611682 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 10:08:02.618451 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611685 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 10:08:02.618451 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611687 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 10:08:02.618451 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611690 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 10:08:02.618451 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611692 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 10:08:02.619023 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611695 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 10:08:02.619023 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611697 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 10:08:02.619023 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611700 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 10:08:02.619023 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611702 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 10:08:02.619023 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611705 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 10:08:02.619023 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611707 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 10:08:02.619023 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611710 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 10:08:02.619023 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611713 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 10:08:02.619023 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611715 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 10:08:02.619023 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611718 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 10:08:02.619023 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611720 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 10:08:02.619023 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611723 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 10:08:02.619023 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611725 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 10:08:02.619023 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611728 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 10:08:02.619023 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611730 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 10:08:02.619023 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611733 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 07 10:08:02.619023 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611736 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 10:08:02.619023 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611738 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 10:08:02.619023 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611741 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 10:08:02.619023 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611743 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 10:08:02.619618 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611746 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 10:08:02.619618 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611748 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 10:08:02.619618 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611751 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 10:08:02.619618 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611754 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 10:08:02.619618 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611757 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 10:08:02.619618 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611759 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 10:08:02.619618 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:02.611762 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 10:08:02.619618 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.611767 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 07 10:08:02.619618 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.611917 2575 server.go:962] "Client rotation is on, will bootstrap in background" Sep 07 10:08:02.619618 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.617875 2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 07 10:08:02.619618 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.619084 2575 server.go:1019] "Starting client certificate rotation" Sep 07 10:08:02.619618 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.619202 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 07 10:08:02.619618 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.619253 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 07 10:08:02.622714 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.622695 2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 07 10:08:02.623850 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.623819 2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 07 10:08:02.634901 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.634878 2575 log.go:25] "Validated CRI v1 runtime API" Sep 07 10:08:02.639370 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.639351 2575 log.go:25] "Validated CRI v1 image API" Sep 07 10:08:02.640300 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.640275 2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 07 10:08:02.642881 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.642859 2575 fs.go:135] Filesystem UUIDs: map[1495fc44-4765-40ea-867b-7032713cc65f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c74468e2-0248-4a24-a804-3ed90986cf57:/dev/nvme0n1p3] Sep 07 10:08:02.642959 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.642880 2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 07 10:08:02.648101 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.648081 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 07 10:08:02.648462 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.648347 2575 manager.go:216] Machine: {Timestamp:2026-09-07 10:08:02.646942853 +0000 UTC m=+0.359184788 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100031 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2531ed274a3ff7d2d3b2438d9b6310 SystemUUID:ec2531ed-274a-3ff7-d2d3-b2438d9b6310 BootID:ee976604-4b57-46ec-a46f-918407ec6cf3 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c3:ac:a8:a2:23 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c3:ac:a8:a2:23 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d2:f1:d7:0d:0d:0b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 07 10:08:02.648462 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.648458 2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 07 10:08:02.648569 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.648557 2575 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 07 10:08:02.652994 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.652962 2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 07 10:08:02.653159 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.652999 2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-85.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 07 10:08:02.654706 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.654694 2575 topology_manager.go:138] "Creating topology manager with none policy" Sep 07 10:08:02.654706 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.654708 2575 container_manager_linux.go:309] "Creating device plugin manager" Sep 07 10:08:02.654773 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.654716 2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 07 10:08:02.656099 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.656086 2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 07 10:08:02.657880 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.657866 2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 07 10:08:02.662767 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.662751 2575 state_mem.go:36] "Initialized new in-memory state store" Sep 07 10:08:02.662909 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.662899 2575 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 07 10:08:02.662954 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.662934 2575 kubelet.go:486] "Attempting to sync node with API server" Sep 07 10:08:02.662985 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.662954 2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 07 10:08:02.662985 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.662970 2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 07 10:08:02.662985 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.662980 2575 kubelet.go:398] "Adding apiserver pod source" Sep 07 10:08:02.663088 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.662989 2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 07 10:08:02.663408 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.663391 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 07 10:08:02.663501 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.663412 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 07 10:08:02.666615 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.666549 2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 07 10:08:02.669397 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.669379 2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 07 10:08:02.669467 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.669412 2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 07 10:08:02.670757 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.670745 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 07 10:08:02.670814 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.670763 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 07 10:08:02.670814 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.670770 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 07 10:08:02.670814 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.670776 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 07 10:08:02.670814 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.670785 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 07 10:08:02.670814 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.670791 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 07 10:08:02.670955 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.670817 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 07 10:08:02.670955 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.670825 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 07 10:08:02.670955 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.670836 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 07 10:08:02.670955 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.670844 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 07 10:08:02.670955 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.670856 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 07 10:08:02.670955 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.670862 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 07 10:08:02.671114 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.670965 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v64r2" Sep 07 10:08:02.671571 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.671560 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 07 10:08:02.671571 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.671571 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 07 10:08:02.673608 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:02.673573 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 07 10:08:02.673685 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:02.673609 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 07 10:08:02.673685 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.673666 2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 10:08:02.675410 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.675397 2575 server.go:1268] "Started kubelet" Sep 07 10:08:02.675568 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.675514 2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 07 10:08:02.675604 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.675593 2575 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 07 10:08:02.675763 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.675717 2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 07 10:08:02.676461 ip-10-0-141-85 systemd[1]: Started Kubernetes Kubelet. Sep 07 10:08:02.678775 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.678598 2575 server.go:310] "Adding debug handlers to kubelet server" Sep 07 10:08:02.678904 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.678644 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v64r2" Sep 07 10:08:02.680604 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:02.680585 2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 07 10:08:02.680701 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.680687 2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 07 10:08:02.686053 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.686033 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 07 10:08:02.687043 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.687028 2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 07 10:08:02.687163 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:02.687145 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-85.ec2.internal\" not found" Sep 07 10:08:02.687785 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.687768 2575 factory.go:55] Registering systemd factory Sep 07 10:08:02.687785 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.687786 2575 factory.go:223] Registration of the systemd container factory successfully Sep 07 10:08:02.688038 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.688016 2575 factory.go:153] Registering CRI-O factory Sep 07 10:08:02.688038 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.688030 2575 factory.go:223] Registration of the crio container factory successfully Sep 07 10:08:02.688140 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.688078 2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 07 10:08:02.688140 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.688093 2575 factory.go:103] Registering Raw factory Sep 07 10:08:02.688140 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.688107 2575 manager.go:1196] Started watching for new ooms in manager Sep 07 10:08:02.688563 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.688545 2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 07 10:08:02.688656 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.688548 2575 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 07 10:08:02.688732 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.688723 2575 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 07 10:08:02.688952 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.688937 2575 reconstruct.go:97] "Volume reconstruction finished" Sep 07 10:08:02.688952 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.688950 2575 reconciler.go:29] "Reconciler: start to sync state" Sep 07 10:08:02.689474 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.689458 2575 manager.go:349] Starting recovery of all containers Sep 07 10:08:02.690844 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:02.689005 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-85.ec2.internal.18d302018cb8e294 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-85.ec2.internal,UID:ip-10-0-141-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-85.ec2.internal,},FirstTimestamp:2026-09-07 10:08:02.675368596 +0000 UTC m=+0.387610531,LastTimestamp:2026-09-07 10:08:02.675368596 +0000 UTC m=+0.387610531,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-85.ec2.internal,}" Sep 07 10:08:02.700057 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.700034 2575 manager.go:354] Recovery completed Sep 07 10:08:02.702684 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.702627 2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 10:08:02.703318 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:02.703220 2575 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Sep 07 10:08:02.706502 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:02.706483 2575 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-85.ec2.internal\" not found" node="ip-10-0-141-85.ec2.internal" Sep 07 10:08:02.707433 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.707419 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 10:08:02.709335 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.709317 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 10:08:02.709422 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.709350 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 10:08:02.709422 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.709360 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 10:08:02.709978 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.709965 2575 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 07 10:08:02.709978 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.709972 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 10:08:02.710083 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.709975 2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 07 10:08:02.710083 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.710015 2575 state_mem.go:36] "Initialized new in-memory state store" Sep 07 10:08:02.710684 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.710670 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 10:08:02.710746 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.710701 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 10:08:02.710746 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.710716 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 10:08:02.712235 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.712222 2575 policy_none.go:49] "None policy: Start" Sep 07 10:08:02.712283 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.712239 2575 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 07 10:08:02.712283 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.712250 2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 07 10:08:02.714225 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.714212 2575 policy_none.go:47] "Start" Sep 07 10:08:02.759641 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.759614 2575 manager.go:337] "Starting Device Plugin manager" Sep 07 10:08:02.769147 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:02.759676 2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 07 10:08:02.769147 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.759691 2575 server.go:87] "Starting device plugin registration server" Sep 07 10:08:02.769147 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.760133 2575 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 07 10:08:02.769147 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.760151 2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 07 10:08:02.769147 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.760265 2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 07 10:08:02.769147 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.760374 2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 07 10:08:02.769147 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.760382 2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 07 10:08:02.769147 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:02.760915 2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 07 10:08:02.769147 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:02.760965 2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-85.ec2.internal\" not found" Sep 07 10:08:02.823787 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.823737 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 07 10:08:02.824999 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.824981 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 07 10:08:02.825054 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.825037 2575 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 07 10:08:02.825088 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.825060 2575 kubelet.go:2448] "Starting kubelet main sync loop" Sep 07 10:08:02.825131 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:02.825103 2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 07 10:08:02.827377 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.827351 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 10:08:02.860602 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.860513 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 10:08:02.862209 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.862188 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 10:08:02.862286 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.862223 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 10:08:02.862286 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.862242 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 10:08:02.862286 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.862270 2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-85.ec2.internal" Sep 07 10:08:02.870076 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.870058 2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-85.ec2.internal" Sep 07 10:08:02.870134 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:02.870085 2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-85.ec2.internal\": node \"ip-10-0-141-85.ec2.internal\" not found" Sep 07 10:08:02.885497 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:02.885468 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-85.ec2.internal\" not found" Sep 07 10:08:02.926225 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.926193 2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal"] Sep 07 10:08:02.926352 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.926273 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 10:08:02.928809 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.928777 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 10:08:02.928921 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.928824 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 10:08:02.928921 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.928837 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 10:08:02.931142 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.931129 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 10:08:02.931259 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.931244 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal" Sep 07 10:08:02.931316 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.931281 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 10:08:02.933370 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.933349 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 10:08:02.933370 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.933374 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 10:08:02.933523 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.933383 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 10:08:02.933523 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.933351 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 10:08:02.933523 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.933417 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 10:08:02.933523 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.933445 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 10:08:02.937649 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.937631 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal" Sep 07 10:08:02.937741 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.937672 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 10:08:02.939020 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.939001 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientMemory" Sep 07 10:08:02.939106 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.939030 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 10:08:02.939106 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:02.939043 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeHasSufficientPID" Sep 07 10:08:02.955442 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:02.955419 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-85.ec2.internal\" not found" node="ip-10-0-141-85.ec2.internal" Sep 07 10:08:02.961255 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:02.961235 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-85.ec2.internal\" not found" node="ip-10-0-141-85.ec2.internal" Sep 07 10:08:02.986432 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:02.986404 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-85.ec2.internal\" not found" Sep 07 10:08:03.086659 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:03.086617 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-85.ec2.internal\" not found" Sep 07 10:08:03.090248 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.090228 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f155589bfc7e2b2766af1b57a8ea8129-config\") pod \"kube-apiserver-proxy-ip-10-0-141-85.ec2.internal\" (UID: \"f155589bfc7e2b2766af1b57a8ea8129\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal" Sep 07 10:08:03.090334 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.090263 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/18850670ff94925fc45b643b7c67aabb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal\" (UID: \"18850670ff94925fc45b643b7c67aabb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal" Sep 07 10:08:03.090334 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.090283 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18850670ff94925fc45b643b7c67aabb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal\" (UID: \"18850670ff94925fc45b643b7c67aabb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal" Sep 07 10:08:03.188057 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:03.187973 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-85.ec2.internal\" not found" Sep 07 10:08:03.190391 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.190363 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f155589bfc7e2b2766af1b57a8ea8129-config\") pod \"kube-apiserver-proxy-ip-10-0-141-85.ec2.internal\" (UID: \"f155589bfc7e2b2766af1b57a8ea8129\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal" Sep 07 10:08:03.190478 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.190413 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/18850670ff94925fc45b643b7c67aabb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal\" (UID: \"18850670ff94925fc45b643b7c67aabb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal" Sep 07 10:08:03.190478 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.190425 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f155589bfc7e2b2766af1b57a8ea8129-config\") pod \"kube-apiserver-proxy-ip-10-0-141-85.ec2.internal\" (UID: \"f155589bfc7e2b2766af1b57a8ea8129\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal" Sep 07 10:08:03.190478 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.190442 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18850670ff94925fc45b643b7c67aabb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal\" (UID: \"18850670ff94925fc45b643b7c67aabb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal" Sep 07 10:08:03.190600 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.190481 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/18850670ff94925fc45b643b7c67aabb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal\" (UID: \"18850670ff94925fc45b643b7c67aabb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal" Sep 07 10:08:03.190600 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.190481 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18850670ff94925fc45b643b7c67aabb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal\" (UID: \"18850670ff94925fc45b643b7c67aabb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal" Sep 07 10:08:03.263122 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.263078 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal" Sep 07 10:08:03.264240 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.264211 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal" Sep 07 10:08:03.288985 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:03.288951 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-85.ec2.internal\" not found" Sep 07 10:08:03.389389 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:03.389329 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-85.ec2.internal\" not found" Sep 07 10:08:03.490369 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:03.490287 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-85.ec2.internal\" not found" Sep 07 10:08:03.591044 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:03.590999 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-85.ec2.internal\" not found" Sep 07 10:08:03.618735 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.618709 2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 07 10:08:03.619204 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.618908 2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 07 10:08:03.619204 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.618908 2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 07 10:08:03.687781 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.687735 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-06 10:03:03 +0000 UTC" deadline="2028-02-28 18:18:05.27756441 +0000 UTC" Sep 07 10:08:03.687781 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.687767 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12944h10m1.58980016s" Sep 07 10:08:03.691749 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:03.691728 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-85.ec2.internal\" not found" Sep 07 10:08:03.744236 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.744144 2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 10:08:03.761344 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.761319 2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 10:08:03.788224 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.788196 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal" Sep 07 10:08:03.800474 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.800449 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 07 10:08:03.801686 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.801669 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal" Sep 07 10:08:03.808590 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.808574 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 07 10:08:03.976883 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:03.976841 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf155589bfc7e2b2766af1b57a8ea8129.slice/crio-f70b0c8da5f30ba755b0c90ba4a871b7bdb3ab2db6d123bf09df7a845b5a99c7 WatchSource:0}: Error finding container f70b0c8da5f30ba755b0c90ba4a871b7bdb3ab2db6d123bf09df7a845b5a99c7: Status 404 returned error can't find the container with id f70b0c8da5f30ba755b0c90ba4a871b7bdb3ab2db6d123bf09df7a845b5a99c7 Sep 07 10:08:03.977480 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:03.977449 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18850670ff94925fc45b643b7c67aabb.slice/crio-90d38706e1acbf7bbc09a2565d596b817c651835acab8a7feb315717aa667d5a WatchSource:0}: Error finding container 90d38706e1acbf7bbc09a2565d596b817c651835acab8a7feb315717aa667d5a: Status 404 returned error can't find the container with id 90d38706e1acbf7bbc09a2565d596b817c651835acab8a7feb315717aa667d5a Sep 07 10:08:03.983026 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:03.983010 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 10:08:04.471605 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.471566 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 10:08:04.536601 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.536564 2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 10:08:04.664021 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.663980 2575 apiserver.go:52] "Watching apiserver" Sep 07 10:08:04.672803 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.672386 2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 07 10:08:04.673535 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.673501 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal","openshift-multus/network-metrics-daemon-4wfhx","openshift-ovn-kubernetes/ovnkube-node-f6mnj","kube-system/global-pull-secret-syncer-6v64l","kube-system/konnectivity-agent-9tdjb","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh","openshift-multus/multus-6q6w4","openshift-multus/multus-additional-cni-plugins-nzg2l","openshift-network-diagnostics/network-check-target-km29h","openshift-network-operator/iptables-alerter-qss98","kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal","openshift-cluster-node-tuning-operator/tuned-4wg78","openshift-dns/node-resolver-w5w2p","openshift-image-registry/node-ca-t6ztt"] Sep 07 10:08:04.676633 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.676605 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.680290 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.680266 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 07 10:08:04.680427 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.680393 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 07 10:08:04.680499 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.680482 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kwkch\"" Sep 07 10:08:04.680657 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.680641 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 07 10:08:04.680749 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.680728 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 07 10:08:04.685463 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.685439 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w5w2p" Sep 07 10:08:04.686888 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.686853 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 07 10:08:04.687865 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.687844 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-slbff\"" Sep 07 10:08:04.688254 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.688207 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 07 10:08:04.688254 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.688238 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 07 10:08:04.690535 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.689564 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.691823 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.691800 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 07 10:08:04.692247 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.692227 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8k9jq\"" Sep 07 10:08:04.692412 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.692390 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 07 10:08:04.692485 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.692407 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:04.692485 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.692475 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 07 10:08:04.692593 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:04.692476 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6v64l" podUID="cff88048-74be-46c2-a8ac-113c5fc3a85d" Sep 07 10:08:04.693252 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.693233 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 07 10:08:04.693464 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.693448 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 07 10:08:04.693848 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.693831 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 07 10:08:04.694970 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.694953 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9tdjb" Sep 07 10:08:04.697079 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.697058 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 07 10:08:04.697327 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.697309 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 07 10:08:04.697669 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.697651 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wlxjf\"" Sep 07 10:08:04.697915 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.697870 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.699690 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.699669 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:04.699797 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:04.699740 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km29h" podUID="d4e7d380-df30-4ee5-9059-6fd7fb6e82b7" Sep 07 10:08:04.700961 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.699988 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-multus-socket-dir-parent\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.700961 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700028 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-host-run-netns\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.700961 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700054 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-run-openvswitch\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.700961 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700080 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-ovnkube-config\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.700961 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700106 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-system-cni-dir\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.700961 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700130 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-cnibin\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.700961 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700157 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-etc-kubernetes\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.700961 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700182 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-run-systemd\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.700961 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700208 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-log-socket\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.700961 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700231 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-ovn-node-metrics-cert\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.700961 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700260 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-ovnkube-script-lib\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.700961 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700283 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-multus-cni-dir\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.700961 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700307 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-host-var-lib-kubelet\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.700961 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700331 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-hostroot\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.700961 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700366 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-multus-conf-dir\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.700961 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700391 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-host-run-multus-certs\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.700961 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700417 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxsdl\" (UniqueName: \"kubernetes.io/projected/19524835-4647-4786-bc0f-c222dd09c075-kube-api-access-vxsdl\") pod \"node-resolver-w5w2p\" (UID: \"19524835-4647-4786-bc0f-c222dd09c075\") " pod="openshift-dns/node-resolver-w5w2p" Sep 07 10:08:04.701821 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700443 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-node-log\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.701821 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700469 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-host-run-ovn-kubernetes\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.701821 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700494 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-host-var-lib-cni-bin\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.701821 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700518 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19524835-4647-4786-bc0f-c222dd09c075-tmp-dir\") pod \"node-resolver-w5w2p\" (UID: \"19524835-4647-4786-bc0f-c222dd09c075\") " pod="openshift-dns/node-resolver-w5w2p" Sep 07 10:08:04.701821 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700542 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-host-kubelet\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.701821 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700566 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-host-cni-bin\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.701821 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700589 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-host-cni-netd\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.701821 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700619 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.701821 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700651 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cff88048-74be-46c2-a8ac-113c5fc3a85d-dbus\") pod \"global-pull-secret-syncer-6v64l\" (UID: \"cff88048-74be-46c2-a8ac-113c5fc3a85d\") " pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:04.701821 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700681 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-host-var-lib-cni-multus\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.701821 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700705 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/19524835-4647-4786-bc0f-c222dd09c075-hosts-file\") pod \"node-resolver-w5w2p\" (UID: \"19524835-4647-4786-bc0f-c222dd09c075\") " pod="openshift-dns/node-resolver-w5w2p" Sep 07 10:08:04.701821 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700747 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-systemd-units\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.701821 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700773 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz7sg\" (UniqueName: \"kubernetes.io/projected/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-kube-api-access-hz7sg\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.701821 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700824 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cff88048-74be-46c2-a8ac-113c5fc3a85d-kubelet-config\") pod \"global-pull-secret-syncer-6v64l\" (UID: \"cff88048-74be-46c2-a8ac-113c5fc3a85d\") " pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:04.701821 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700855 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/448a5179-febc-460a-ab61-f7810c8bf09b-cni-binary-copy\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.701821 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700879 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-host-run-k8s-cni-cncf-io\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.702694 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700905 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-etc-openvswitch\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.702694 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700933 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-env-overrides\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.702694 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700957 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret\") pod \"global-pull-secret-syncer-6v64l\" (UID: \"cff88048-74be-46c2-a8ac-113c5fc3a85d\") " pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:04.702694 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.700987 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/448a5179-febc-460a-ab61-f7810c8bf09b-multus-daemon-config\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.702694 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.701011 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-run-ovn\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.702694 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.701043 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp4gj\" (UniqueName: \"kubernetes.io/projected/448a5179-febc-460a-ab61-f7810c8bf09b-kube-api-access-gp4gj\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.702694 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.701067 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-host-slash\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.702694 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.701094 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-host-run-netns\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.702694 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.701120 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-var-lib-openvswitch\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.702694 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.701687 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 07 10:08:04.702694 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.701705 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 07 10:08:04.702694 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.701861 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 07 10:08:04.702694 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.702068 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8gkd5\"" Sep 07 10:08:04.702694 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.702695 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t6ztt" Sep 07 10:08:04.704770 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.704746 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 07 10:08:04.705176 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.705158 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:04.705264 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:04.705246 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wfhx" podUID="d11a5179-d3c5-48d5-a7c6-a7c367fd2584" Sep 07 10:08:04.705711 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.705689 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-v244m\"" Sep 07 10:08:04.706431 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.706413 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 07 10:08:04.706673 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.706657 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 07 10:08:04.707076 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.707059 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 07 10:08:04.707932 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.707915 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qss98" Sep 07 10:08:04.710270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.710246 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 07 10:08:04.710543 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.710528 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6b5mn\"" Sep 07 10:08:04.710917 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.710864 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 07 10:08:04.711485 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.711439 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 07 10:08:04.711880 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.711860 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.717548 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.717519 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 07 10:08:04.717656 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.717597 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.717928 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.717896 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bzdrq\"" Sep 07 10:08:04.718192 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.718169 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 07 10:08:04.720597 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.720579 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 07 10:08:04.720774 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.720754 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 07 10:08:04.720874 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.720828 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xjmtb\"" Sep 07 10:08:04.731461 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.731435 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-spgtb" Sep 07 10:08:04.742270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.742144 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-spgtb" Sep 07 10:08:04.789662 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.789636 2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 07 10:08:04.802233 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802204 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-host-run-k8s-cni-cncf-io\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.802400 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802239 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-etc-openvswitch\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.802400 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802269 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret\") pod \"global-pull-secret-syncer-6v64l\" (UID: \"cff88048-74be-46c2-a8ac-113c5fc3a85d\") " pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:04.802400 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802337 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-host-run-k8s-cni-cncf-io\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.802400 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802380 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-host\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.802400 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:04.802389 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:04.802717 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802413 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6vgv\" (UniqueName: \"kubernetes.io/projected/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-kube-api-access-g6vgv\") pod \"network-metrics-daemon-4wfhx\" (UID: \"d11a5179-d3c5-48d5-a7c6-a7c367fd2584\") " pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:04.802717 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802409 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-etc-openvswitch\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.802717 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802439 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a6a30db9-93e6-41b1-a0e6-28e5088df925-host\") pod \"node-ca-t6ztt\" (UID: \"a6a30db9-93e6-41b1-a0e6-28e5088df925\") " pod="openshift-image-registry/node-ca-t6ztt" Sep 07 10:08:04.802717 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:04.802463 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret podName:cff88048-74be-46c2-a8ac-113c5fc3a85d nodeName:}" failed. No retries permitted until 2026-09-07 10:08:05.302438427 +0000 UTC m=+3.014680367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret") pod "global-pull-secret-syncer-6v64l" (UID: "cff88048-74be-46c2-a8ac-113c5fc3a85d") : object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:04.802717 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802496 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/adcc0041-9a3b-4ff3-8a9a-25741c8e9baa-host-slash\") pod \"iptables-alerter-qss98\" (UID: \"adcc0041-9a3b-4ff3-8a9a-25741c8e9baa\") " pod="openshift-network-operator/iptables-alerter-qss98" Sep 07 10:08:04.802717 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802527 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/448a5179-febc-460a-ab61-f7810c8bf09b-multus-daemon-config\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.802717 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802554 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9aeb0af4-1773-4b88-b17f-d1d4d6490101-cnibin\") pod \"multus-additional-cni-plugins-nzg2l\" (UID: \"9aeb0af4-1773-4b88-b17f-d1d4d6490101\") " pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.802717 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802578 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-run\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.802717 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802603 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/18c4a942-f1db-473e-aab6-d35c7bde40ae-device-dir\") pod \"aws-ebs-csi-driver-node-fr4jh\" (UID: \"18c4a942-f1db-473e-aab6-d35c7bde40ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.803199 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802740 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs\") pod \"network-metrics-daemon-4wfhx\" (UID: \"d11a5179-d3c5-48d5-a7c6-a7c367fd2584\") " pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:04.803199 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802769 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp5m8\" (UniqueName: \"kubernetes.io/projected/adcc0041-9a3b-4ff3-8a9a-25741c8e9baa-kube-api-access-hp5m8\") pod \"iptables-alerter-qss98\" (UID: \"adcc0041-9a3b-4ff3-8a9a-25741c8e9baa\") " pod="openshift-network-operator/iptables-alerter-qss98" Sep 07 10:08:04.803199 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802813 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-host-slash\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.803199 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802839 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-var-lib-openvswitch\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.803199 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802867 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9aeb0af4-1773-4b88-b17f-d1d4d6490101-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nzg2l\" (UID: \"9aeb0af4-1773-4b88-b17f-d1d4d6490101\") " pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.803199 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802892 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k94pz\" (UniqueName: \"kubernetes.io/projected/a6a30db9-93e6-41b1-a0e6-28e5088df925-kube-api-access-k94pz\") pod \"node-ca-t6ztt\" (UID: \"a6a30db9-93e6-41b1-a0e6-28e5088df925\") " pod="openshift-image-registry/node-ca-t6ztt" Sep 07 10:08:04.803199 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802918 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-host-run-netns\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.803199 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802944 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-run-openvswitch\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.803199 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802971 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-ovnkube-config\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.803199 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.802996 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a6a30db9-93e6-41b1-a0e6-28e5088df925-serviceca\") pod \"node-ca-t6ztt\" (UID: \"a6a30db9-93e6-41b1-a0e6-28e5088df925\") " pod="openshift-image-registry/node-ca-t6ztt" Sep 07 10:08:04.803199 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803023 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-cnibin\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.803199 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803048 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-etc-kubernetes\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.803199 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803072 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-log-socket\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.803199 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803097 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-ovnkube-script-lib\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.803199 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803122 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9aeb0af4-1773-4b88-b17f-d1d4d6490101-system-cni-dir\") pod \"multus-additional-cni-plugins-nzg2l\" (UID: \"9aeb0af4-1773-4b88-b17f-d1d4d6490101\") " pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.803199 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803147 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-var-lib-kubelet\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.803199 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803174 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-multus-cni-dir\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.804115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803198 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-hostroot\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.804115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803223 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-multus-conf-dir\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.804115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803245 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/448a5179-febc-460a-ab61-f7810c8bf09b-multus-daemon-config\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.804115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803267 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-host-run-multus-certs\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.804115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803297 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-node-log\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.804115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803323 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-host-run-ovn-kubernetes\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.804115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803349 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-host-cni-bin\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.804115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803460 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-host-slash\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.804115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803469 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-multus-cni-dir\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.804115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803501 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-var-lib-openvswitch\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.804115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803520 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-etc-kubernetes\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.804115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803540 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-hostroot\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.804115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803559 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9aeb0af4-1773-4b88-b17f-d1d4d6490101-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nzg2l\" (UID: \"9aeb0af4-1773-4b88-b17f-d1d4d6490101\") " pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.804115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803576 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-multus-conf-dir\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.804115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803595 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19524835-4647-4786-bc0f-c222dd09c075-tmp-dir\") pod \"node-resolver-w5w2p\" (UID: \"19524835-4647-4786-bc0f-c222dd09c075\") " pod="openshift-dns/node-resolver-w5w2p" Sep 07 10:08:04.804115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803622 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-host-kubelet\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.804115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803653 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.804115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803684 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-etc-kubernetes\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.805054 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803711 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb028709-a2f7-4338-8699-120a4879ffaf-konnectivity-ca\") pod \"konnectivity-agent-9tdjb\" (UID: \"cb028709-a2f7-4338-8699-120a4879ffaf\") " pod="kube-system/konnectivity-agent-9tdjb" Sep 07 10:08:04.805054 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803735 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/18c4a942-f1db-473e-aab6-d35c7bde40ae-sys-fs\") pod \"aws-ebs-csi-driver-node-fr4jh\" (UID: \"18c4a942-f1db-473e-aab6-d35c7bde40ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.805054 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803763 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/19524835-4647-4786-bc0f-c222dd09c075-hosts-file\") pod \"node-resolver-w5w2p\" (UID: \"19524835-4647-4786-bc0f-c222dd09c075\") " pod="openshift-dns/node-resolver-w5w2p" Sep 07 10:08:04.805054 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803793 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cff88048-74be-46c2-a8ac-113c5fc3a85d-kubelet-config\") pod \"global-pull-secret-syncer-6v64l\" (UID: \"cff88048-74be-46c2-a8ac-113c5fc3a85d\") " pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:04.805054 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803819 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9zzn\" (UniqueName: \"kubernetes.io/projected/9aeb0af4-1773-4b88-b17f-d1d4d6490101-kube-api-access-f9zzn\") pod \"multus-additional-cni-plugins-nzg2l\" (UID: \"9aeb0af4-1773-4b88-b17f-d1d4d6490101\") " pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.805054 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803625 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-log-socket\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.805054 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803656 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-host-run-multus-certs\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.805054 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803938 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-etc-sysconfig\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.805054 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803965 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-sys\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.805054 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.803989 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-lib-modules\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.805054 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804024 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/448a5179-febc-460a-ab61-f7810c8bf09b-cni-binary-copy\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.805054 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804054 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-env-overrides\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.805054 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804080 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/18c4a942-f1db-473e-aab6-d35c7bde40ae-registration-dir\") pod \"aws-ebs-csi-driver-node-fr4jh\" (UID: \"18c4a942-f1db-473e-aab6-d35c7bde40ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.805054 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804107 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-run-ovn\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.805054 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804133 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9aeb0af4-1773-4b88-b17f-d1d4d6490101-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nzg2l\" (UID: \"9aeb0af4-1773-4b88-b17f-d1d4d6490101\") " pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.805054 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804163 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-etc-modprobe-d\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.805054 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804167 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.805990 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804193 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-host-run-ovn-kubernetes\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.805990 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804206 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb028709-a2f7-4338-8699-120a4879ffaf-agent-certs\") pod \"konnectivity-agent-9tdjb\" (UID: \"cb028709-a2f7-4338-8699-120a4879ffaf\") " pod="kube-system/konnectivity-agent-9tdjb" Sep 07 10:08:04.805990 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804193 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-node-log\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.805990 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804221 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-host-run-netns\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.805990 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804237 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpl4s\" (UniqueName: \"kubernetes.io/projected/18c4a942-f1db-473e-aab6-d35c7bde40ae-kube-api-access-tpl4s\") pod \"aws-ebs-csi-driver-node-fr4jh\" (UID: \"18c4a942-f1db-473e-aab6-d35c7bde40ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.805990 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804313 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-host-kubelet\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.805990 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804330 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/19524835-4647-4786-bc0f-c222dd09c075-hosts-file\") pod \"node-resolver-w5w2p\" (UID: \"19524835-4647-4786-bc0f-c222dd09c075\") " pod="openshift-dns/node-resolver-w5w2p" Sep 07 10:08:04.805990 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804338 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-ovnkube-script-lib\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.805990 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804351 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-run-openvswitch\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.805990 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804375 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cff88048-74be-46c2-a8ac-113c5fc3a85d-kubelet-config\") pod \"global-pull-secret-syncer-6v64l\" (UID: \"cff88048-74be-46c2-a8ac-113c5fc3a85d\") " pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:04.805990 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804390 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-host-cni-bin\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.805990 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804434 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-run-ovn\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.805990 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804439 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-cnibin\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.805990 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804484 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp4gj\" (UniqueName: \"kubernetes.io/projected/448a5179-febc-460a-ab61-f7810c8bf09b-kube-api-access-gp4gj\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.805990 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804512 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-host-run-netns\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.805990 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804518 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19524835-4647-4786-bc0f-c222dd09c075-tmp-dir\") pod \"node-resolver-w5w2p\" (UID: \"19524835-4647-4786-bc0f-c222dd09c075\") " pod="openshift-dns/node-resolver-w5w2p" Sep 07 10:08:04.805990 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804574 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-multus-socket-dir-parent\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.805990 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804601 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-host-run-netns\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.806935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804653 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a1a1ee72-afe3-4901-8a66-95c456e21ada-etc-tuned\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.806935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804687 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-system-cni-dir\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.806935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804704 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-multus-socket-dir-parent\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.806935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804729 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-run-systemd\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.806935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804735 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-ovnkube-config\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.806935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804779 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-run-systemd\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.806935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804804 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-ovn-node-metrics-cert\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.806935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804853 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/448a5179-febc-460a-ab61-f7810c8bf09b-cni-binary-copy\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.806935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804862 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-etc-sysctl-conf\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.806935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804878 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-system-cni-dir\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.806935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.804989 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/18c4a942-f1db-473e-aab6-d35c7bde40ae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fr4jh\" (UID: \"18c4a942-f1db-473e-aab6-d35c7bde40ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.806935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805032 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2s2g\" (UniqueName: \"kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g\") pod \"network-check-target-km29h\" (UID: \"d4e7d380-df30-4ee5-9059-6fd7fb6e82b7\") " pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:04.806935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805054 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-host-var-lib-kubelet\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.806935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805080 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxsdl\" (UniqueName: \"kubernetes.io/projected/19524835-4647-4786-bc0f-c222dd09c075-kube-api-access-vxsdl\") pod \"node-resolver-w5w2p\" (UID: \"19524835-4647-4786-bc0f-c222dd09c075\") " pod="openshift-dns/node-resolver-w5w2p" Sep 07 10:08:04.806935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805105 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-etc-sysctl-d\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.806935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805122 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1a1ee72-afe3-4901-8a66-95c456e21ada-tmp\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.806935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805126 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-env-overrides\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.807778 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805139 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/adcc0041-9a3b-4ff3-8a9a-25741c8e9baa-iptables-alerter-script\") pod \"iptables-alerter-qss98\" (UID: \"adcc0041-9a3b-4ff3-8a9a-25741c8e9baa\") " pod="openshift-network-operator/iptables-alerter-qss98" Sep 07 10:08:04.807778 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805171 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-host-var-lib-cni-bin\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.807778 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805195 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-host-cni-netd\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.807778 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805198 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-host-var-lib-kubelet\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.807778 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805211 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cff88048-74be-46c2-a8ac-113c5fc3a85d-dbus\") pod \"global-pull-secret-syncer-6v64l\" (UID: \"cff88048-74be-46c2-a8ac-113c5fc3a85d\") " pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:04.807778 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805250 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cff88048-74be-46c2-a8ac-113c5fc3a85d-dbus\") pod \"global-pull-secret-syncer-6v64l\" (UID: \"cff88048-74be-46c2-a8ac-113c5fc3a85d\") " pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:04.807778 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805280 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-host-var-lib-cni-bin\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.807778 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805309 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-host-cni-netd\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.807778 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805333 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-host-var-lib-cni-multus\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.807778 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805361 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-systemd-units\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.807778 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805382 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz7sg\" (UniqueName: \"kubernetes.io/projected/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-kube-api-access-hz7sg\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.807778 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805434 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/448a5179-febc-460a-ab61-f7810c8bf09b-host-var-lib-cni-multus\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.807778 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805445 2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 07 10:08:04.807778 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805487 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9aeb0af4-1773-4b88-b17f-d1d4d6490101-cni-binary-copy\") pod \"multus-additional-cni-plugins-nzg2l\" (UID: \"9aeb0af4-1773-4b88-b17f-d1d4d6490101\") " pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.807778 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805519 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-systemd-units\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.807778 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805574 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-etc-systemd\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.807778 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805623 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qmzv\" (UniqueName: \"kubernetes.io/projected/a1a1ee72-afe3-4901-8a66-95c456e21ada-kube-api-access-8qmzv\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.808721 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805664 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/18c4a942-f1db-473e-aab6-d35c7bde40ae-socket-dir\") pod \"aws-ebs-csi-driver-node-fr4jh\" (UID: \"18c4a942-f1db-473e-aab6-d35c7bde40ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.808721 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.805703 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/18c4a942-f1db-473e-aab6-d35c7bde40ae-etc-selinux\") pod \"aws-ebs-csi-driver-node-fr4jh\" (UID: \"18c4a942-f1db-473e-aab6-d35c7bde40ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.812981 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.811926 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-ovn-node-metrics-cert\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.816845 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.816624 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp4gj\" (UniqueName: \"kubernetes.io/projected/448a5179-febc-460a-ab61-f7810c8bf09b-kube-api-access-gp4gj\") pod \"multus-6q6w4\" (UID: \"448a5179-febc-460a-ab61-f7810c8bf09b\") " pod="openshift-multus/multus-6q6w4" Sep 07 10:08:04.817275 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.817231 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz7sg\" (UniqueName: \"kubernetes.io/projected/f245ed6c-716b-4e7f-89bf-2db47ba2e83c-kube-api-access-hz7sg\") pod \"ovnkube-node-f6mnj\" (UID: \"f245ed6c-716b-4e7f-89bf-2db47ba2e83c\") " pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:04.819321 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.819295 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxsdl\" (UniqueName: \"kubernetes.io/projected/19524835-4647-4786-bc0f-c222dd09c075-kube-api-access-vxsdl\") pod \"node-resolver-w5w2p\" (UID: \"19524835-4647-4786-bc0f-c222dd09c075\") " pod="openshift-dns/node-resolver-w5w2p" Sep 07 10:08:04.831338 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.831208 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal" event={"ID":"18850670ff94925fc45b643b7c67aabb","Type":"ContainerStarted","Data":"90d38706e1acbf7bbc09a2565d596b817c651835acab8a7feb315717aa667d5a"} Sep 07 10:08:04.833248 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.833219 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal" event={"ID":"f155589bfc7e2b2766af1b57a8ea8129","Type":"ContainerStarted","Data":"f70b0c8da5f30ba755b0c90ba4a871b7bdb3ab2db6d123bf09df7a845b5a99c7"} Sep 07 10:08:04.906824 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.906778 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-etc-sysctl-conf\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.907043 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.906842 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/18c4a942-f1db-473e-aab6-d35c7bde40ae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fr4jh\" (UID: \"18c4a942-f1db-473e-aab6-d35c7bde40ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.907043 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.906882 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2s2g\" (UniqueName: \"kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g\") pod \"network-check-target-km29h\" (UID: \"d4e7d380-df30-4ee5-9059-6fd7fb6e82b7\") " pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:04.907043 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.906928 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/18c4a942-f1db-473e-aab6-d35c7bde40ae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fr4jh\" (UID: \"18c4a942-f1db-473e-aab6-d35c7bde40ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.907043 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.906918 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-etc-sysctl-d\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.907043 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.906973 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1a1ee72-afe3-4901-8a66-95c456e21ada-tmp\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.907043 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.906997 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/adcc0041-9a3b-4ff3-8a9a-25741c8e9baa-iptables-alerter-script\") pod \"iptables-alerter-qss98\" (UID: \"adcc0041-9a3b-4ff3-8a9a-25741c8e9baa\") " pod="openshift-network-operator/iptables-alerter-qss98" Sep 07 10:08:04.907043 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.907033 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9aeb0af4-1773-4b88-b17f-d1d4d6490101-cni-binary-copy\") pod \"multus-additional-cni-plugins-nzg2l\" (UID: \"9aeb0af4-1773-4b88-b17f-d1d4d6490101\") " pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.907414 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.907055 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-etc-systemd\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.907414 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.907079 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qmzv\" (UniqueName: \"kubernetes.io/projected/a1a1ee72-afe3-4901-8a66-95c456e21ada-kube-api-access-8qmzv\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.907414 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.907103 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/18c4a942-f1db-473e-aab6-d35c7bde40ae-socket-dir\") pod \"aws-ebs-csi-driver-node-fr4jh\" (UID: \"18c4a942-f1db-473e-aab6-d35c7bde40ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.907414 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.907126 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/18c4a942-f1db-473e-aab6-d35c7bde40ae-etc-selinux\") pod \"aws-ebs-csi-driver-node-fr4jh\" (UID: \"18c4a942-f1db-473e-aab6-d35c7bde40ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.907414 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.907163 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-host\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.907414 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.907175 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-etc-sysctl-d\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.907414 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.907189 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6vgv\" (UniqueName: \"kubernetes.io/projected/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-kube-api-access-g6vgv\") pod \"network-metrics-daemon-4wfhx\" (UID: \"d11a5179-d3c5-48d5-a7c6-a7c367fd2584\") " pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:04.907414 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.907215 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a6a30db9-93e6-41b1-a0e6-28e5088df925-host\") pod \"node-ca-t6ztt\" (UID: \"a6a30db9-93e6-41b1-a0e6-28e5088df925\") " pod="openshift-image-registry/node-ca-t6ztt" Sep 07 10:08:04.907414 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.907237 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-etc-systemd\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.907414 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.907255 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/18c4a942-f1db-473e-aab6-d35c7bde40ae-etc-selinux\") pod \"aws-ebs-csi-driver-node-fr4jh\" (UID: \"18c4a942-f1db-473e-aab6-d35c7bde40ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.907414 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.907283 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-host\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.907414 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.907283 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/adcc0041-9a3b-4ff3-8a9a-25741c8e9baa-host-slash\") pod \"iptables-alerter-qss98\" (UID: \"adcc0041-9a3b-4ff3-8a9a-25741c8e9baa\") " pod="openshift-network-operator/iptables-alerter-qss98" Sep 07 10:08:04.907414 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.907408 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-etc-sysctl-conf\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.908107 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.907441 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a6a30db9-93e6-41b1-a0e6-28e5088df925-host\") pod \"node-ca-t6ztt\" (UID: \"a6a30db9-93e6-41b1-a0e6-28e5088df925\") " pod="openshift-image-registry/node-ca-t6ztt" Sep 07 10:08:04.908107 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.907637 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/18c4a942-f1db-473e-aab6-d35c7bde40ae-socket-dir\") pod \"aws-ebs-csi-driver-node-fr4jh\" (UID: \"18c4a942-f1db-473e-aab6-d35c7bde40ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.908107 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908058 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9aeb0af4-1773-4b88-b17f-d1d4d6490101-cni-binary-copy\") pod \"multus-additional-cni-plugins-nzg2l\" (UID: \"9aeb0af4-1773-4b88-b17f-d1d4d6490101\") " pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.908107 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.907239 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/adcc0041-9a3b-4ff3-8a9a-25741c8e9baa-host-slash\") pod \"iptables-alerter-qss98\" (UID: \"adcc0041-9a3b-4ff3-8a9a-25741c8e9baa\") " pod="openshift-network-operator/iptables-alerter-qss98" Sep 07 10:08:04.908317 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908118 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9aeb0af4-1773-4b88-b17f-d1d4d6490101-cnibin\") pod \"multus-additional-cni-plugins-nzg2l\" (UID: \"9aeb0af4-1773-4b88-b17f-d1d4d6490101\") " pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.908317 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908145 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-run\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.908317 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908167 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/18c4a942-f1db-473e-aab6-d35c7bde40ae-device-dir\") pod \"aws-ebs-csi-driver-node-fr4jh\" (UID: \"18c4a942-f1db-473e-aab6-d35c7bde40ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.908317 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908190 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs\") pod \"network-metrics-daemon-4wfhx\" (UID: \"d11a5179-d3c5-48d5-a7c6-a7c367fd2584\") " pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:04.908317 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908215 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp5m8\" (UniqueName: \"kubernetes.io/projected/adcc0041-9a3b-4ff3-8a9a-25741c8e9baa-kube-api-access-hp5m8\") pod \"iptables-alerter-qss98\" (UID: \"adcc0041-9a3b-4ff3-8a9a-25741c8e9baa\") " pod="openshift-network-operator/iptables-alerter-qss98" Sep 07 10:08:04.908317 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908225 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/adcc0041-9a3b-4ff3-8a9a-25741c8e9baa-iptables-alerter-script\") pod \"iptables-alerter-qss98\" (UID: \"adcc0041-9a3b-4ff3-8a9a-25741c8e9baa\") " pod="openshift-network-operator/iptables-alerter-qss98" Sep 07 10:08:04.908317 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908246 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9aeb0af4-1773-4b88-b17f-d1d4d6490101-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nzg2l\" (UID: \"9aeb0af4-1773-4b88-b17f-d1d4d6490101\") " pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.908317 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908273 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k94pz\" (UniqueName: \"kubernetes.io/projected/a6a30db9-93e6-41b1-a0e6-28e5088df925-kube-api-access-k94pz\") pod \"node-ca-t6ztt\" (UID: \"a6a30db9-93e6-41b1-a0e6-28e5088df925\") " pod="openshift-image-registry/node-ca-t6ztt" Sep 07 10:08:04.908317 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908296 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9aeb0af4-1773-4b88-b17f-d1d4d6490101-cnibin\") pod \"multus-additional-cni-plugins-nzg2l\" (UID: \"9aeb0af4-1773-4b88-b17f-d1d4d6490101\") " pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.908317 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908301 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a6a30db9-93e6-41b1-a0e6-28e5088df925-serviceca\") pod \"node-ca-t6ztt\" (UID: \"a6a30db9-93e6-41b1-a0e6-28e5088df925\") " pod="openshift-image-registry/node-ca-t6ztt" Sep 07 10:08:04.908860 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908346 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9aeb0af4-1773-4b88-b17f-d1d4d6490101-system-cni-dir\") pod \"multus-additional-cni-plugins-nzg2l\" (UID: \"9aeb0af4-1773-4b88-b17f-d1d4d6490101\") " pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.908860 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908375 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-var-lib-kubelet\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.908860 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908412 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9aeb0af4-1773-4b88-b17f-d1d4d6490101-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nzg2l\" (UID: \"9aeb0af4-1773-4b88-b17f-d1d4d6490101\") " pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.908860 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908446 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-etc-kubernetes\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.908860 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908464 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-run\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.908860 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908473 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb028709-a2f7-4338-8699-120a4879ffaf-konnectivity-ca\") pod \"konnectivity-agent-9tdjb\" (UID: \"cb028709-a2f7-4338-8699-120a4879ffaf\") " pod="kube-system/konnectivity-agent-9tdjb" Sep 07 10:08:04.908860 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908501 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/18c4a942-f1db-473e-aab6-d35c7bde40ae-sys-fs\") pod \"aws-ebs-csi-driver-node-fr4jh\" (UID: \"18c4a942-f1db-473e-aab6-d35c7bde40ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.908860 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908526 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-var-lib-kubelet\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.908860 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908529 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9zzn\" (UniqueName: \"kubernetes.io/projected/9aeb0af4-1773-4b88-b17f-d1d4d6490101-kube-api-access-f9zzn\") pod \"multus-additional-cni-plugins-nzg2l\" (UID: \"9aeb0af4-1773-4b88-b17f-d1d4d6490101\") " pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.908860 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908562 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-etc-sysconfig\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.908860 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908588 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-sys\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.908860 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908612 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-lib-modules\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.908860 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908641 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/18c4a942-f1db-473e-aab6-d35c7bde40ae-registration-dir\") pod \"aws-ebs-csi-driver-node-fr4jh\" (UID: \"18c4a942-f1db-473e-aab6-d35c7bde40ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.908860 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908671 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9aeb0af4-1773-4b88-b17f-d1d4d6490101-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nzg2l\" (UID: \"9aeb0af4-1773-4b88-b17f-d1d4d6490101\") " pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.908860 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908695 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a6a30db9-93e6-41b1-a0e6-28e5088df925-serviceca\") pod \"node-ca-t6ztt\" (UID: \"a6a30db9-93e6-41b1-a0e6-28e5088df925\") " pod="openshift-image-registry/node-ca-t6ztt" Sep 07 10:08:04.908860 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908698 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-etc-modprobe-d\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.908860 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908738 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb028709-a2f7-4338-8699-120a4879ffaf-agent-certs\") pod \"konnectivity-agent-9tdjb\" (UID: \"cb028709-a2f7-4338-8699-120a4879ffaf\") " pod="kube-system/konnectivity-agent-9tdjb" Sep 07 10:08:04.909728 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908766 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpl4s\" (UniqueName: \"kubernetes.io/projected/18c4a942-f1db-473e-aab6-d35c7bde40ae-kube-api-access-tpl4s\") pod \"aws-ebs-csi-driver-node-fr4jh\" (UID: \"18c4a942-f1db-473e-aab6-d35c7bde40ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.909728 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908791 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-etc-modprobe-d\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.909728 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908797 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a1a1ee72-afe3-4901-8a66-95c456e21ada-etc-tuned\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.909728 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908798 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-etc-kubernetes\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.909728 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908864 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-etc-sysconfig\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.909728 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908916 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-sys\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.909728 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:04.908934 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:04.909728 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.908994 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a1a1ee72-afe3-4901-8a66-95c456e21ada-lib-modules\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.909728 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:04.909024 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs podName:d11a5179-d3c5-48d5-a7c6-a7c367fd2584 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:05.409005788 +0000 UTC m=+3.121247720 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs") pod "network-metrics-daemon-4wfhx" (UID: "d11a5179-d3c5-48d5-a7c6-a7c367fd2584") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:04.909728 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.909043 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/18c4a942-f1db-473e-aab6-d35c7bde40ae-registration-dir\") pod \"aws-ebs-csi-driver-node-fr4jh\" (UID: \"18c4a942-f1db-473e-aab6-d35c7bde40ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.909728 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.909348 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb028709-a2f7-4338-8699-120a4879ffaf-konnectivity-ca\") pod \"konnectivity-agent-9tdjb\" (UID: \"cb028709-a2f7-4338-8699-120a4879ffaf\") " pod="kube-system/konnectivity-agent-9tdjb" Sep 07 10:08:04.909728 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.909352 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9aeb0af4-1773-4b88-b17f-d1d4d6490101-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nzg2l\" (UID: \"9aeb0af4-1773-4b88-b17f-d1d4d6490101\") " pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.909728 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.909397 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/18c4a942-f1db-473e-aab6-d35c7bde40ae-device-dir\") pod \"aws-ebs-csi-driver-node-fr4jh\" (UID: \"18c4a942-f1db-473e-aab6-d35c7bde40ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.909728 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.909436 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/18c4a942-f1db-473e-aab6-d35c7bde40ae-sys-fs\") pod \"aws-ebs-csi-driver-node-fr4jh\" (UID: \"18c4a942-f1db-473e-aab6-d35c7bde40ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.909728 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.909478 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9aeb0af4-1773-4b88-b17f-d1d4d6490101-system-cni-dir\") pod \"multus-additional-cni-plugins-nzg2l\" (UID: \"9aeb0af4-1773-4b88-b17f-d1d4d6490101\") " pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.909728 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.909531 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9aeb0af4-1773-4b88-b17f-d1d4d6490101-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nzg2l\" (UID: \"9aeb0af4-1773-4b88-b17f-d1d4d6490101\") " pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.910565 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.910147 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9aeb0af4-1773-4b88-b17f-d1d4d6490101-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nzg2l\" (UID: \"9aeb0af4-1773-4b88-b17f-d1d4d6490101\") " pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.910565 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.910388 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1a1ee72-afe3-4901-8a66-95c456e21ada-tmp\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.913527 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.913501 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb028709-a2f7-4338-8699-120a4879ffaf-agent-certs\") pod \"konnectivity-agent-9tdjb\" (UID: \"cb028709-a2f7-4338-8699-120a4879ffaf\") " pod="kube-system/konnectivity-agent-9tdjb" Sep 07 10:08:04.913754 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.913728 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a1a1ee72-afe3-4901-8a66-95c456e21ada-etc-tuned\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.915310 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:04.915287 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 10:08:04.915419 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:04.915315 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 10:08:04.915419 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:04.915329 2575 projected.go:196] Error preparing data for projected volume kube-api-access-r2s2g for pod openshift-network-diagnostics/network-check-target-km29h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:04.915526 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:04.915441 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g podName:d4e7d380-df30-4ee5-9059-6fd7fb6e82b7 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:05.415420773 +0000 UTC m=+3.127662719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-r2s2g" (UniqueName: "kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g") pod "network-check-target-km29h" (UID: "d4e7d380-df30-4ee5-9059-6fd7fb6e82b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:04.920158 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.920131 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9zzn\" (UniqueName: \"kubernetes.io/projected/9aeb0af4-1773-4b88-b17f-d1d4d6490101-kube-api-access-f9zzn\") pod \"multus-additional-cni-plugins-nzg2l\" (UID: \"9aeb0af4-1773-4b88-b17f-d1d4d6490101\") " pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:04.921268 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.921205 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qmzv\" (UniqueName: \"kubernetes.io/projected/a1a1ee72-afe3-4901-8a66-95c456e21ada-kube-api-access-8qmzv\") pod \"tuned-4wg78\" (UID: \"a1a1ee72-afe3-4901-8a66-95c456e21ada\") " pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:04.922517 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.922472 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpl4s\" (UniqueName: \"kubernetes.io/projected/18c4a942-f1db-473e-aab6-d35c7bde40ae-kube-api-access-tpl4s\") pod \"aws-ebs-csi-driver-node-fr4jh\" (UID: \"18c4a942-f1db-473e-aab6-d35c7bde40ae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:04.922844 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.922818 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k94pz\" (UniqueName: \"kubernetes.io/projected/a6a30db9-93e6-41b1-a0e6-28e5088df925-kube-api-access-k94pz\") pod \"node-ca-t6ztt\" (UID: \"a6a30db9-93e6-41b1-a0e6-28e5088df925\") " pod="openshift-image-registry/node-ca-t6ztt" Sep 07 10:08:04.924321 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.924274 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp5m8\" (UniqueName: \"kubernetes.io/projected/adcc0041-9a3b-4ff3-8a9a-25741c8e9baa-kube-api-access-hp5m8\") pod \"iptables-alerter-qss98\" (UID: \"adcc0041-9a3b-4ff3-8a9a-25741c8e9baa\") " pod="openshift-network-operator/iptables-alerter-qss98" Sep 07 10:08:04.924321 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.924292 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6vgv\" (UniqueName: \"kubernetes.io/projected/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-kube-api-access-g6vgv\") pod \"network-metrics-daemon-4wfhx\" (UID: \"d11a5179-d3c5-48d5-a7c6-a7c367fd2584\") " pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:04.992755 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:04.992663 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6q6w4" Sep 07 10:08:05.003999 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.003971 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w5w2p" Sep 07 10:08:05.004277 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:05.004249 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod448a5179_febc_460a_ab61_f7810c8bf09b.slice/crio-62213cf86b77b5e87342779436737c829d49c7070e83335c5f859268a8632ad3 WatchSource:0}: Error finding container 62213cf86b77b5e87342779436737c829d49c7070e83335c5f859268a8632ad3: Status 404 returned error can't find the container with id 62213cf86b77b5e87342779436737c829d49c7070e83335c5f859268a8632ad3 Sep 07 10:08:05.016022 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:05.015987 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19524835_4647_4786_bc0f_c222dd09c075.slice/crio-a1cd27a1d9b27685ab712f893977dba4d38f1fa21212fbd8f441680c1f79dde3 WatchSource:0}: Error finding container a1cd27a1d9b27685ab712f893977dba4d38f1fa21212fbd8f441680c1f79dde3: Status 404 returned error can't find the container with id a1cd27a1d9b27685ab712f893977dba4d38f1fa21212fbd8f441680c1f79dde3 Sep 07 10:08:05.020093 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.020066 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:05.023611 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.023470 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9tdjb" Sep 07 10:08:05.031644 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.031415 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" Sep 07 10:08:05.031898 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:05.031849 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf245ed6c_716b_4e7f_89bf_2db47ba2e83c.slice/crio-c2ff6c5d67387ed8933fe24a2a495ce25cdf9aa7e8437968f41fd8944d7e07a5 WatchSource:0}: Error finding container c2ff6c5d67387ed8933fe24a2a495ce25cdf9aa7e8437968f41fd8944d7e07a5: Status 404 returned error can't find the container with id c2ff6c5d67387ed8933fe24a2a495ce25cdf9aa7e8437968f41fd8944d7e07a5 Sep 07 10:08:05.035537 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:05.035489 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb028709_a2f7_4338_8699_120a4879ffaf.slice/crio-bc0db9c6134e02aa1201660061be5264fce5aed48e15dd5f463af79582dc921f WatchSource:0}: Error finding container bc0db9c6134e02aa1201660061be5264fce5aed48e15dd5f463af79582dc921f: Status 404 returned error can't find the container with id bc0db9c6134e02aa1201660061be5264fce5aed48e15dd5f463af79582dc921f Sep 07 10:08:05.041197 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.040977 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t6ztt" Sep 07 10:08:05.044033 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:05.044007 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18c4a942_f1db_473e_aab6_d35c7bde40ae.slice/crio-fec2ed3d3d7750626d5b9ea2bf7972adc5827f0a169fe6b0943ca9981d0db534 WatchSource:0}: Error finding container fec2ed3d3d7750626d5b9ea2bf7972adc5827f0a169fe6b0943ca9981d0db534: Status 404 returned error can't find the container with id fec2ed3d3d7750626d5b9ea2bf7972adc5827f0a169fe6b0943ca9981d0db534 Sep 07 10:08:05.051658 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.051627 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qss98" Sep 07 10:08:05.052704 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:05.052645 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6a30db9_93e6_41b1_a0e6_28e5088df925.slice/crio-c6316e9b84b62ef3cfbeb216112c50b0e9f02bb4e03991736884ab4e49f9e663 WatchSource:0}: Error finding container c6316e9b84b62ef3cfbeb216112c50b0e9f02bb4e03991736884ab4e49f9e663: Status 404 returned error can't find the container with id c6316e9b84b62ef3cfbeb216112c50b0e9f02bb4e03991736884ab4e49f9e663 Sep 07 10:08:05.061290 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.061260 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nzg2l" Sep 07 10:08:05.064167 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:05.064139 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadcc0041_9a3b_4ff3_8a9a_25741c8e9baa.slice/crio-0c4b05c2c3938cb47e41764b4d5f26b75e29446f3daa029017dfff324e279658 WatchSource:0}: Error finding container 0c4b05c2c3938cb47e41764b4d5f26b75e29446f3daa029017dfff324e279658: Status 404 returned error can't find the container with id 0c4b05c2c3938cb47e41764b4d5f26b75e29446f3daa029017dfff324e279658 Sep 07 10:08:05.066882 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.066838 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4wg78" Sep 07 10:08:05.078025 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:05.077994 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9aeb0af4_1773_4b88_b17f_d1d4d6490101.slice/crio-2ab2c681744402672bd61d501b534ef2ba1b653abdf6b6fa664bb680055a41a3 WatchSource:0}: Error finding container 2ab2c681744402672bd61d501b534ef2ba1b653abdf6b6fa664bb680055a41a3: Status 404 returned error can't find the container with id 2ab2c681744402672bd61d501b534ef2ba1b653abdf6b6fa664bb680055a41a3 Sep 07 10:08:05.079542 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:05.079509 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1a1ee72_afe3_4901_8a66_95c456e21ada.slice/crio-95e04936cafd5496a701088a3c56ce9b14dd0f918490234d97bf16a453b39083 WatchSource:0}: Error finding container 95e04936cafd5496a701088a3c56ce9b14dd0f918490234d97bf16a453b39083: Status 404 returned error can't find the container with id 95e04936cafd5496a701088a3c56ce9b14dd0f918490234d97bf16a453b39083 Sep 07 10:08:05.312771 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.312667 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret\") pod \"global-pull-secret-syncer-6v64l\" (UID: \"cff88048-74be-46c2-a8ac-113c5fc3a85d\") " pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:05.312943 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:05.312873 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:05.312998 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:05.312951 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret podName:cff88048-74be-46c2-a8ac-113c5fc3a85d nodeName:}" failed. No retries permitted until 2026-09-07 10:08:06.312934624 +0000 UTC m=+4.025176548 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret") pod "global-pull-secret-syncer-6v64l" (UID: "cff88048-74be-46c2-a8ac-113c5fc3a85d") : object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:05.413736 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.413694 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs\") pod \"network-metrics-daemon-4wfhx\" (UID: \"d11a5179-d3c5-48d5-a7c6-a7c367fd2584\") " pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:05.413923 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:05.413859 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:05.413923 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:05.413922 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs podName:d11a5179-d3c5-48d5-a7c6-a7c367fd2584 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:06.413907366 +0000 UTC m=+4.126149290 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs") pod "network-metrics-daemon-4wfhx" (UID: "d11a5179-d3c5-48d5-a7c6-a7c367fd2584") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:05.514071 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.514038 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2s2g\" (UniqueName: \"kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g\") pod \"network-check-target-km29h\" (UID: \"d4e7d380-df30-4ee5-9059-6fd7fb6e82b7\") " pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:05.514306 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:05.514283 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 10:08:05.514419 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:05.514313 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 10:08:05.514419 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:05.514328 2575 projected.go:196] Error preparing data for projected volume kube-api-access-r2s2g for pod openshift-network-diagnostics/network-check-target-km29h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:05.514419 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:05.514404 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g podName:d4e7d380-df30-4ee5-9059-6fd7fb6e82b7 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:06.514382455 +0000 UTC m=+4.226624401 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-r2s2g" (UniqueName: "kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g") pod "network-check-target-km29h" (UID: "d4e7d380-df30-4ee5-9059-6fd7fb6e82b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:05.747160 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.747060 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 10:03:05 +0000 UTC" deadline="2028-05-19 10:30:44.325217252 +0000 UTC" Sep 07 10:08:05.747160 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.747101 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14880h22m38.578120452s" Sep 07 10:08:05.826066 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.826035 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:05.826328 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:05.826178 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6v64l" podUID="cff88048-74be-46c2-a8ac-113c5fc3a85d" Sep 07 10:08:05.836780 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.836741 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4wg78" event={"ID":"a1a1ee72-afe3-4901-8a66-95c456e21ada","Type":"ContainerStarted","Data":"95e04936cafd5496a701088a3c56ce9b14dd0f918490234d97bf16a453b39083"} Sep 07 10:08:05.838230 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.838148 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzg2l" event={"ID":"9aeb0af4-1773-4b88-b17f-d1d4d6490101","Type":"ContainerStarted","Data":"2ab2c681744402672bd61d501b534ef2ba1b653abdf6b6fa664bb680055a41a3"} Sep 07 10:08:05.839761 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.839728 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qss98" event={"ID":"adcc0041-9a3b-4ff3-8a9a-25741c8e9baa","Type":"ContainerStarted","Data":"0c4b05c2c3938cb47e41764b4d5f26b75e29446f3daa029017dfff324e279658"} Sep 07 10:08:05.841223 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.841196 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" event={"ID":"18c4a942-f1db-473e-aab6-d35c7bde40ae","Type":"ContainerStarted","Data":"fec2ed3d3d7750626d5b9ea2bf7972adc5827f0a169fe6b0943ca9981d0db534"} Sep 07 10:08:05.842710 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.842658 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9tdjb" event={"ID":"cb028709-a2f7-4338-8699-120a4879ffaf","Type":"ContainerStarted","Data":"bc0db9c6134e02aa1201660061be5264fce5aed48e15dd5f463af79582dc921f"} Sep 07 10:08:05.844235 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.844208 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" event={"ID":"f245ed6c-716b-4e7f-89bf-2db47ba2e83c","Type":"ContainerStarted","Data":"c2ff6c5d67387ed8933fe24a2a495ce25cdf9aa7e8437968f41fd8944d7e07a5"} Sep 07 10:08:05.845742 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.845710 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w5w2p" event={"ID":"19524835-4647-4786-bc0f-c222dd09c075","Type":"ContainerStarted","Data":"a1cd27a1d9b27685ab712f893977dba4d38f1fa21212fbd8f441680c1f79dde3"} Sep 07 10:08:05.847218 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.847191 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6q6w4" event={"ID":"448a5179-febc-460a-ab61-f7810c8bf09b","Type":"ContainerStarted","Data":"62213cf86b77b5e87342779436737c829d49c7070e83335c5f859268a8632ad3"} Sep 07 10:08:05.848655 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:05.848633 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t6ztt" event={"ID":"a6a30db9-93e6-41b1-a0e6-28e5088df925","Type":"ContainerStarted","Data":"c6316e9b84b62ef3cfbeb216112c50b0e9f02bb4e03991736884ab4e49f9e663"} Sep 07 10:08:06.321806 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:06.321171 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret\") pod \"global-pull-secret-syncer-6v64l\" (UID: \"cff88048-74be-46c2-a8ac-113c5fc3a85d\") " pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:06.321806 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:06.321354 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:06.321806 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:06.321426 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret podName:cff88048-74be-46c2-a8ac-113c5fc3a85d nodeName:}" failed. No retries permitted until 2026-09-07 10:08:08.321405648 +0000 UTC m=+6.033647598 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret") pod "global-pull-secret-syncer-6v64l" (UID: "cff88048-74be-46c2-a8ac-113c5fc3a85d") : object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:06.422642 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:06.421918 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs\") pod \"network-metrics-daemon-4wfhx\" (UID: \"d11a5179-d3c5-48d5-a7c6-a7c367fd2584\") " pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:06.422642 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:06.422250 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:06.422642 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:06.422372 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs podName:d11a5179-d3c5-48d5-a7c6-a7c367fd2584 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:08.422351589 +0000 UTC m=+6.134593518 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs") pod "network-metrics-daemon-4wfhx" (UID: "d11a5179-d3c5-48d5-a7c6-a7c367fd2584") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:06.523371 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:06.523280 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2s2g\" (UniqueName: \"kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g\") pod \"network-check-target-km29h\" (UID: \"d4e7d380-df30-4ee5-9059-6fd7fb6e82b7\") " pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:06.523549 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:06.523485 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 10:08:06.523549 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:06.523512 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 10:08:06.523549 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:06.523526 2575 projected.go:196] Error preparing data for projected volume kube-api-access-r2s2g for pod openshift-network-diagnostics/network-check-target-km29h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:06.523721 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:06.523594 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g podName:d4e7d380-df30-4ee5-9059-6fd7fb6e82b7 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:08.52357162 +0000 UTC m=+6.235813549 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-r2s2g" (UniqueName: "kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g") pod "network-check-target-km29h" (UID: "d4e7d380-df30-4ee5-9059-6fd7fb6e82b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:06.748487 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:06.748334 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 10:03:05 +0000 UTC" deadline="2028-04-14 22:52:06.188264858 +0000 UTC" Sep 07 10:08:06.748487 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:06.748402 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14052h43m59.4398687s" Sep 07 10:08:06.828375 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:06.828285 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:06.828545 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:06.828445 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wfhx" podUID="d11a5179-d3c5-48d5-a7c6-a7c367fd2584" Sep 07 10:08:06.828545 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:06.828476 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:06.828662 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:06.828558 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km29h" podUID="d4e7d380-df30-4ee5-9059-6fd7fb6e82b7" Sep 07 10:08:06.864306 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:06.864262 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal" event={"ID":"f155589bfc7e2b2766af1b57a8ea8129","Type":"ContainerStarted","Data":"a76cdb81801deec5ff6125a306ebd986dd9fab5921729710fe5846556a940b32"} Sep 07 10:08:06.880757 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:06.880332 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-85.ec2.internal" podStartSLOduration=2.880310879 podStartE2EDuration="2.880310879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:06.880031637 +0000 UTC m=+4.592273587" watchObservedRunningTime="2026-09-07 10:08:06.880310879 +0000 UTC m=+4.592552829" Sep 07 10:08:07.825734 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:07.825640 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:07.826370 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:07.825785 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6v64l" podUID="cff88048-74be-46c2-a8ac-113c5fc3a85d" Sep 07 10:08:07.894779 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:07.894736 2575 generic.go:356] "Generic (PLEG): container finished" podID="18850670ff94925fc45b643b7c67aabb" containerID="5305351c4cae578af8d42e7c3110e5ba6594a4683386b4172f7209a133603734" exitCode=0 Sep 07 10:08:07.895024 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:07.894883 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal" event={"ID":"18850670ff94925fc45b643b7c67aabb","Type":"ContainerDied","Data":"5305351c4cae578af8d42e7c3110e5ba6594a4683386b4172f7209a133603734"} Sep 07 10:08:08.343491 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:08.343308 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret\") pod \"global-pull-secret-syncer-6v64l\" (UID: \"cff88048-74be-46c2-a8ac-113c5fc3a85d\") " pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:08.343687 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:08.343525 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:08.343687 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:08.343607 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret podName:cff88048-74be-46c2-a8ac-113c5fc3a85d nodeName:}" failed. No retries permitted until 2026-09-07 10:08:12.343585572 +0000 UTC m=+10.055827522 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret") pod "global-pull-secret-syncer-6v64l" (UID: "cff88048-74be-46c2-a8ac-113c5fc3a85d") : object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:08.444132 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:08.444086 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs\") pod \"network-metrics-daemon-4wfhx\" (UID: \"d11a5179-d3c5-48d5-a7c6-a7c367fd2584\") " pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:08.444336 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:08.444277 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:08.444405 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:08.444348 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs podName:d11a5179-d3c5-48d5-a7c6-a7c367fd2584 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:12.444326506 +0000 UTC m=+10.156568451 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs") pod "network-metrics-daemon-4wfhx" (UID: "d11a5179-d3c5-48d5-a7c6-a7c367fd2584") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:08.545953 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:08.545187 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2s2g\" (UniqueName: \"kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g\") pod \"network-check-target-km29h\" (UID: \"d4e7d380-df30-4ee5-9059-6fd7fb6e82b7\") " pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:08.545953 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:08.545436 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 10:08:08.545953 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:08.545461 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 10:08:08.545953 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:08.545474 2575 projected.go:196] Error preparing data for projected volume kube-api-access-r2s2g for pod openshift-network-diagnostics/network-check-target-km29h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:08.545953 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:08.545539 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g podName:d4e7d380-df30-4ee5-9059-6fd7fb6e82b7 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:12.54551816 +0000 UTC m=+10.257760104 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-r2s2g" (UniqueName: "kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g") pod "network-check-target-km29h" (UID: "d4e7d380-df30-4ee5-9059-6fd7fb6e82b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:08.827707 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:08.826955 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:08.827707 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:08.827092 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wfhx" podUID="d11a5179-d3c5-48d5-a7c6-a7c367fd2584" Sep 07 10:08:08.827707 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:08.827405 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:08.827707 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:08.827477 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km29h" podUID="d4e7d380-df30-4ee5-9059-6fd7fb6e82b7" Sep 07 10:08:09.826278 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:09.826237 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:09.826488 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:09.826406 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6v64l" podUID="cff88048-74be-46c2-a8ac-113c5fc3a85d" Sep 07 10:08:10.825811 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:10.825762 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:10.826267 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:10.825762 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:10.826267 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:10.825940 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wfhx" podUID="d11a5179-d3c5-48d5-a7c6-a7c367fd2584" Sep 07 10:08:10.826267 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:10.826033 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km29h" podUID="d4e7d380-df30-4ee5-9059-6fd7fb6e82b7" Sep 07 10:08:11.825868 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:11.825526 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:11.825868 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:11.825668 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6v64l" podUID="cff88048-74be-46c2-a8ac-113c5fc3a85d" Sep 07 10:08:12.380727 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:12.379995 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret\") pod \"global-pull-secret-syncer-6v64l\" (UID: \"cff88048-74be-46c2-a8ac-113c5fc3a85d\") " pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:12.380727 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:12.380139 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:12.380727 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:12.380207 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret podName:cff88048-74be-46c2-a8ac-113c5fc3a85d nodeName:}" failed. No retries permitted until 2026-09-07 10:08:20.38018874 +0000 UTC m=+18.092430667 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret") pod "global-pull-secret-syncer-6v64l" (UID: "cff88048-74be-46c2-a8ac-113c5fc3a85d") : object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:12.480400 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:12.480360 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs\") pod \"network-metrics-daemon-4wfhx\" (UID: \"d11a5179-d3c5-48d5-a7c6-a7c367fd2584\") " pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:12.480589 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:12.480539 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:12.480697 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:12.480658 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs podName:d11a5179-d3c5-48d5-a7c6-a7c367fd2584 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:20.480600629 +0000 UTC m=+18.192842566 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs") pod "network-metrics-daemon-4wfhx" (UID: "d11a5179-d3c5-48d5-a7c6-a7c367fd2584") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:12.581577 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:12.581152 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2s2g\" (UniqueName: \"kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g\") pod \"network-check-target-km29h\" (UID: \"d4e7d380-df30-4ee5-9059-6fd7fb6e82b7\") " pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:12.581577 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:12.581369 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 10:08:12.581577 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:12.581389 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 10:08:12.581577 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:12.581402 2575 projected.go:196] Error preparing data for projected volume kube-api-access-r2s2g for pod openshift-network-diagnostics/network-check-target-km29h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:12.581577 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:12.581469 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g podName:d4e7d380-df30-4ee5-9059-6fd7fb6e82b7 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:20.581449972 +0000 UTC m=+18.293691918 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-r2s2g" (UniqueName: "kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g") pod "network-check-target-km29h" (UID: "d4e7d380-df30-4ee5-9059-6fd7fb6e82b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:12.826328 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:12.826242 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:12.826751 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:12.826372 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wfhx" podUID="d11a5179-d3c5-48d5-a7c6-a7c367fd2584" Sep 07 10:08:12.826751 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:12.826430 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:12.826751 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:12.826559 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km29h" podUID="d4e7d380-df30-4ee5-9059-6fd7fb6e82b7" Sep 07 10:08:13.826046 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:13.825935 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:13.826218 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:13.826060 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6v64l" podUID="cff88048-74be-46c2-a8ac-113c5fc3a85d" Sep 07 10:08:14.826313 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:14.826279 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:14.826740 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:14.826279 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:14.826740 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:14.826432 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wfhx" podUID="d11a5179-d3c5-48d5-a7c6-a7c367fd2584" Sep 07 10:08:14.826740 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:14.826498 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km29h" podUID="d4e7d380-df30-4ee5-9059-6fd7fb6e82b7" Sep 07 10:08:15.825483 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:15.825446 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:15.825651 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:15.825595 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6v64l" podUID="cff88048-74be-46c2-a8ac-113c5fc3a85d" Sep 07 10:08:16.825861 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:16.825822 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:16.826293 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:16.825822 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:16.826293 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:16.825952 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km29h" podUID="d4e7d380-df30-4ee5-9059-6fd7fb6e82b7" Sep 07 10:08:16.826293 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:16.826000 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wfhx" podUID="d11a5179-d3c5-48d5-a7c6-a7c367fd2584" Sep 07 10:08:17.826158 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:17.826120 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:17.826627 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:17.826233 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6v64l" podUID="cff88048-74be-46c2-a8ac-113c5fc3a85d" Sep 07 10:08:18.826316 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:18.826278 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:18.826711 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:18.826409 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wfhx" podUID="d11a5179-d3c5-48d5-a7c6-a7c367fd2584" Sep 07 10:08:18.826711 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:18.826447 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:18.826711 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:18.826562 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km29h" podUID="d4e7d380-df30-4ee5-9059-6fd7fb6e82b7" Sep 07 10:08:19.826267 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:19.826079 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:19.826468 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:19.826344 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6v64l" podUID="cff88048-74be-46c2-a8ac-113c5fc3a85d" Sep 07 10:08:20.439155 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:20.439115 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret\") pod \"global-pull-secret-syncer-6v64l\" (UID: \"cff88048-74be-46c2-a8ac-113c5fc3a85d\") " pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:20.439333 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:20.439273 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:20.439389 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:20.439347 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret podName:cff88048-74be-46c2-a8ac-113c5fc3a85d nodeName:}" failed. No retries permitted until 2026-09-07 10:08:36.439327934 +0000 UTC m=+34.151569857 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret") pod "global-pull-secret-syncer-6v64l" (UID: "cff88048-74be-46c2-a8ac-113c5fc3a85d") : object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:20.539467 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:20.539430 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs\") pod \"network-metrics-daemon-4wfhx\" (UID: \"d11a5179-d3c5-48d5-a7c6-a7c367fd2584\") " pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:20.539654 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:20.539600 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:20.539719 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:20.539670 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs podName:d11a5179-d3c5-48d5-a7c6-a7c367fd2584 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:36.539648521 +0000 UTC m=+34.251890444 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs") pod "network-metrics-daemon-4wfhx" (UID: "d11a5179-d3c5-48d5-a7c6-a7c367fd2584") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:20.639959 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:20.639930 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2s2g\" (UniqueName: \"kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g\") pod \"network-check-target-km29h\" (UID: \"d4e7d380-df30-4ee5-9059-6fd7fb6e82b7\") " pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:20.640136 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:20.640089 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 10:08:20.640136 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:20.640114 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 10:08:20.640136 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:20.640128 2575 projected.go:196] Error preparing data for projected volume kube-api-access-r2s2g for pod openshift-network-diagnostics/network-check-target-km29h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:20.640276 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:20.640189 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g podName:d4e7d380-df30-4ee5-9059-6fd7fb6e82b7 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:36.640170686 +0000 UTC m=+34.352412616 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-r2s2g" (UniqueName: "kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g") pod "network-check-target-km29h" (UID: "d4e7d380-df30-4ee5-9059-6fd7fb6e82b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:20.826362 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:20.826278 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:20.826504 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:20.826282 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:20.826504 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:20.826432 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km29h" podUID="d4e7d380-df30-4ee5-9059-6fd7fb6e82b7" Sep 07 10:08:20.826504 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:20.826479 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wfhx" podUID="d11a5179-d3c5-48d5-a7c6-a7c367fd2584" Sep 07 10:08:21.826078 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:21.826025 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:21.826234 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:21.826178 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6v64l" podUID="cff88048-74be-46c2-a8ac-113c5fc3a85d" Sep 07 10:08:22.829188 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:22.829153 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:22.829593 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:22.829193 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:22.829593 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:22.829268 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km29h" podUID="d4e7d380-df30-4ee5-9059-6fd7fb6e82b7" Sep 07 10:08:22.829593 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:22.829457 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wfhx" podUID="d11a5179-d3c5-48d5-a7c6-a7c367fd2584" Sep 07 10:08:23.825747 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:23.825722 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:23.825944 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:23.825924 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6v64l" podUID="cff88048-74be-46c2-a8ac-113c5fc3a85d" Sep 07 10:08:23.934074 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:23.934019 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4wg78" event={"ID":"a1a1ee72-afe3-4901-8a66-95c456e21ada","Type":"ContainerStarted","Data":"9ba09b9b566a7f86b6ae48727c76d99182d29f44f4653a29e80231ca8529de25"} Sep 07 10:08:23.935718 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:23.935693 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzg2l" event={"ID":"9aeb0af4-1773-4b88-b17f-d1d4d6490101","Type":"ContainerStarted","Data":"c0dfbfb77db2654de44c0137fee7e5779f1772ff56bbf0dad24945654d709762"} Sep 07 10:08:23.937200 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:23.937180 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" event={"ID":"18c4a942-f1db-473e-aab6-d35c7bde40ae","Type":"ContainerStarted","Data":"46ac1919ebc04a7957db23a9bd894202b672c9d55f1bc9fc2003af041eabd59e"} Sep 07 10:08:23.938771 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:23.938746 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9tdjb" event={"ID":"cb028709-a2f7-4338-8699-120a4879ffaf","Type":"ContainerStarted","Data":"dc0e03a47e44d2e31a3c879cdbb33e34bc48cc5366263e92a2bd1aabbafa6cb7"} Sep 07 10:08:23.940365 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:23.940345 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/ovn-acl-logging/0.log" Sep 07 10:08:23.940766 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:23.940743 2575 generic.go:356] "Generic (PLEG): container finished" podID="f245ed6c-716b-4e7f-89bf-2db47ba2e83c" containerID="97dbaa132ddd2815b6656e836be504aa2fc0f74b55f6509b64035c8fc5afbc7d" exitCode=1 Sep 07 10:08:23.940846 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:23.940829 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" event={"ID":"f245ed6c-716b-4e7f-89bf-2db47ba2e83c","Type":"ContainerDied","Data":"97dbaa132ddd2815b6656e836be504aa2fc0f74b55f6509b64035c8fc5afbc7d"} Sep 07 10:08:23.940919 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:23.940906 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" event={"ID":"f245ed6c-716b-4e7f-89bf-2db47ba2e83c","Type":"ContainerStarted","Data":"a81ec4c15d843ee02595cfabe0e14f85fe24653e465ea63c134211360494a6d8"} Sep 07 10:08:23.942300 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:23.942279 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w5w2p" event={"ID":"19524835-4647-4786-bc0f-c222dd09c075","Type":"ContainerStarted","Data":"58e303b5add332297b6d755a6b70bd363feed3c7e25b1941bc03099250255907"} Sep 07 10:08:23.944181 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:23.944160 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal" event={"ID":"18850670ff94925fc45b643b7c67aabb","Type":"ContainerStarted","Data":"a7606ea50008a75b33ad07d6e6d0a25f2ddea26292ab2be6dadd7e998c1c3f5d"} Sep 07 10:08:23.945418 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:23.945400 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t6ztt" event={"ID":"a6a30db9-93e6-41b1-a0e6-28e5088df925","Type":"ContainerStarted","Data":"0f7075848534d826a44f475404aef9c957891bffd2e7206624e4b5d7fe36f1eb"} Sep 07 10:08:23.996274 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:23.996229 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4wg78" podStartSLOduration=2.539371568 podStartE2EDuration="20.996212543s" totalImagesPullingTime="18.456840975s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:23.957694638 +0000 UTC m=+21.669936583" watchObservedRunningTime="2026-09-07 10:08:23.996212543 +0000 UTC m=+21.708454487" Sep 07 10:08:24.017418 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:24.017362 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-w5w2p" podStartSLOduration=2.515046571 podStartE2EDuration="21.017344287s" totalImagesPullingTime="18.502297716s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:24.017308076 +0000 UTC m=+21.729550020" watchObservedRunningTime="2026-09-07 10:08:24.017344287 +0000 UTC m=+21.729586231" Sep 07 10:08:24.036354 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:24.036147 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-t6ztt" podStartSLOduration=2.570883936 podStartE2EDuration="21.036130162s" totalImagesPullingTime="18.465246226s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:24.035958746 +0000 UTC m=+21.748200692" watchObservedRunningTime="2026-09-07 10:08:24.036130162 +0000 UTC m=+21.748372197" Sep 07 10:08:24.056608 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:24.056567 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-85.ec2.internal" podStartSLOduration=20.056550797 podStartE2EDuration="20.056550797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:24.056197693 +0000 UTC m=+21.768439638" watchObservedRunningTime="2026-09-07 10:08:24.056550797 +0000 UTC m=+21.768792741" Sep 07 10:08:24.072633 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:24.072588 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9tdjb" podStartSLOduration=2.5914880780000003 podStartE2EDuration="21.072572858s" totalImagesPullingTime="18.48108478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:24.072556038 +0000 UTC m=+21.784797979" watchObservedRunningTime="2026-09-07 10:08:24.072572858 +0000 UTC m=+21.784814781" Sep 07 10:08:24.830204 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:24.830129 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:24.830380 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:24.830129 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:24.830380 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:24.830264 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wfhx" podUID="d11a5179-d3c5-48d5-a7c6-a7c367fd2584" Sep 07 10:08:24.830380 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:24.830343 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km29h" podUID="d4e7d380-df30-4ee5-9059-6fd7fb6e82b7" Sep 07 10:08:24.954111 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:24.953928 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/ovn-acl-logging/0.log" Sep 07 10:08:24.955105 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:24.955078 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" event={"ID":"f245ed6c-716b-4e7f-89bf-2db47ba2e83c","Type":"ContainerStarted","Data":"015f6d9be7dae48e1020cf1745a5e8f01f26e720c0736e8352437b99176e00c1"} Sep 07 10:08:24.955200 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:24.955156 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" event={"ID":"f245ed6c-716b-4e7f-89bf-2db47ba2e83c","Type":"ContainerStarted","Data":"3df6faf72a63fb7397d6dc9b892d9f777cbeab4c7f9e49b12b1e8f701f4aa5c6"} Sep 07 10:08:24.955256 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:24.955203 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" event={"ID":"f245ed6c-716b-4e7f-89bf-2db47ba2e83c","Type":"ContainerStarted","Data":"09ac9a5580c042080d7198d3390b70b2148b72bb8798c3c5e7b16f77daa619c5"} Sep 07 10:08:24.955256 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:24.955237 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" event={"ID":"f245ed6c-716b-4e7f-89bf-2db47ba2e83c","Type":"ContainerStarted","Data":"ac7fa4926929e1939605fcbc01cc21035c7c427253840de40cb71b525ea2db46"} Sep 07 10:08:24.957104 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:24.957063 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6q6w4" event={"ID":"448a5179-febc-460a-ab61-f7810c8bf09b","Type":"ContainerStarted","Data":"a35a37e6eb7c424b7ff3cf94dd9b550f856645cd3e080f5f0ee624ef04992741"} Sep 07 10:08:24.959069 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:24.958978 2575 generic.go:356] "Generic (PLEG): container finished" podID="9aeb0af4-1773-4b88-b17f-d1d4d6490101" containerID="c0dfbfb77db2654de44c0137fee7e5779f1772ff56bbf0dad24945654d709762" exitCode=0 Sep 07 10:08:24.959069 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:24.959051 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzg2l" event={"ID":"9aeb0af4-1773-4b88-b17f-d1d4d6490101","Type":"ContainerDied","Data":"c0dfbfb77db2654de44c0137fee7e5779f1772ff56bbf0dad24945654d709762"} Sep 07 10:08:24.961373 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:24.961347 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qss98" event={"ID":"adcc0041-9a3b-4ff3-8a9a-25741c8e9baa","Type":"ContainerStarted","Data":"ad6c55a7b660f3086bb5454f53f64248c3ea87769d9dfe9eb5e1b2a1eef8d8ce"} Sep 07 10:08:24.975575 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:24.975531 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6q6w4" podStartSLOduration=2.885713305 podStartE2EDuration="21.975516418s" totalImagesPullingTime="19.089803113s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:24.974772627 +0000 UTC m=+22.687014572" watchObservedRunningTime="2026-09-07 10:08:24.975516418 +0000 UTC m=+22.687758362" Sep 07 10:08:24.990338 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:24.990297 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qss98" podStartSLOduration=3.536873123 podStartE2EDuration="21.990280802s" totalImagesPullingTime="18.453407679s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:24.989913878 +0000 UTC m=+22.702155828" watchObservedRunningTime="2026-09-07 10:08:24.990280802 +0000 UTC m=+22.702522749" Sep 07 10:08:25.035877 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:25.035851 2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 07 10:08:25.774593 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:25.774473 2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-07T10:08:25.03587327Z","UUID":"dd9d7186-c57d-45c5-acde-2933488df1f0","Handler":null,"Name":"","Endpoint":""} Sep 07 10:08:25.776249 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:25.776227 2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 07 10:08:25.776406 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:25.776261 2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 07 10:08:25.809449 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:25.809419 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9tdjb" Sep 07 10:08:25.810138 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:25.810105 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9tdjb" Sep 07 10:08:25.825763 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:25.825603 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:25.825763 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:25.825717 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6v64l" podUID="cff88048-74be-46c2-a8ac-113c5fc3a85d" Sep 07 10:08:25.965644 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:25.965602 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" event={"ID":"18c4a942-f1db-473e-aab6-d35c7bde40ae","Type":"ContainerStarted","Data":"b9f36a205fe2de0185ad9782e6ec889f5b3bf05b97de0ca2303221e9f9943260"} Sep 07 10:08:26.828828 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:26.828785 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:26.828828 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:26.828822 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:26.829059 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:26.828912 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km29h" podUID="d4e7d380-df30-4ee5-9059-6fd7fb6e82b7" Sep 07 10:08:26.829059 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:26.828997 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wfhx" podUID="d11a5179-d3c5-48d5-a7c6-a7c367fd2584" Sep 07 10:08:26.970329 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:26.970292 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" event={"ID":"18c4a942-f1db-473e-aab6-d35c7bde40ae","Type":"ContainerStarted","Data":"eb30fc2994e993c5bb4f89ed1595a9acd8ca6a3ea7a9f1067d5158882b4c1664"} Sep 07 10:08:26.973543 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:26.973521 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/ovn-acl-logging/0.log" Sep 07 10:08:26.973948 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:26.973930 2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 07 10:08:26.974056 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:26.973924 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" event={"ID":"f245ed6c-716b-4e7f-89bf-2db47ba2e83c","Type":"ContainerStarted","Data":"f8ba7ff9e7fc0f4dde88b8830fceed1d8aa75b8ea6a13b224ce58a1b80f873f0"} Sep 07 10:08:26.989355 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:26.989304 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fr4jh" podStartSLOduration=3.277766901 podStartE2EDuration="23.989283937s" totalImagesPullingTime="20.711517036s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:26.988694808 +0000 UTC m=+24.700936764" watchObservedRunningTime="2026-09-07 10:08:26.989283937 +0000 UTC m=+24.701525883" Sep 07 10:08:27.485412 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:27.485377 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9tdjb" Sep 07 10:08:27.486158 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:27.486132 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9tdjb" Sep 07 10:08:27.825822 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:27.825738 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:27.825974 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:27.825924 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6v64l" podUID="cff88048-74be-46c2-a8ac-113c5fc3a85d" Sep 07 10:08:28.825656 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:28.825495 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:28.825998 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:28.825508 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:28.825998 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:28.825735 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wfhx" podUID="d11a5179-d3c5-48d5-a7c6-a7c367fd2584" Sep 07 10:08:28.825998 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:28.825850 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km29h" podUID="d4e7d380-df30-4ee5-9059-6fd7fb6e82b7" Sep 07 10:08:28.983584 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:28.983548 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzg2l" event={"ID":"9aeb0af4-1773-4b88-b17f-d1d4d6490101","Type":"ContainerStarted","Data":"1c648ae0aa18f7025562edc83d2a1171899004e6d11fbdc65b84cc7cfb30693d"} Sep 07 10:08:28.987323 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:28.987299 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/ovn-acl-logging/0.log" Sep 07 10:08:28.987821 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:28.987778 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" event={"ID":"f245ed6c-716b-4e7f-89bf-2db47ba2e83c","Type":"ContainerStarted","Data":"9fa122ad36944f8fb024d79db7665dd57efdc724fcac04a7ba07b8a516e6b2c1"} Sep 07 10:08:28.988187 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:28.988173 2575 scope.go:117] "RemoveContainer" containerID="97dbaa132ddd2815b6656e836be504aa2fc0f74b55f6509b64035c8fc5afbc7d" Sep 07 10:08:29.825435 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:29.825359 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:29.825608 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:29.825466 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6v64l" podUID="cff88048-74be-46c2-a8ac-113c5fc3a85d" Sep 07 10:08:29.992907 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:29.992867 2575 generic.go:356] "Generic (PLEG): container finished" podID="9aeb0af4-1773-4b88-b17f-d1d4d6490101" containerID="1c648ae0aa18f7025562edc83d2a1171899004e6d11fbdc65b84cc7cfb30693d" exitCode=0 Sep 07 10:08:29.993847 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:29.992972 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzg2l" event={"ID":"9aeb0af4-1773-4b88-b17f-d1d4d6490101","Type":"ContainerDied","Data":"1c648ae0aa18f7025562edc83d2a1171899004e6d11fbdc65b84cc7cfb30693d"} Sep 07 10:08:30.003911 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:30.003890 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/ovn-acl-logging/0.log" Sep 07 10:08:30.004279 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:30.004256 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" event={"ID":"f245ed6c-716b-4e7f-89bf-2db47ba2e83c","Type":"ContainerStarted","Data":"7c2e95ae0242e5ec43e3543712647a8ace2c3a6901aa686d7b9259e50e5be230"} Sep 07 10:08:30.004651 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:30.004631 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:30.004751 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:30.004710 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:30.004834 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:30.004755 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:30.020352 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:30.020329 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:30.020863 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:30.020844 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:08:30.043994 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:30.043945 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" podStartSLOduration=8.5307638 podStartE2EDuration="27.043929882s" totalImagesPullingTime="18.513166082s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:30.043265819 +0000 UTC m=+27.755507764" watchObservedRunningTime="2026-09-07 10:08:30.043929882 +0000 UTC m=+27.756171828" Sep 07 10:08:30.574768 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:30.574550 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6v64l"] Sep 07 10:08:30.574967 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:30.574932 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:30.575066 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:30.575040 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6v64l" podUID="cff88048-74be-46c2-a8ac-113c5fc3a85d" Sep 07 10:08:30.575238 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:30.575217 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-km29h"] Sep 07 10:08:30.575315 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:30.575305 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:30.575384 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:30.575370 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km29h" podUID="d4e7d380-df30-4ee5-9059-6fd7fb6e82b7" Sep 07 10:08:30.576758 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:30.576736 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4wfhx"] Sep 07 10:08:30.576890 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:30.576844 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:30.576954 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:30.576924 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wfhx" podUID="d11a5179-d3c5-48d5-a7c6-a7c367fd2584" Sep 07 10:08:32.010290 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:32.010254 2575 generic.go:356] "Generic (PLEG): container finished" podID="9aeb0af4-1773-4b88-b17f-d1d4d6490101" containerID="4c557aff57dc86d403f5a0f1b0f146d5d472fdd52c67821e112287b7db010003" exitCode=0 Sep 07 10:08:32.010919 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:32.010338 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzg2l" event={"ID":"9aeb0af4-1773-4b88-b17f-d1d4d6490101","Type":"ContainerDied","Data":"4c557aff57dc86d403f5a0f1b0f146d5d472fdd52c67821e112287b7db010003"} Sep 07 10:08:32.829554 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:32.828649 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:32.829554 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:32.828918 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wfhx" podUID="d11a5179-d3c5-48d5-a7c6-a7c367fd2584" Sep 07 10:08:32.829824 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:32.829601 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:32.829824 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:32.829754 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:32.829907 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:32.829753 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km29h" podUID="d4e7d380-df30-4ee5-9059-6fd7fb6e82b7" Sep 07 10:08:32.830642 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:32.830612 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6v64l" podUID="cff88048-74be-46c2-a8ac-113c5fc3a85d" Sep 07 10:08:33.015339 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:33.015306 2575 generic.go:356] "Generic (PLEG): container finished" podID="9aeb0af4-1773-4b88-b17f-d1d4d6490101" containerID="0d3c81f622cbbd142665c99275f994f10236247df6f3e458d59d12d4ed052fef" exitCode=0 Sep 07 10:08:33.015661 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:33.015351 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzg2l" event={"ID":"9aeb0af4-1773-4b88-b17f-d1d4d6490101","Type":"ContainerDied","Data":"0d3c81f622cbbd142665c99275f994f10236247df6f3e458d59d12d4ed052fef"} Sep 07 10:08:34.826255 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:34.826217 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:34.826700 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:34.826217 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:34.826700 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:34.826360 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-km29h" podUID="d4e7d380-df30-4ee5-9059-6fd7fb6e82b7" Sep 07 10:08:34.826700 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:34.826469 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wfhx" podUID="d11a5179-d3c5-48d5-a7c6-a7c367fd2584" Sep 07 10:08:34.826700 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:34.826227 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:34.826700 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:34.826595 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6v64l" podUID="cff88048-74be-46c2-a8ac-113c5fc3a85d" Sep 07 10:08:36.455283 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.455240 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret\") pod \"global-pull-secret-syncer-6v64l\" (UID: \"cff88048-74be-46c2-a8ac-113c5fc3a85d\") " pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:36.455790 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:36.455430 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:36.455790 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:36.455501 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret podName:cff88048-74be-46c2-a8ac-113c5fc3a85d nodeName:}" failed. No retries permitted until 2026-09-07 10:09:08.455483272 +0000 UTC m=+66.167725208 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret") pod "global-pull-secret-syncer-6v64l" (UID: "cff88048-74be-46c2-a8ac-113c5fc3a85d") : object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:36.556959 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.556705 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs\") pod \"network-metrics-daemon-4wfhx\" (UID: \"d11a5179-d3c5-48d5-a7c6-a7c367fd2584\") " pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:36.557118 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:36.556896 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:36.557118 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:36.557065 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs podName:d11a5179-d3c5-48d5-a7c6-a7c367fd2584 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:08.557047523 +0000 UTC m=+66.269289445 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs") pod "network-metrics-daemon-4wfhx" (UID: "d11a5179-d3c5-48d5-a7c6-a7c367fd2584") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:36.598591 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.598556 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-85.ec2.internal" event="NodeReady" Sep 07 10:08:36.598748 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.598699 2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 07 10:08:36.647806 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.647768 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nc7j5"] Sep 07 10:08:36.657765 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.657687 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2s2g\" (UniqueName: \"kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g\") pod \"network-check-target-km29h\" (UID: \"d4e7d380-df30-4ee5-9059-6fd7fb6e82b7\") " pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:36.657954 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:36.657896 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 10:08:36.657954 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:36.657919 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 10:08:36.657954 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:36.657931 2575 projected.go:196] Error preparing data for projected volume kube-api-access-r2s2g for pod openshift-network-diagnostics/network-check-target-km29h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:36.658089 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:36.657990 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g podName:d4e7d380-df30-4ee5-9059-6fd7fb6e82b7 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:08.65797237 +0000 UTC m=+66.370214305 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-r2s2g" (UniqueName: "kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g") pod "network-check-target-km29h" (UID: "d4e7d380-df30-4ee5-9059-6fd7fb6e82b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:36.676801 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.676761 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jfq75"] Sep 07 10:08:36.676954 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.676904 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nc7j5" Sep 07 10:08:36.679405 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.679380 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 07 10:08:36.679531 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.679380 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 07 10:08:36.679531 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.679515 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5dlqq\"" Sep 07 10:08:36.696775 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.696748 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jfq75"] Sep 07 10:08:36.696939 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.696789 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nc7j5"] Sep 07 10:08:36.696939 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.696876 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jfq75" Sep 07 10:08:36.699122 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.699010 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 07 10:08:36.699122 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.699009 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 07 10:08:36.699122 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.699063 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 07 10:08:36.699362 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.699175 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-926dj\"" Sep 07 10:08:36.758265 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.758172 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2fth\" (UniqueName: \"kubernetes.io/projected/b07b7355-ed97-4a04-958b-c675ec096d60-kube-api-access-q2fth\") pod \"dns-default-nc7j5\" (UID: \"b07b7355-ed97-4a04-958b-c675ec096d60\") " pod="openshift-dns/dns-default-nc7j5" Sep 07 10:08:36.758439 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.758317 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b07b7355-ed97-4a04-958b-c675ec096d60-config-volume\") pod \"dns-default-nc7j5\" (UID: \"b07b7355-ed97-4a04-958b-c675ec096d60\") " pod="openshift-dns/dns-default-nc7j5" Sep 07 10:08:36.758439 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.758387 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls\") pod \"dns-default-nc7j5\" (UID: \"b07b7355-ed97-4a04-958b-c675ec096d60\") " pod="openshift-dns/dns-default-nc7j5" Sep 07 10:08:36.758439 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.758435 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b07b7355-ed97-4a04-958b-c675ec096d60-tmp-dir\") pod \"dns-default-nc7j5\" (UID: \"b07b7355-ed97-4a04-958b-c675ec096d60\") " pod="openshift-dns/dns-default-nc7j5" Sep 07 10:08:36.825892 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.825820 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:08:36.825892 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.825836 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:08:36.825892 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.825869 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:08:36.828700 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.828677 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2frx2\"" Sep 07 10:08:36.828920 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.828682 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5tggq\"" Sep 07 10:08:36.828920 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.828893 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 07 10:08:36.829047 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.829015 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 07 10:08:36.829178 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.829124 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 07 10:08:36.829270 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.829206 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 07 10:08:36.859631 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.859599 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b07b7355-ed97-4a04-958b-c675ec096d60-tmp-dir\") pod \"dns-default-nc7j5\" (UID: \"b07b7355-ed97-4a04-958b-c675ec096d60\") " pod="openshift-dns/dns-default-nc7j5" Sep 07 10:08:36.859763 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.859657 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2fth\" (UniqueName: \"kubernetes.io/projected/b07b7355-ed97-4a04-958b-c675ec096d60-kube-api-access-q2fth\") pod \"dns-default-nc7j5\" (UID: \"b07b7355-ed97-4a04-958b-c675ec096d60\") " pod="openshift-dns/dns-default-nc7j5" Sep 07 10:08:36.859763 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.859687 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4jsq\" (UniqueName: \"kubernetes.io/projected/1352b671-2ab8-4c45-8f10-f82960e1ff1c-kube-api-access-k4jsq\") pod \"ingress-canary-jfq75\" (UID: \"1352b671-2ab8-4c45-8f10-f82960e1ff1c\") " pod="openshift-ingress-canary/ingress-canary-jfq75" Sep 07 10:08:36.859763 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.859741 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b07b7355-ed97-4a04-958b-c675ec096d60-config-volume\") pod \"dns-default-nc7j5\" (UID: \"b07b7355-ed97-4a04-958b-c675ec096d60\") " pod="openshift-dns/dns-default-nc7j5" Sep 07 10:08:36.859936 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.859773 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1352b671-2ab8-4c45-8f10-f82960e1ff1c-cert\") pod \"ingress-canary-jfq75\" (UID: \"1352b671-2ab8-4c45-8f10-f82960e1ff1c\") " pod="openshift-ingress-canary/ingress-canary-jfq75" Sep 07 10:08:36.859936 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.859820 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls\") pod \"dns-default-nc7j5\" (UID: \"b07b7355-ed97-4a04-958b-c675ec096d60\") " pod="openshift-dns/dns-default-nc7j5" Sep 07 10:08:36.859936 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:36.859905 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 10:08:36.860066 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:36.859958 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls podName:b07b7355-ed97-4a04-958b-c675ec096d60 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:37.359940721 +0000 UTC m=+35.072182645 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls") pod "dns-default-nc7j5" (UID: "b07b7355-ed97-4a04-958b-c675ec096d60") : secret "dns-default-metrics-tls" not found Sep 07 10:08:36.860066 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.859990 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b07b7355-ed97-4a04-958b-c675ec096d60-tmp-dir\") pod \"dns-default-nc7j5\" (UID: \"b07b7355-ed97-4a04-958b-c675ec096d60\") " pod="openshift-dns/dns-default-nc7j5" Sep 07 10:08:36.860296 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.860280 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b07b7355-ed97-4a04-958b-c675ec096d60-config-volume\") pod \"dns-default-nc7j5\" (UID: \"b07b7355-ed97-4a04-958b-c675ec096d60\") " pod="openshift-dns/dns-default-nc7j5" Sep 07 10:08:36.869697 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.869671 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2fth\" (UniqueName: \"kubernetes.io/projected/b07b7355-ed97-4a04-958b-c675ec096d60-kube-api-access-q2fth\") pod \"dns-default-nc7j5\" (UID: \"b07b7355-ed97-4a04-958b-c675ec096d60\") " pod="openshift-dns/dns-default-nc7j5" Sep 07 10:08:36.960744 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.960704 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4jsq\" (UniqueName: \"kubernetes.io/projected/1352b671-2ab8-4c45-8f10-f82960e1ff1c-kube-api-access-k4jsq\") pod \"ingress-canary-jfq75\" (UID: \"1352b671-2ab8-4c45-8f10-f82960e1ff1c\") " pod="openshift-ingress-canary/ingress-canary-jfq75" Sep 07 10:08:36.960939 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.960784 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1352b671-2ab8-4c45-8f10-f82960e1ff1c-cert\") pod \"ingress-canary-jfq75\" (UID: \"1352b671-2ab8-4c45-8f10-f82960e1ff1c\") " pod="openshift-ingress-canary/ingress-canary-jfq75" Sep 07 10:08:36.963652 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.963624 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1352b671-2ab8-4c45-8f10-f82960e1ff1c-cert\") pod \"ingress-canary-jfq75\" (UID: \"1352b671-2ab8-4c45-8f10-f82960e1ff1c\") " pod="openshift-ingress-canary/ingress-canary-jfq75" Sep 07 10:08:36.968966 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:36.968941 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4jsq\" (UniqueName: \"kubernetes.io/projected/1352b671-2ab8-4c45-8f10-f82960e1ff1c-kube-api-access-k4jsq\") pod \"ingress-canary-jfq75\" (UID: \"1352b671-2ab8-4c45-8f10-f82960e1ff1c\") " pod="openshift-ingress-canary/ingress-canary-jfq75" Sep 07 10:08:37.006906 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:37.006873 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jfq75" Sep 07 10:08:37.364147 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:37.364063 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls\") pod \"dns-default-nc7j5\" (UID: \"b07b7355-ed97-4a04-958b-c675ec096d60\") " pod="openshift-dns/dns-default-nc7j5" Sep 07 10:08:37.364317 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:37.364257 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 10:08:37.364378 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:37.364334 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls podName:b07b7355-ed97-4a04-958b-c675ec096d60 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:38.364313212 +0000 UTC m=+36.076555149 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls") pod "dns-default-nc7j5" (UID: "b07b7355-ed97-4a04-958b-c675ec096d60") : secret "dns-default-metrics-tls" not found Sep 07 10:08:37.766861 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:37.766757 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jfq75"] Sep 07 10:08:37.771016 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:08:37.770989 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1352b671_2ab8_4c45_8f10_f82960e1ff1c.slice/crio-89af88ef39067418feac7a035e5e5214e80bf0eb7f7e6806c0e3371f267a0145 WatchSource:0}: Error finding container 89af88ef39067418feac7a035e5e5214e80bf0eb7f7e6806c0e3371f267a0145: Status 404 returned error can't find the container with id 89af88ef39067418feac7a035e5e5214e80bf0eb7f7e6806c0e3371f267a0145 Sep 07 10:08:38.031437 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:38.030923 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jfq75" event={"ID":"1352b671-2ab8-4c45-8f10-f82960e1ff1c","Type":"ContainerStarted","Data":"89af88ef39067418feac7a035e5e5214e80bf0eb7f7e6806c0e3371f267a0145"} Sep 07 10:08:38.034316 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:38.034285 2575 generic.go:356] "Generic (PLEG): container finished" podID="9aeb0af4-1773-4b88-b17f-d1d4d6490101" containerID="aba94af69330b5d37f68e814297ab54291b0800f05a47307de77f33fe24a9e64" exitCode=0 Sep 07 10:08:38.034458 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:38.034328 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzg2l" event={"ID":"9aeb0af4-1773-4b88-b17f-d1d4d6490101","Type":"ContainerDied","Data":"aba94af69330b5d37f68e814297ab54291b0800f05a47307de77f33fe24a9e64"} Sep 07 10:08:38.370132 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:38.370037 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls\") pod \"dns-default-nc7j5\" (UID: \"b07b7355-ed97-4a04-958b-c675ec096d60\") " pod="openshift-dns/dns-default-nc7j5" Sep 07 10:08:38.370279 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:38.370203 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 10:08:38.370279 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:38.370270 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls podName:b07b7355-ed97-4a04-958b-c675ec096d60 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:40.370252851 +0000 UTC m=+38.082494779 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls") pod "dns-default-nc7j5" (UID: "b07b7355-ed97-4a04-958b-c675ec096d60") : secret "dns-default-metrics-tls" not found Sep 07 10:08:39.040245 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:39.040010 2575 generic.go:356] "Generic (PLEG): container finished" podID="9aeb0af4-1773-4b88-b17f-d1d4d6490101" containerID="a4e18e37747cb83375f49baf10c9b2fb676ac525d0b3e6a8cb75c610d12751a2" exitCode=0 Sep 07 10:08:39.040784 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:39.040101 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzg2l" event={"ID":"9aeb0af4-1773-4b88-b17f-d1d4d6490101","Type":"ContainerDied","Data":"a4e18e37747cb83375f49baf10c9b2fb676ac525d0b3e6a8cb75c610d12751a2"} Sep 07 10:08:40.046967 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:40.046917 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzg2l" event={"ID":"9aeb0af4-1773-4b88-b17f-d1d4d6490101","Type":"ContainerStarted","Data":"bfed7d42efe9045d1d36f794ee09e141a7a3ab31c1701caedeca613fe556d4b0"} Sep 07 10:08:40.069308 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:40.069245 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nzg2l" podStartSLOduration=14.314977411 podStartE2EDuration="37.069228527s" totalImagesPullingTime="22.754251116s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:40.067954464 +0000 UTC m=+37.780196433" watchObservedRunningTime="2026-09-07 10:08:40.069228527 +0000 UTC m=+37.781470472" Sep 07 10:08:40.388090 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:40.387989 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls\") pod \"dns-default-nc7j5\" (UID: \"b07b7355-ed97-4a04-958b-c675ec096d60\") " pod="openshift-dns/dns-default-nc7j5" Sep 07 10:08:40.388262 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:40.388146 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 10:08:40.388262 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:40.388213 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls podName:b07b7355-ed97-4a04-958b-c675ec096d60 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:44.388196626 +0000 UTC m=+42.100438549 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls") pod "dns-default-nc7j5" (UID: "b07b7355-ed97-4a04-958b-c675ec096d60") : secret "dns-default-metrics-tls" not found Sep 07 10:08:41.051049 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:41.051007 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jfq75" event={"ID":"1352b671-2ab8-4c45-8f10-f82960e1ff1c","Type":"ContainerStarted","Data":"7e52bf21261f3b1ac30eec49e86c3514c4d5448d39b3863bd3d2e0ae9de56302"} Sep 07 10:08:41.067666 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:41.067616 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jfq75" podStartSLOduration=2.6773722170000003 podStartE2EDuration="5.067603198s" totalImagesPullingTime="2.390230981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:08:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:41.066891247 +0000 UTC m=+38.779133192" watchObservedRunningTime="2026-09-07 10:08:41.067603198 +0000 UTC m=+38.779845142" Sep 07 10:08:44.422527 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:44.422482 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls\") pod \"dns-default-nc7j5\" (UID: \"b07b7355-ed97-4a04-958b-c675ec096d60\") " pod="openshift-dns/dns-default-nc7j5" Sep 07 10:08:44.423033 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:44.422622 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 10:08:44.423033 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:44.422696 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls podName:b07b7355-ed97-4a04-958b-c675ec096d60 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:52.422675499 +0000 UTC m=+50.134917425 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls") pod "dns-default-nc7j5" (UID: "b07b7355-ed97-4a04-958b-c675ec096d60") : secret "dns-default-metrics-tls" not found Sep 07 10:08:52.479228 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:08:52.479188 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls\") pod \"dns-default-nc7j5\" (UID: \"b07b7355-ed97-4a04-958b-c675ec096d60\") " pod="openshift-dns/dns-default-nc7j5" Sep 07 10:08:52.479649 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:52.479309 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 10:08:52.479649 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:08:52.479361 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls podName:b07b7355-ed97-4a04-958b-c675ec096d60 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:08.47934842 +0000 UTC m=+66.191590356 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls") pod "dns-default-nc7j5" (UID: "b07b7355-ed97-4a04-958b-c675ec096d60") : secret "dns-default-metrics-tls" not found Sep 07 10:09:02.025450 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:02.025419 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-f6mnj" Sep 07 10:09:08.487163 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:08.487106 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret\") pod \"global-pull-secret-syncer-6v64l\" (UID: \"cff88048-74be-46c2-a8ac-113c5fc3a85d\") " pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:09:08.487629 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:08.487184 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls\") pod \"dns-default-nc7j5\" (UID: \"b07b7355-ed97-4a04-958b-c675ec096d60\") " pod="openshift-dns/dns-default-nc7j5" Sep 07 10:09:08.487629 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:08.487296 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 10:09:08.487629 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:08.487365 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls podName:b07b7355-ed97-4a04-958b-c675ec096d60 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:40.487347182 +0000 UTC m=+98.199589127 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls") pod "dns-default-nc7j5" (UID: "b07b7355-ed97-4a04-958b-c675ec096d60") : secret "dns-default-metrics-tls" not found Sep 07 10:09:08.489837 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:08.489817 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 07 10:09:08.501509 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:08.501473 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cff88048-74be-46c2-a8ac-113c5fc3a85d-original-pull-secret\") pod \"global-pull-secret-syncer-6v64l\" (UID: \"cff88048-74be-46c2-a8ac-113c5fc3a85d\") " pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:09:08.588341 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:08.588300 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs\") pod \"network-metrics-daemon-4wfhx\" (UID: \"d11a5179-d3c5-48d5-a7c6-a7c367fd2584\") " pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:09:08.591009 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:08.590991 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 07 10:09:08.599118 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:08.599097 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 07 10:09:08.599183 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:08.599173 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs podName:d11a5179-d3c5-48d5-a7c6-a7c367fd2584 nodeName:}" failed. No retries permitted until 2026-09-07 10:10:12.599155707 +0000 UTC m=+130.311397629 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs") pod "network-metrics-daemon-4wfhx" (UID: "d11a5179-d3c5-48d5-a7c6-a7c367fd2584") : secret "metrics-daemon-secret" not found Sep 07 10:09:08.638221 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:08.638187 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6v64l" Sep 07 10:09:08.689059 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:08.689029 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2s2g\" (UniqueName: \"kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g\") pod \"network-check-target-km29h\" (UID: \"d4e7d380-df30-4ee5-9059-6fd7fb6e82b7\") " pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:09:08.691540 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:08.691519 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 07 10:09:08.701997 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:08.701970 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 07 10:09:08.712167 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:08.712140 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2s2g\" (UniqueName: \"kubernetes.io/projected/d4e7d380-df30-4ee5-9059-6fd7fb6e82b7-kube-api-access-r2s2g\") pod \"network-check-target-km29h\" (UID: \"d4e7d380-df30-4ee5-9059-6fd7fb6e82b7\") " pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:09:08.947580 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:08.947427 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6v64l"] Sep 07 10:09:08.950182 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:09:08.950151 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcff88048_74be_46c2_a8ac_113c5fc3a85d.slice/crio-9ddec8b4c959b0260c24b166ac29c032483b56d436b0833785013ae6a0f35912 WatchSource:0}: Error finding container 9ddec8b4c959b0260c24b166ac29c032483b56d436b0833785013ae6a0f35912: Status 404 returned error can't find the container with id 9ddec8b4c959b0260c24b166ac29c032483b56d436b0833785013ae6a0f35912 Sep 07 10:09:08.952105 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:08.952087 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5tggq\"" Sep 07 10:09:08.960535 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:08.960514 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:09:09.117758 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:09.117727 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6v64l" event={"ID":"cff88048-74be-46c2-a8ac-113c5fc3a85d","Type":"ContainerStarted","Data":"9ddec8b4c959b0260c24b166ac29c032483b56d436b0833785013ae6a0f35912"} Sep 07 10:09:09.264458 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:09.264380 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-km29h"] Sep 07 10:09:09.267966 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:09:09.267940 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4e7d380_df30_4ee5_9059_6fd7fb6e82b7.slice/crio-17ae63fee851e14e0624c6c9c0c2bb30afe87933d1abf4ec7b35fa47073d45a5 WatchSource:0}: Error finding container 17ae63fee851e14e0624c6c9c0c2bb30afe87933d1abf4ec7b35fa47073d45a5: Status 404 returned error can't find the container with id 17ae63fee851e14e0624c6c9c0c2bb30afe87933d1abf4ec7b35fa47073d45a5 Sep 07 10:09:10.122891 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:10.122837 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-km29h" event={"ID":"d4e7d380-df30-4ee5-9059-6fd7fb6e82b7","Type":"ContainerStarted","Data":"17ae63fee851e14e0624c6c9c0c2bb30afe87933d1abf4ec7b35fa47073d45a5"} Sep 07 10:09:14.134627 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:14.134530 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6v64l" event={"ID":"cff88048-74be-46c2-a8ac-113c5fc3a85d","Type":"ContainerStarted","Data":"4fd73e87d324ecb3d2f87078f0891760c19d31efa73bf66143edc2e6774b25d1"} Sep 07 10:09:14.136004 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:14.135978 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-km29h" event={"ID":"d4e7d380-df30-4ee5-9059-6fd7fb6e82b7","Type":"ContainerStarted","Data":"d17539bda52f77d6b5ffe0d5740ec54861bd4b91418d2e0ca73e2508f8e480c8"} Sep 07 10:09:14.136125 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:14.136107 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:09:14.149045 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:14.148994 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6v64l" podStartSLOduration=66.26811821 podStartE2EDuration="1m11.148974936s" totalImagesPullingTime="4.880856726s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:09:14.148833675 +0000 UTC m=+71.861075620" watchObservedRunningTime="2026-09-07 10:09:14.148974936 +0000 UTC m=+71.861216882" Sep 07 10:09:14.163170 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:14.163122 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-km29h" podStartSLOduration=66.600025961 podStartE2EDuration="1m11.163105772s" totalImagesPullingTime="4.563079811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:09:14.162468062 +0000 UTC m=+71.874710008" watchObservedRunningTime="2026-09-07 10:09:14.163105772 +0000 UTC m=+71.875347716" Sep 07 10:09:29.966880 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:29.966724 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz"] Sep 07 10:09:29.971078 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:29.971056 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6c558849d-hzpf7"] Sep 07 10:09:29.971200 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:29.971173 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz" Sep 07 10:09:29.973635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:29.973616 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:29.974439 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:29.974417 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 07 10:09:29.974525 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:29.974469 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 07 10:09:29.974525 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:29.974490 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-sht2z\"" Sep 07 10:09:29.974610 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:29.974549 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 07 10:09:29.974610 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:29.974556 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 07 10:09:29.976338 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:29.976314 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 07 10:09:29.976449 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:29.976409 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 07 10:09:29.976449 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:29.976442 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 07 10:09:29.976565 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:29.976553 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2wqlc\"" Sep 07 10:09:29.977316 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:29.977298 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 07 10:09:29.977510 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:29.977493 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 07 10:09:29.977750 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:29.977731 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 07 10:09:29.980586 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:29.980559 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz"] Sep 07 10:09:29.997367 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:29.997341 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6c558849d-hzpf7"] Sep 07 10:09:30.034035 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.034007 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/502e0127-90b6-4388-860b-5dd7979db869-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-dx9lz\" (UID: \"502e0127-90b6-4388-860b-5dd7979db869\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz" Sep 07 10:09:30.034189 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.034043 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pjcp\" (UniqueName: \"kubernetes.io/projected/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-kube-api-access-4pjcp\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:30.034189 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.034125 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/502e0127-90b6-4388-860b-5dd7979db869-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-dx9lz\" (UID: \"502e0127-90b6-4388-860b-5dd7979db869\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz" Sep 07 10:09:30.034189 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.034164 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-default-certificate\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:30.034309 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.034203 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59ffg\" (UniqueName: \"kubernetes.io/projected/502e0127-90b6-4388-860b-5dd7979db869-kube-api-access-59ffg\") pod \"cluster-monitoring-operator-6cdcc96965-dx9lz\" (UID: \"502e0127-90b6-4388-860b-5dd7979db869\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz" Sep 07 10:09:30.034309 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.034224 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-service-ca-bundle\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:30.034309 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.034240 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-stats-auth\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:30.034309 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.034261 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-metrics-certs\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:30.134762 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.134726 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/502e0127-90b6-4388-860b-5dd7979db869-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-dx9lz\" (UID: \"502e0127-90b6-4388-860b-5dd7979db869\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz" Sep 07 10:09:30.134762 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.134763 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-default-certificate\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:30.135018 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.134821 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59ffg\" (UniqueName: \"kubernetes.io/projected/502e0127-90b6-4388-860b-5dd7979db869-kube-api-access-59ffg\") pod \"cluster-monitoring-operator-6cdcc96965-dx9lz\" (UID: \"502e0127-90b6-4388-860b-5dd7979db869\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz" Sep 07 10:09:30.135018 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.134850 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-service-ca-bundle\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:30.135018 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.134872 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-stats-auth\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:30.135018 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.134896 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-metrics-certs\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:30.135018 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:30.134914 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 07 10:09:30.135018 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.134929 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/502e0127-90b6-4388-860b-5dd7979db869-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-dx9lz\" (UID: \"502e0127-90b6-4388-860b-5dd7979db869\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz" Sep 07 10:09:30.135018 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:30.134981 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/502e0127-90b6-4388-860b-5dd7979db869-cluster-monitoring-operator-tls podName:502e0127-90b6-4388-860b-5dd7979db869 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:30.63496158 +0000 UTC m=+88.347203519 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/502e0127-90b6-4388-860b-5dd7979db869-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-dx9lz" (UID: "502e0127-90b6-4388-860b-5dd7979db869") : secret "cluster-monitoring-operator-tls" not found Sep 07 10:09:30.135018 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.134996 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pjcp\" (UniqueName: \"kubernetes.io/projected/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-kube-api-access-4pjcp\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:30.135386 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:30.135024 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 07 10:09:30.135386 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:30.135059 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-service-ca-bundle podName:c3c51a39-edc3-44e7-a0ed-843ac7ec5059 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:30.635040074 +0000 UTC m=+88.347282001 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-service-ca-bundle") pod "router-default-6c558849d-hzpf7" (UID: "c3c51a39-edc3-44e7-a0ed-843ac7ec5059") : configmap references non-existent config key: service-ca.crt Sep 07 10:09:30.135386 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:30.135098 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-metrics-certs podName:c3c51a39-edc3-44e7-a0ed-843ac7ec5059 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:30.635086458 +0000 UTC m=+88.347328393 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-metrics-certs") pod "router-default-6c558849d-hzpf7" (UID: "c3c51a39-edc3-44e7-a0ed-843ac7ec5059") : secret "router-metrics-certs-default" not found Sep 07 10:09:30.135639 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.135622 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/502e0127-90b6-4388-860b-5dd7979db869-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-dx9lz\" (UID: \"502e0127-90b6-4388-860b-5dd7979db869\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz" Sep 07 10:09:30.137354 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.137329 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-stats-auth\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:30.137444 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.137384 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-default-certificate\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:30.143401 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.143370 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59ffg\" (UniqueName: \"kubernetes.io/projected/502e0127-90b6-4388-860b-5dd7979db869-kube-api-access-59ffg\") pod \"cluster-monitoring-operator-6cdcc96965-dx9lz\" (UID: \"502e0127-90b6-4388-860b-5dd7979db869\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz" Sep 07 10:09:30.143518 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.143503 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pjcp\" (UniqueName: \"kubernetes.io/projected/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-kube-api-access-4pjcp\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:30.168661 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.168623 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8bl4l"] Sep 07 10:09:30.171783 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.171760 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-m6mmt"] Sep 07 10:09:30.171922 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.171901 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8bl4l" Sep 07 10:09:30.174224 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.174202 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 07 10:09:30.174324 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.174201 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 07 10:09:30.174324 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.174256 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 07 10:09:30.174549 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.174534 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-m6mmt" Sep 07 10:09:30.174995 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.174979 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-b82wf\"" Sep 07 10:09:30.175120 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.175102 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 07 10:09:30.177584 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.176736 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 07 10:09:30.178274 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.177755 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 07 10:09:30.178274 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.177919 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 07 10:09:30.178274 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.178211 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 07 10:09:30.179964 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.179947 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-prrqt\"" Sep 07 10:09:30.182860 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.182835 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-m6mmt"] Sep 07 10:09:30.184106 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.184084 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8bl4l"] Sep 07 10:09:30.184711 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.184695 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 07 10:09:30.236236 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.236150 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfbqj\" (UniqueName: \"kubernetes.io/projected/70c229fe-b5da-413c-b231-625155198200-kube-api-access-tfbqj\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8bl4l\" (UID: \"70c229fe-b5da-413c-b231-625155198200\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8bl4l" Sep 07 10:09:30.236236 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.236186 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sjts\" (UniqueName: \"kubernetes.io/projected/6aad73e9-87ee-49b0-8651-d3fd8390e4e4-kube-api-access-6sjts\") pod \"console-operator-85b454874b-m6mmt\" (UID: \"6aad73e9-87ee-49b0-8651-d3fd8390e4e4\") " pod="openshift-console-operator/console-operator-85b454874b-m6mmt" Sep 07 10:09:30.236236 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.236208 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6aad73e9-87ee-49b0-8651-d3fd8390e4e4-trusted-ca\") pod \"console-operator-85b454874b-m6mmt\" (UID: \"6aad73e9-87ee-49b0-8651-d3fd8390e4e4\") " pod="openshift-console-operator/console-operator-85b454874b-m6mmt" Sep 07 10:09:30.236236 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.236225 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/70c229fe-b5da-413c-b231-625155198200-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8bl4l\" (UID: \"70c229fe-b5da-413c-b231-625155198200\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8bl4l" Sep 07 10:09:30.236471 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.236304 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/70c229fe-b5da-413c-b231-625155198200-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8bl4l\" (UID: \"70c229fe-b5da-413c-b231-625155198200\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8bl4l" Sep 07 10:09:30.236471 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.236333 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6aad73e9-87ee-49b0-8651-d3fd8390e4e4-config\") pod \"console-operator-85b454874b-m6mmt\" (UID: \"6aad73e9-87ee-49b0-8651-d3fd8390e4e4\") " pod="openshift-console-operator/console-operator-85b454874b-m6mmt" Sep 07 10:09:30.236471 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.236349 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6aad73e9-87ee-49b0-8651-d3fd8390e4e4-serving-cert\") pod \"console-operator-85b454874b-m6mmt\" (UID: \"6aad73e9-87ee-49b0-8651-d3fd8390e4e4\") " pod="openshift-console-operator/console-operator-85b454874b-m6mmt" Sep 07 10:09:30.337405 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.337364 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfbqj\" (UniqueName: \"kubernetes.io/projected/70c229fe-b5da-413c-b231-625155198200-kube-api-access-tfbqj\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8bl4l\" (UID: \"70c229fe-b5da-413c-b231-625155198200\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8bl4l" Sep 07 10:09:30.337405 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.337405 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sjts\" (UniqueName: \"kubernetes.io/projected/6aad73e9-87ee-49b0-8651-d3fd8390e4e4-kube-api-access-6sjts\") pod \"console-operator-85b454874b-m6mmt\" (UID: \"6aad73e9-87ee-49b0-8651-d3fd8390e4e4\") " pod="openshift-console-operator/console-operator-85b454874b-m6mmt" Sep 07 10:09:30.337649 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.337431 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6aad73e9-87ee-49b0-8651-d3fd8390e4e4-trusted-ca\") pod \"console-operator-85b454874b-m6mmt\" (UID: \"6aad73e9-87ee-49b0-8651-d3fd8390e4e4\") " pod="openshift-console-operator/console-operator-85b454874b-m6mmt" Sep 07 10:09:30.337649 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.337455 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/70c229fe-b5da-413c-b231-625155198200-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8bl4l\" (UID: \"70c229fe-b5da-413c-b231-625155198200\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8bl4l" Sep 07 10:09:30.337649 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.337486 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/70c229fe-b5da-413c-b231-625155198200-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8bl4l\" (UID: \"70c229fe-b5da-413c-b231-625155198200\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8bl4l" Sep 07 10:09:30.337649 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.337509 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6aad73e9-87ee-49b0-8651-d3fd8390e4e4-config\") pod \"console-operator-85b454874b-m6mmt\" (UID: \"6aad73e9-87ee-49b0-8651-d3fd8390e4e4\") " pod="openshift-console-operator/console-operator-85b454874b-m6mmt" Sep 07 10:09:30.337649 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.337531 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6aad73e9-87ee-49b0-8651-d3fd8390e4e4-serving-cert\") pod \"console-operator-85b454874b-m6mmt\" (UID: \"6aad73e9-87ee-49b0-8651-d3fd8390e4e4\") " pod="openshift-console-operator/console-operator-85b454874b-m6mmt" Sep 07 10:09:30.338203 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.338179 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/70c229fe-b5da-413c-b231-625155198200-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8bl4l\" (UID: \"70c229fe-b5da-413c-b231-625155198200\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8bl4l" Sep 07 10:09:30.338316 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.338297 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6aad73e9-87ee-49b0-8651-d3fd8390e4e4-config\") pod \"console-operator-85b454874b-m6mmt\" (UID: \"6aad73e9-87ee-49b0-8651-d3fd8390e4e4\") " pod="openshift-console-operator/console-operator-85b454874b-m6mmt" Sep 07 10:09:30.338377 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.338361 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6aad73e9-87ee-49b0-8651-d3fd8390e4e4-trusted-ca\") pod \"console-operator-85b454874b-m6mmt\" (UID: \"6aad73e9-87ee-49b0-8651-d3fd8390e4e4\") " pod="openshift-console-operator/console-operator-85b454874b-m6mmt" Sep 07 10:09:30.339835 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.339788 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/70c229fe-b5da-413c-b231-625155198200-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8bl4l\" (UID: \"70c229fe-b5da-413c-b231-625155198200\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8bl4l" Sep 07 10:09:30.339835 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.339823 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6aad73e9-87ee-49b0-8651-d3fd8390e4e4-serving-cert\") pod \"console-operator-85b454874b-m6mmt\" (UID: \"6aad73e9-87ee-49b0-8651-d3fd8390e4e4\") " pod="openshift-console-operator/console-operator-85b454874b-m6mmt" Sep 07 10:09:30.344787 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.344767 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfbqj\" (UniqueName: \"kubernetes.io/projected/70c229fe-b5da-413c-b231-625155198200-kube-api-access-tfbqj\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8bl4l\" (UID: \"70c229fe-b5da-413c-b231-625155198200\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8bl4l" Sep 07 10:09:30.345077 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.345060 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sjts\" (UniqueName: \"kubernetes.io/projected/6aad73e9-87ee-49b0-8651-d3fd8390e4e4-kube-api-access-6sjts\") pod \"console-operator-85b454874b-m6mmt\" (UID: \"6aad73e9-87ee-49b0-8651-d3fd8390e4e4\") " pod="openshift-console-operator/console-operator-85b454874b-m6mmt" Sep 07 10:09:30.486415 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.486326 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8bl4l" Sep 07 10:09:30.491545 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.491526 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-m6mmt" Sep 07 10:09:30.640518 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.640472 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-service-ca-bundle\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:30.640518 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.640521 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-metrics-certs\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:30.640785 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.640584 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/502e0127-90b6-4388-860b-5dd7979db869-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-dx9lz\" (UID: \"502e0127-90b6-4388-860b-5dd7979db869\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz" Sep 07 10:09:30.640785 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:30.640654 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-service-ca-bundle podName:c3c51a39-edc3-44e7-a0ed-843ac7ec5059 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:31.640635465 +0000 UTC m=+89.352877414 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-service-ca-bundle") pod "router-default-6c558849d-hzpf7" (UID: "c3c51a39-edc3-44e7-a0ed-843ac7ec5059") : configmap references non-existent config key: service-ca.crt Sep 07 10:09:30.640785 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:30.640688 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 07 10:09:30.640785 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:30.640701 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 07 10:09:30.640785 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:30.640744 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-metrics-certs podName:c3c51a39-edc3-44e7-a0ed-843ac7ec5059 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:31.640730568 +0000 UTC m=+89.352972491 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-metrics-certs") pod "router-default-6c558849d-hzpf7" (UID: "c3c51a39-edc3-44e7-a0ed-843ac7ec5059") : secret "router-metrics-certs-default" not found Sep 07 10:09:30.640785 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:30.640762 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/502e0127-90b6-4388-860b-5dd7979db869-cluster-monitoring-operator-tls podName:502e0127-90b6-4388-860b-5dd7979db869 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:31.640752425 +0000 UTC m=+89.352994349 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/502e0127-90b6-4388-860b-5dd7979db869-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-dx9lz" (UID: "502e0127-90b6-4388-860b-5dd7979db869") : secret "cluster-monitoring-operator-tls" not found Sep 07 10:09:30.764631 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.764398 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8bl4l"] Sep 07 10:09:30.765135 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:30.765119 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-m6mmt"] Sep 07 10:09:30.767053 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:09:30.767018 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aad73e9_87ee_49b0_8651_d3fd8390e4e4.slice/crio-7833959d0805c7c7e651a8e3224b55943c9d17630ddbd09dca6ea676a80e4ddf WatchSource:0}: Error finding container 7833959d0805c7c7e651a8e3224b55943c9d17630ddbd09dca6ea676a80e4ddf: Status 404 returned error can't find the container with id 7833959d0805c7c7e651a8e3224b55943c9d17630ddbd09dca6ea676a80e4ddf Sep 07 10:09:30.767501 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:09:30.767478 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70c229fe_b5da_413c_b231_625155198200.slice/crio-e52a4229f7028e9a223c53ffb79997bd82ab7d8fe1341173f16c506cf85cbc49 WatchSource:0}: Error finding container e52a4229f7028e9a223c53ffb79997bd82ab7d8fe1341173f16c506cf85cbc49: Status 404 returned error can't find the container with id e52a4229f7028e9a223c53ffb79997bd82ab7d8fe1341173f16c506cf85cbc49 Sep 07 10:09:31.177674 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:31.177635 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8bl4l" event={"ID":"70c229fe-b5da-413c-b231-625155198200","Type":"ContainerStarted","Data":"e52a4229f7028e9a223c53ffb79997bd82ab7d8fe1341173f16c506cf85cbc49"} Sep 07 10:09:31.178650 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:31.178629 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-m6mmt" event={"ID":"6aad73e9-87ee-49b0-8651-d3fd8390e4e4","Type":"ContainerStarted","Data":"7833959d0805c7c7e651a8e3224b55943c9d17630ddbd09dca6ea676a80e4ddf"} Sep 07 10:09:31.648757 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:31.648638 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/502e0127-90b6-4388-860b-5dd7979db869-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-dx9lz\" (UID: \"502e0127-90b6-4388-860b-5dd7979db869\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz" Sep 07 10:09:31.648757 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:31.648715 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-service-ca-bundle\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:31.648757 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:31.648744 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-metrics-certs\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:31.649078 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:31.648840 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 07 10:09:31.649078 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:31.648867 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 07 10:09:31.649078 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:31.648933 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/502e0127-90b6-4388-860b-5dd7979db869-cluster-monitoring-operator-tls podName:502e0127-90b6-4388-860b-5dd7979db869 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:33.648910812 +0000 UTC m=+91.361152773 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/502e0127-90b6-4388-860b-5dd7979db869-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-dx9lz" (UID: "502e0127-90b6-4388-860b-5dd7979db869") : secret "cluster-monitoring-operator-tls" not found Sep 07 10:09:31.649078 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:31.648955 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-metrics-certs podName:c3c51a39-edc3-44e7-a0ed-843ac7ec5059 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:33.648944769 +0000 UTC m=+91.361186698 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-metrics-certs") pod "router-default-6c558849d-hzpf7" (UID: "c3c51a39-edc3-44e7-a0ed-843ac7ec5059") : secret "router-metrics-certs-default" not found Sep 07 10:09:31.649078 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:31.648970 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-service-ca-bundle podName:c3c51a39-edc3-44e7-a0ed-843ac7ec5059 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:33.648961446 +0000 UTC m=+91.361203375 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-service-ca-bundle") pod "router-default-6c558849d-hzpf7" (UID: "c3c51a39-edc3-44e7-a0ed-843ac7ec5059") : configmap references non-existent config key: service-ca.crt Sep 07 10:09:33.664412 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:33.664371 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/502e0127-90b6-4388-860b-5dd7979db869-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-dx9lz\" (UID: \"502e0127-90b6-4388-860b-5dd7979db869\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz" Sep 07 10:09:33.664871 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:33.664433 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-service-ca-bundle\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:33.664871 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:33.664454 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-metrics-certs\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:33.664871 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:33.664542 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 07 10:09:33.664871 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:33.664574 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 07 10:09:33.664871 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:33.664615 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-service-ca-bundle podName:c3c51a39-edc3-44e7-a0ed-843ac7ec5059 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:37.664595594 +0000 UTC m=+95.376837522 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-service-ca-bundle") pod "router-default-6c558849d-hzpf7" (UID: "c3c51a39-edc3-44e7-a0ed-843ac7ec5059") : configmap references non-existent config key: service-ca.crt Sep 07 10:09:33.664871 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:33.664637 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/502e0127-90b6-4388-860b-5dd7979db869-cluster-monitoring-operator-tls podName:502e0127-90b6-4388-860b-5dd7979db869 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:37.664630488 +0000 UTC m=+95.376872411 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/502e0127-90b6-4388-860b-5dd7979db869-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-dx9lz" (UID: "502e0127-90b6-4388-860b-5dd7979db869") : secret "cluster-monitoring-operator-tls" not found Sep 07 10:09:33.664871 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:33.664648 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-metrics-certs podName:c3c51a39-edc3-44e7-a0ed-843ac7ec5059 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:37.664642578 +0000 UTC m=+95.376884500 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-metrics-certs") pod "router-default-6c558849d-hzpf7" (UID: "c3c51a39-edc3-44e7-a0ed-843ac7ec5059") : secret "router-metrics-certs-default" not found Sep 07 10:09:33.927072 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:33.926997 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gv4pf"] Sep 07 10:09:33.930145 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:33.930126 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:33.932610 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:33.932585 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 07 10:09:33.933243 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:33.933222 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 07 10:09:33.933359 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:33.933225 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 07 10:09:33.933359 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:33.933223 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rmr27\"" Sep 07 10:09:33.933359 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:33.933223 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 07 10:09:33.942381 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:33.942354 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gv4pf"] Sep 07 10:09:34.067308 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:34.067272 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/51adbf04-234f-4a6c-8aec-c65483741ae6-data-volume\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:34.067308 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:34.067311 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51adbf04-234f-4a6c-8aec-c65483741ae6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:34.067560 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:34.067369 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/51adbf04-234f-4a6c-8aec-c65483741ae6-crio-socket\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:34.067560 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:34.067389 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51adbf04-234f-4a6c-8aec-c65483741ae6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:34.067560 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:34.067406 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdxnp\" (UniqueName: \"kubernetes.io/projected/51adbf04-234f-4a6c-8aec-c65483741ae6-kube-api-access-sdxnp\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:34.168512 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:34.168475 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/51adbf04-234f-4a6c-8aec-c65483741ae6-data-volume\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:34.168512 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:34.168513 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51adbf04-234f-4a6c-8aec-c65483741ae6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:34.168700 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:34.168621 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 10:09:34.168700 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:34.168667 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/51adbf04-234f-4a6c-8aec-c65483741ae6-crio-socket\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:34.168700 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:34.168687 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51adbf04-234f-4a6c-8aec-c65483741ae6-insights-runtime-extractor-tls podName:51adbf04-234f-4a6c-8aec-c65483741ae6 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:34.668667956 +0000 UTC m=+92.380909880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/51adbf04-234f-4a6c-8aec-c65483741ae6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gv4pf" (UID: "51adbf04-234f-4a6c-8aec-c65483741ae6") : secret "insights-runtime-extractor-tls" not found Sep 07 10:09:34.168875 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:34.168714 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51adbf04-234f-4a6c-8aec-c65483741ae6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:34.168875 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:34.168728 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/51adbf04-234f-4a6c-8aec-c65483741ae6-crio-socket\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:34.168875 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:34.168740 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdxnp\" (UniqueName: \"kubernetes.io/projected/51adbf04-234f-4a6c-8aec-c65483741ae6-kube-api-access-sdxnp\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:34.168875 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:34.168822 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/51adbf04-234f-4a6c-8aec-c65483741ae6-data-volume\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:34.168875 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:34.168833 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 10:09:34.169150 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:34.168887 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/51adbf04-234f-4a6c-8aec-c65483741ae6-kube-rbac-proxy-cm podName:51adbf04-234f-4a6c-8aec-c65483741ae6 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:34.668870421 +0000 UTC m=+92.381112356 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/51adbf04-234f-4a6c-8aec-c65483741ae6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gv4pf" (UID: "51adbf04-234f-4a6c-8aec-c65483741ae6") : configmap "kube-rbac-proxy" not found Sep 07 10:09:34.176847 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:34.176815 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdxnp\" (UniqueName: \"kubernetes.io/projected/51adbf04-234f-4a6c-8aec-c65483741ae6-kube-api-access-sdxnp\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:34.189124 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:34.189065 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-m6mmt_6aad73e9-87ee-49b0-8651-d3fd8390e4e4/console-operator/0.log" Sep 07 10:09:34.189124 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:34.189100 2575 generic.go:356] "Generic (PLEG): container finished" podID="6aad73e9-87ee-49b0-8651-d3fd8390e4e4" containerID="7328d8b927a9f265ebfc22baa4227cb265cecd8eea38bef6a520db1e60e9c630" exitCode=255 Sep 07 10:09:34.189282 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:34.189193 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-m6mmt" event={"ID":"6aad73e9-87ee-49b0-8651-d3fd8390e4e4","Type":"ContainerDied","Data":"7328d8b927a9f265ebfc22baa4227cb265cecd8eea38bef6a520db1e60e9c630"} Sep 07 10:09:34.189408 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:34.189388 2575 scope.go:117] "RemoveContainer" containerID="7328d8b927a9f265ebfc22baa4227cb265cecd8eea38bef6a520db1e60e9c630" Sep 07 10:09:34.190813 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:34.190773 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8bl4l" event={"ID":"70c229fe-b5da-413c-b231-625155198200","Type":"ContainerStarted","Data":"73e7bcbc253c28f6acf091a7e9366e080f0d2eef3574adbc1ef40fba58debd73"} Sep 07 10:09:34.218826 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:34.218757 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8bl4l" podStartSLOduration=1.736881788 podStartE2EDuration="4.21873712s" totalImagesPullingTime="2.481855332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:09:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:09:34.218697966 +0000 UTC m=+91.930939923" watchObservedRunningTime="2026-09-07 10:09:34.21873712 +0000 UTC m=+91.930979068" Sep 07 10:09:34.673099 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:34.673055 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51adbf04-234f-4a6c-8aec-c65483741ae6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:34.673580 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:34.673149 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51adbf04-234f-4a6c-8aec-c65483741ae6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:34.673580 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:34.673222 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 10:09:34.673580 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:34.673280 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 10:09:34.673580 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:34.673288 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51adbf04-234f-4a6c-8aec-c65483741ae6-insights-runtime-extractor-tls podName:51adbf04-234f-4a6c-8aec-c65483741ae6 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:35.673271637 +0000 UTC m=+93.385513579 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/51adbf04-234f-4a6c-8aec-c65483741ae6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gv4pf" (UID: "51adbf04-234f-4a6c-8aec-c65483741ae6") : secret "insights-runtime-extractor-tls" not found Sep 07 10:09:34.673580 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:34.673348 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/51adbf04-234f-4a6c-8aec-c65483741ae6-kube-rbac-proxy-cm podName:51adbf04-234f-4a6c-8aec-c65483741ae6 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:35.673332348 +0000 UTC m=+93.385574272 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/51adbf04-234f-4a6c-8aec-c65483741ae6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gv4pf" (UID: "51adbf04-234f-4a6c-8aec-c65483741ae6") : configmap "kube-rbac-proxy" not found Sep 07 10:09:35.195613 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:35.195585 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-m6mmt_6aad73e9-87ee-49b0-8651-d3fd8390e4e4/console-operator/1.log" Sep 07 10:09:35.196006 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:35.195989 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-m6mmt_6aad73e9-87ee-49b0-8651-d3fd8390e4e4/console-operator/0.log" Sep 07 10:09:35.196077 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:35.196024 2575 generic.go:356] "Generic (PLEG): container finished" podID="6aad73e9-87ee-49b0-8651-d3fd8390e4e4" containerID="4d5d5648494d6302f2039752f144455ca9c88b82b91440620017cf7e4f14a800" exitCode=255 Sep 07 10:09:35.196077 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:35.196051 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-m6mmt" event={"ID":"6aad73e9-87ee-49b0-8651-d3fd8390e4e4","Type":"ContainerDied","Data":"4d5d5648494d6302f2039752f144455ca9c88b82b91440620017cf7e4f14a800"} Sep 07 10:09:35.196162 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:35.196103 2575 scope.go:117] "RemoveContainer" containerID="7328d8b927a9f265ebfc22baa4227cb265cecd8eea38bef6a520db1e60e9c630" Sep 07 10:09:35.196403 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:35.196387 2575 scope.go:117] "RemoveContainer" containerID="4d5d5648494d6302f2039752f144455ca9c88b82b91440620017cf7e4f14a800" Sep 07 10:09:35.196576 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:35.196558 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-m6mmt_openshift-console-operator(6aad73e9-87ee-49b0-8651-d3fd8390e4e4)\"" pod="openshift-console-operator/console-operator-85b454874b-m6mmt" podUID="6aad73e9-87ee-49b0-8651-d3fd8390e4e4" Sep 07 10:09:35.681582 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:35.681535 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51adbf04-234f-4a6c-8aec-c65483741ae6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:35.681989 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:35.681622 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51adbf04-234f-4a6c-8aec-c65483741ae6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:35.681989 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:35.681656 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 10:09:35.681989 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:35.681727 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/51adbf04-234f-4a6c-8aec-c65483741ae6-kube-rbac-proxy-cm podName:51adbf04-234f-4a6c-8aec-c65483741ae6 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:37.681710586 +0000 UTC m=+95.393952509 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/51adbf04-234f-4a6c-8aec-c65483741ae6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gv4pf" (UID: "51adbf04-234f-4a6c-8aec-c65483741ae6") : configmap "kube-rbac-proxy" not found Sep 07 10:09:35.681989 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:35.681738 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 10:09:35.681989 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:35.681784 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51adbf04-234f-4a6c-8aec-c65483741ae6-insights-runtime-extractor-tls podName:51adbf04-234f-4a6c-8aec-c65483741ae6 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:37.681768891 +0000 UTC m=+95.394010815 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/51adbf04-234f-4a6c-8aec-c65483741ae6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gv4pf" (UID: "51adbf04-234f-4a6c-8aec-c65483741ae6") : secret "insights-runtime-extractor-tls" not found Sep 07 10:09:36.059482 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:36.059401 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w5w2p_19524835-4647-4786-bc0f-c222dd09c075/dns-node-resolver/0.log" Sep 07 10:09:36.200316 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:36.200290 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-m6mmt_6aad73e9-87ee-49b0-8651-d3fd8390e4e4/console-operator/1.log" Sep 07 10:09:36.200651 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:36.200632 2575 scope.go:117] "RemoveContainer" containerID="4d5d5648494d6302f2039752f144455ca9c88b82b91440620017cf7e4f14a800" Sep 07 10:09:36.200843 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:36.200823 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-m6mmt_openshift-console-operator(6aad73e9-87ee-49b0-8651-d3fd8390e4e4)\"" pod="openshift-console-operator/console-operator-85b454874b-m6mmt" podUID="6aad73e9-87ee-49b0-8651-d3fd8390e4e4" Sep 07 10:09:36.858672 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:36.858645 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t6ztt_a6a30db9-93e6-41b1-a0e6-28e5088df925/node-ca/0.log" Sep 07 10:09:37.658845 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:37.658771 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jfq75_1352b671-2ab8-4c45-8f10-f82960e1ff1c/serve-healthcheck-canary/0.log" Sep 07 10:09:37.698263 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:37.698226 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/502e0127-90b6-4388-860b-5dd7979db869-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-dx9lz\" (UID: \"502e0127-90b6-4388-860b-5dd7979db869\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz" Sep 07 10:09:37.698365 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:37.698273 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51adbf04-234f-4a6c-8aec-c65483741ae6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:37.698365 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:37.698307 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-service-ca-bundle\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:37.698365 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:37.698326 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-metrics-certs\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:37.698365 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:37.698347 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51adbf04-234f-4a6c-8aec-c65483741ae6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:37.698512 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:37.698376 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 07 10:09:37.698512 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:37.698419 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 10:09:37.698512 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:37.698455 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/502e0127-90b6-4388-860b-5dd7979db869-cluster-monitoring-operator-tls podName:502e0127-90b6-4388-860b-5dd7979db869 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:45.698433229 +0000 UTC m=+103.410675178 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/502e0127-90b6-4388-860b-5dd7979db869-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-dx9lz" (UID: "502e0127-90b6-4388-860b-5dd7979db869") : secret "cluster-monitoring-operator-tls" not found Sep 07 10:09:37.698512 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:37.698462 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 10:09:37.698512 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:37.698477 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/51adbf04-234f-4a6c-8aec-c65483741ae6-kube-rbac-proxy-cm podName:51adbf04-234f-4a6c-8aec-c65483741ae6 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:41.698465924 +0000 UTC m=+99.410707848 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/51adbf04-234f-4a6c-8aec-c65483741ae6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gv4pf" (UID: "51adbf04-234f-4a6c-8aec-c65483741ae6") : configmap "kube-rbac-proxy" not found Sep 07 10:09:37.698512 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:37.698492 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-service-ca-bundle podName:c3c51a39-edc3-44e7-a0ed-843ac7ec5059 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:45.698484133 +0000 UTC m=+103.410726057 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-service-ca-bundle") pod "router-default-6c558849d-hzpf7" (UID: "c3c51a39-edc3-44e7-a0ed-843ac7ec5059") : configmap references non-existent config key: service-ca.crt Sep 07 10:09:37.698720 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:37.698537 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51adbf04-234f-4a6c-8aec-c65483741ae6-insights-runtime-extractor-tls podName:51adbf04-234f-4a6c-8aec-c65483741ae6 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:41.698526099 +0000 UTC m=+99.410768035 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/51adbf04-234f-4a6c-8aec-c65483741ae6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gv4pf" (UID: "51adbf04-234f-4a6c-8aec-c65483741ae6") : secret "insights-runtime-extractor-tls" not found Sep 07 10:09:37.698720 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:37.698463 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 07 10:09:37.698720 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:37.698576 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-metrics-certs podName:c3c51a39-edc3-44e7-a0ed-843ac7ec5059 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:45.698567367 +0000 UTC m=+103.410809305 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-metrics-certs") pod "router-default-6c558849d-hzpf7" (UID: "c3c51a39-edc3-44e7-a0ed-843ac7ec5059") : secret "router-metrics-certs-default" not found Sep 07 10:09:40.492636 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:40.492539 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-m6mmt" Sep 07 10:09:40.492636 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:40.492609 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-m6mmt" Sep 07 10:09:40.493061 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:40.492915 2575 scope.go:117] "RemoveContainer" containerID="4d5d5648494d6302f2039752f144455ca9c88b82b91440620017cf7e4f14a800" Sep 07 10:09:40.493061 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:40.493050 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-m6mmt_openshift-console-operator(6aad73e9-87ee-49b0-8651-d3fd8390e4e4)\"" pod="openshift-console-operator/console-operator-85b454874b-m6mmt" podUID="6aad73e9-87ee-49b0-8651-d3fd8390e4e4" Sep 07 10:09:40.522130 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:40.522057 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls\") pod \"dns-default-nc7j5\" (UID: \"b07b7355-ed97-4a04-958b-c675ec096d60\") " pod="openshift-dns/dns-default-nc7j5" Sep 07 10:09:40.522267 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:40.522196 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 10:09:40.522267 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:40.522253 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls podName:b07b7355-ed97-4a04-958b-c675ec096d60 nodeName:}" failed. No retries permitted until 2026-09-07 10:10:44.522240326 +0000 UTC m=+162.234482249 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls") pod "dns-default-nc7j5" (UID: "b07b7355-ed97-4a04-958b-c675ec096d60") : secret "dns-default-metrics-tls" not found Sep 07 10:09:41.731467 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:41.731407 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51adbf04-234f-4a6c-8aec-c65483741ae6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:41.731930 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:41.731509 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51adbf04-234f-4a6c-8aec-c65483741ae6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:41.731930 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:41.731538 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 10:09:41.731930 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:41.731606 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 10:09:41.731930 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:41.731607 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/51adbf04-234f-4a6c-8aec-c65483741ae6-kube-rbac-proxy-cm podName:51adbf04-234f-4a6c-8aec-c65483741ae6 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:49.73159156 +0000 UTC m=+107.443833509 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/51adbf04-234f-4a6c-8aec-c65483741ae6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gv4pf" (UID: "51adbf04-234f-4a6c-8aec-c65483741ae6") : configmap "kube-rbac-proxy" not found Sep 07 10:09:41.731930 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:09:41.731662 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51adbf04-234f-4a6c-8aec-c65483741ae6-insights-runtime-extractor-tls podName:51adbf04-234f-4a6c-8aec-c65483741ae6 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:49.731645938 +0000 UTC m=+107.443887862 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/51adbf04-234f-4a6c-8aec-c65483741ae6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gv4pf" (UID: "51adbf04-234f-4a6c-8aec-c65483741ae6") : secret "insights-runtime-extractor-tls" not found Sep 07 10:09:45.141101 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:45.141067 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-km29h" Sep 07 10:09:45.762729 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:45.762687 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/502e0127-90b6-4388-860b-5dd7979db869-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-dx9lz\" (UID: \"502e0127-90b6-4388-860b-5dd7979db869\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz" Sep 07 10:09:45.762960 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:45.762749 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-service-ca-bundle\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:45.762960 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:45.762768 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-metrics-certs\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:45.763401 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:45.763368 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-service-ca-bundle\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:45.765142 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:45.765119 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3c51a39-edc3-44e7-a0ed-843ac7ec5059-metrics-certs\") pod \"router-default-6c558849d-hzpf7\" (UID: \"c3c51a39-edc3-44e7-a0ed-843ac7ec5059\") " pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:45.765224 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:45.765123 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/502e0127-90b6-4388-860b-5dd7979db869-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-dx9lz\" (UID: \"502e0127-90b6-4388-860b-5dd7979db869\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz" Sep 07 10:09:45.882164 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:45.882128 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz" Sep 07 10:09:45.886636 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:45.886609 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:46.159761 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:46.159654 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz"] Sep 07 10:09:46.163030 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:46.163004 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6c558849d-hzpf7"] Sep 07 10:09:46.166007 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:09:46.165983 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3c51a39_edc3_44e7_a0ed_843ac7ec5059.slice/crio-4ce2c0e658ce2d91b253cf6d6771d647c5163b184fc80082d43920f03848fedc WatchSource:0}: Error finding container 4ce2c0e658ce2d91b253cf6d6771d647c5163b184fc80082d43920f03848fedc: Status 404 returned error can't find the container with id 4ce2c0e658ce2d91b253cf6d6771d647c5163b184fc80082d43920f03848fedc Sep 07 10:09:46.233169 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:46.233139 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz" event={"ID":"502e0127-90b6-4388-860b-5dd7979db869","Type":"ContainerStarted","Data":"db59e39e8e4c3ab39f39031b9a88a7859234665631db11ae7c9db067cb2bd045"} Sep 07 10:09:46.237507 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:46.237483 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6c558849d-hzpf7" event={"ID":"c3c51a39-edc3-44e7-a0ed-843ac7ec5059","Type":"ContainerStarted","Data":"4ce2c0e658ce2d91b253cf6d6771d647c5163b184fc80082d43920f03848fedc"} Sep 07 10:09:47.242413 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:47.242376 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6c558849d-hzpf7" event={"ID":"c3c51a39-edc3-44e7-a0ed-843ac7ec5059","Type":"ContainerStarted","Data":"8495e0a5b6915203a3551f576c8a7ce7d535d20b1aed5e8dbb7f3b7ed3855176"} Sep 07 10:09:47.260978 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:47.260929 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6c558849d-hzpf7" podStartSLOduration=18.26091289 podStartE2EDuration="18.26091289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:09:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:09:47.25970663 +0000 UTC m=+104.971948578" watchObservedRunningTime="2026-09-07 10:09:47.26091289 +0000 UTC m=+104.973154831" Sep 07 10:09:47.887385 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:47.887346 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:47.890296 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:47.890273 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:48.246103 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:48.246024 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:48.247303 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:48.247279 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6c558849d-hzpf7" Sep 07 10:09:49.250030 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:49.249993 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz" event={"ID":"502e0127-90b6-4388-860b-5dd7979db869","Type":"ContainerStarted","Data":"3818039d011f385715decae61232c5589824db2015a861bacfda82370ec9b2be"} Sep 07 10:09:49.794677 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:49.794623 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51adbf04-234f-4a6c-8aec-c65483741ae6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:49.794677 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:49.794696 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51adbf04-234f-4a6c-8aec-c65483741ae6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:49.795190 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:49.795171 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51adbf04-234f-4a6c-8aec-c65483741ae6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:49.797061 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:49.797039 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51adbf04-234f-4a6c-8aec-c65483741ae6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gv4pf\" (UID: \"51adbf04-234f-4a6c-8aec-c65483741ae6\") " pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:49.838837 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:49.838808 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gv4pf" Sep 07 10:09:50.104128 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:50.104059 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-dx9lz" podStartSLOduration=18.681826988 podStartE2EDuration="21.104033872s" totalImagesPullingTime="2.422206884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:09:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:09:49.26460557 +0000 UTC m=+106.976847516" watchObservedRunningTime="2026-09-07 10:09:50.104033872 +0000 UTC m=+107.816275818" Sep 07 10:09:50.105070 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:50.105047 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gv4pf"] Sep 07 10:09:50.108460 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:09:50.108430 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51adbf04_234f_4a6c_8aec_c65483741ae6.slice/crio-356454287ba7ed0a8bb0f2b7a6474d5cac5a1d6f0d18ca31a83fe7281e3ca2df WatchSource:0}: Error finding container 356454287ba7ed0a8bb0f2b7a6474d5cac5a1d6f0d18ca31a83fe7281e3ca2df: Status 404 returned error can't find the container with id 356454287ba7ed0a8bb0f2b7a6474d5cac5a1d6f0d18ca31a83fe7281e3ca2df Sep 07 10:09:50.254464 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:50.254426 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gv4pf" event={"ID":"51adbf04-234f-4a6c-8aec-c65483741ae6","Type":"ContainerStarted","Data":"e5704cc2b4824f20b0312d1432cb92275fa94154c5a3466403baab30691a5f14"} Sep 07 10:09:50.254859 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:50.254484 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gv4pf" event={"ID":"51adbf04-234f-4a6c-8aec-c65483741ae6","Type":"ContainerStarted","Data":"356454287ba7ed0a8bb0f2b7a6474d5cac5a1d6f0d18ca31a83fe7281e3ca2df"} Sep 07 10:09:51.258863 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:51.258775 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gv4pf" event={"ID":"51adbf04-234f-4a6c-8aec-c65483741ae6","Type":"ContainerStarted","Data":"5b66b55bdbda7e92e996db8190ccb4b01fdcf0d4be66d0d32620879e34ff20ab"} Sep 07 10:09:51.825636 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:51.825598 2575 scope.go:117] "RemoveContainer" containerID="4d5d5648494d6302f2039752f144455ca9c88b82b91440620017cf7e4f14a800" Sep 07 10:09:52.266039 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:52.265960 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-m6mmt_6aad73e9-87ee-49b0-8651-d3fd8390e4e4/console-operator/1.log" Sep 07 10:09:52.266474 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:52.266046 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-m6mmt" event={"ID":"6aad73e9-87ee-49b0-8651-d3fd8390e4e4","Type":"ContainerStarted","Data":"ad5822c3c38c8b4688add0f24b3e98bb2cc5119c5ca0d6a69065f0eda9eb6ff8"} Sep 07 10:09:52.266474 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:52.266338 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-m6mmt" Sep 07 10:09:52.282889 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:52.282825 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-m6mmt" podStartSLOduration=19.799219856 podStartE2EDuration="22.282785023s" totalImagesPullingTime="2.483565167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:09:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:09:52.282197399 +0000 UTC m=+109.994439345" watchObservedRunningTime="2026-09-07 10:09:52.282785023 +0000 UTC m=+109.995026969" Sep 07 10:09:52.436033 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:52.436001 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-m6mmt" Sep 07 10:09:53.271787 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:53.271747 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gv4pf" event={"ID":"51adbf04-234f-4a6c-8aec-c65483741ae6","Type":"ContainerStarted","Data":"ed7546e4c5d76589a70c33848eb0f8931cbdc6d60bc0a9a56139c95311337342"} Sep 07 10:09:53.288922 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:53.288864 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gv4pf" podStartSLOduration=18.176048835 podStartE2EDuration="20.288846372s" totalImagesPullingTime="2.112797537s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:09:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:09:53.287605067 +0000 UTC m=+110.999847016" watchObservedRunningTime="2026-09-07 10:09:53.288846372 +0000 UTC m=+111.001088653" Sep 07 10:09:55.125731 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.125694 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc45f5654-4fzxh"] Sep 07 10:09:55.128773 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.128744 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc45f5654-4fzxh" Sep 07 10:09:55.132002 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.131973 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 07 10:09:55.132731 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.132711 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 07 10:09:55.132731 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.132721 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-q6h7f\"" Sep 07 10:09:55.133104 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.133085 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 07 10:09:55.133185 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.133106 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 07 10:09:55.140469 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.140445 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc45f5654-4fzxh"] Sep 07 10:09:55.239853 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.239785 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr7fc\" (UniqueName: \"kubernetes.io/projected/07f98b11-1d94-4b43-a007-b3c51a2a9807-kube-api-access-cr7fc\") pod \"managed-serviceaccount-addon-agent-6bc45f5654-4fzxh\" (UID: \"07f98b11-1d94-4b43-a007-b3c51a2a9807\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc45f5654-4fzxh" Sep 07 10:09:55.240071 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.239944 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/07f98b11-1d94-4b43-a007-b3c51a2a9807-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bc45f5654-4fzxh\" (UID: \"07f98b11-1d94-4b43-a007-b3c51a2a9807\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc45f5654-4fzxh" Sep 07 10:09:55.241349 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.241326 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-86f7794bb5-pxqrg"] Sep 07 10:09:55.244216 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.244202 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.247931 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.247907 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 07 10:09:55.248043 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.247999 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 07 10:09:55.248203 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.248189 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-djfnm\"" Sep 07 10:09:55.249130 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.249111 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 07 10:09:55.259815 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.255646 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 07 10:09:55.267769 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.260861 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86f7794bb5-pxqrg"] Sep 07 10:09:55.341233 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.341196 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-registry-certificates\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.341233 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.341230 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd7q6\" (UniqueName: \"kubernetes.io/projected/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-kube-api-access-dd7q6\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.341433 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.341257 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-ca-trust-extracted\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.341433 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.341367 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-registry-tls\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.341505 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.341455 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-installation-pull-secrets\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.341505 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.341482 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-trusted-ca\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.341573 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.341504 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/07f98b11-1d94-4b43-a007-b3c51a2a9807-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bc45f5654-4fzxh\" (UID: \"07f98b11-1d94-4b43-a007-b3c51a2a9807\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc45f5654-4fzxh" Sep 07 10:09:55.341573 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.341533 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-bound-sa-token\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.341573 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.341567 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-image-registry-private-configuration\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.341670 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.341596 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr7fc\" (UniqueName: \"kubernetes.io/projected/07f98b11-1d94-4b43-a007-b3c51a2a9807-kube-api-access-cr7fc\") pod \"managed-serviceaccount-addon-agent-6bc45f5654-4fzxh\" (UID: \"07f98b11-1d94-4b43-a007-b3c51a2a9807\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc45f5654-4fzxh" Sep 07 10:09:55.343991 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.343965 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/07f98b11-1d94-4b43-a007-b3c51a2a9807-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bc45f5654-4fzxh\" (UID: \"07f98b11-1d94-4b43-a007-b3c51a2a9807\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc45f5654-4fzxh" Sep 07 10:09:55.349843 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.349819 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr7fc\" (UniqueName: \"kubernetes.io/projected/07f98b11-1d94-4b43-a007-b3c51a2a9807-kube-api-access-cr7fc\") pod \"managed-serviceaccount-addon-agent-6bc45f5654-4fzxh\" (UID: \"07f98b11-1d94-4b43-a007-b3c51a2a9807\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc45f5654-4fzxh" Sep 07 10:09:55.442400 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.442313 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-image-registry-private-configuration\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.442400 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.442373 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-registry-certificates\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.442400 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.442393 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd7q6\" (UniqueName: \"kubernetes.io/projected/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-kube-api-access-dd7q6\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.442658 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.442425 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-ca-trust-extracted\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.442658 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.442465 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-registry-tls\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.442658 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.442517 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-installation-pull-secrets\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.442658 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.442544 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-trusted-ca\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.442658 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.442570 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-bound-sa-token\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.442968 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.442946 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-ca-trust-extracted\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.443569 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.443537 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-trusted-ca\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.443569 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.443556 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-registry-certificates\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.444726 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.444694 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc45f5654-4fzxh" Sep 07 10:09:55.444936 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.444917 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-image-registry-private-configuration\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.445000 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.444974 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-installation-pull-secrets\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.445114 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.445097 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-registry-tls\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.457732 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.457706 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd7q6\" (UniqueName: \"kubernetes.io/projected/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-kube-api-access-dd7q6\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.457898 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.457754 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc-bound-sa-token\") pod \"image-registry-86f7794bb5-pxqrg\" (UID: \"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc\") " pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.555858 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.555817 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:55.721164 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.721086 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc45f5654-4fzxh"] Sep 07 10:09:55.723826 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:09:55.723776 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07f98b11_1d94_4b43_a007_b3c51a2a9807.slice/crio-f83dab7605b584ec364f1a2eafb86e610e2256009a06e2dbab46c69e8d3452ad WatchSource:0}: Error finding container f83dab7605b584ec364f1a2eafb86e610e2256009a06e2dbab46c69e8d3452ad: Status 404 returned error can't find the container with id f83dab7605b584ec364f1a2eafb86e610e2256009a06e2dbab46c69e8d3452ad Sep 07 10:09:55.825009 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:55.824976 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86f7794bb5-pxqrg"] Sep 07 10:09:55.827840 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:09:55.827814 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0a13746_7fc9_4fdb_ad9a_17e2020bd2bc.slice/crio-48b42229dcebfa977885f3897e39c7018ea73402819932d3b6d02ede517acda3 WatchSource:0}: Error finding container 48b42229dcebfa977885f3897e39c7018ea73402819932d3b6d02ede517acda3: Status 404 returned error can't find the container with id 48b42229dcebfa977885f3897e39c7018ea73402819932d3b6d02ede517acda3 Sep 07 10:09:56.284212 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:56.284171 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" event={"ID":"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc","Type":"ContainerStarted","Data":"0bd4f2d75e03d8a14843975819a3432b85a18f7d7c347a5a1619e384c7c51d1b"} Sep 07 10:09:56.284646 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:56.284226 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" event={"ID":"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc","Type":"ContainerStarted","Data":"48b42229dcebfa977885f3897e39c7018ea73402819932d3b6d02ede517acda3"} Sep 07 10:09:56.284646 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:56.284289 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:09:56.285394 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:56.285376 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc45f5654-4fzxh" event={"ID":"07f98b11-1d94-4b43-a007-b3c51a2a9807","Type":"ContainerStarted","Data":"f83dab7605b584ec364f1a2eafb86e610e2256009a06e2dbab46c69e8d3452ad"} Sep 07 10:09:56.302569 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:56.302508 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" podStartSLOduration=1.302490532 podStartE2EDuration="1.302490532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:09:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:09:56.300776278 +0000 UTC m=+114.013018246" watchObservedRunningTime="2026-09-07 10:09:56.302490532 +0000 UTC m=+114.014732476" Sep 07 10:09:58.176441 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.176404 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-djr8f"] Sep 07 10:09:58.179666 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.179646 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-djr8f" Sep 07 10:09:58.182988 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.182959 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 07 10:09:58.183127 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.182994 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bk52t\"" Sep 07 10:09:58.183127 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.183002 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 07 10:09:58.183127 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.183003 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 07 10:09:58.189227 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.189204 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-djr8f"] Sep 07 10:09:58.267998 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.267900 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d198470-13aa-4025-a865-7d1f9f8ca2cc-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-djr8f\" (UID: \"4d198470-13aa-4025-a865-7d1f9f8ca2cc\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-djr8f" Sep 07 10:09:58.267998 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.267951 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2b4v\" (UniqueName: \"kubernetes.io/projected/4d198470-13aa-4025-a865-7d1f9f8ca2cc-kube-api-access-v2b4v\") pod \"prometheus-operator-6f4946db56-djr8f\" (UID: \"4d198470-13aa-4025-a865-7d1f9f8ca2cc\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-djr8f" Sep 07 10:09:58.267998 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.267990 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d198470-13aa-4025-a865-7d1f9f8ca2cc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-djr8f\" (UID: \"4d198470-13aa-4025-a865-7d1f9f8ca2cc\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-djr8f" Sep 07 10:09:58.268261 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.268030 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d198470-13aa-4025-a865-7d1f9f8ca2cc-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-djr8f\" (UID: \"4d198470-13aa-4025-a865-7d1f9f8ca2cc\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-djr8f" Sep 07 10:09:58.294545 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.294509 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc45f5654-4fzxh" event={"ID":"07f98b11-1d94-4b43-a007-b3c51a2a9807","Type":"ContainerStarted","Data":"396eecee39ebf676996102e9fbd3beb64b938829887e7be510c447be7d98ee86"} Sep 07 10:09:58.308351 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.308297 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc45f5654-4fzxh" podStartSLOduration=1.017728889 podStartE2EDuration="3.308280395s" totalImagesPullingTime="2.290551506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:09:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:09:58.307869112 +0000 UTC m=+116.020111059" watchObservedRunningTime="2026-09-07 10:09:58.308280395 +0000 UTC m=+116.020522339" Sep 07 10:09:58.368374 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.368338 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d198470-13aa-4025-a865-7d1f9f8ca2cc-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-djr8f\" (UID: \"4d198470-13aa-4025-a865-7d1f9f8ca2cc\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-djr8f" Sep 07 10:09:58.368490 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.368380 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2b4v\" (UniqueName: \"kubernetes.io/projected/4d198470-13aa-4025-a865-7d1f9f8ca2cc-kube-api-access-v2b4v\") pod \"prometheus-operator-6f4946db56-djr8f\" (UID: \"4d198470-13aa-4025-a865-7d1f9f8ca2cc\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-djr8f" Sep 07 10:09:58.368490 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.368424 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d198470-13aa-4025-a865-7d1f9f8ca2cc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-djr8f\" (UID: \"4d198470-13aa-4025-a865-7d1f9f8ca2cc\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-djr8f" Sep 07 10:09:58.368490 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.368473 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d198470-13aa-4025-a865-7d1f9f8ca2cc-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-djr8f\" (UID: \"4d198470-13aa-4025-a865-7d1f9f8ca2cc\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-djr8f" Sep 07 10:09:58.369171 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.369146 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d198470-13aa-4025-a865-7d1f9f8ca2cc-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-djr8f\" (UID: \"4d198470-13aa-4025-a865-7d1f9f8ca2cc\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-djr8f" Sep 07 10:09:58.370769 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.370740 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d198470-13aa-4025-a865-7d1f9f8ca2cc-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-djr8f\" (UID: \"4d198470-13aa-4025-a865-7d1f9f8ca2cc\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-djr8f" Sep 07 10:09:58.370890 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.370783 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d198470-13aa-4025-a865-7d1f9f8ca2cc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-djr8f\" (UID: \"4d198470-13aa-4025-a865-7d1f9f8ca2cc\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-djr8f" Sep 07 10:09:58.376278 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.376252 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2b4v\" (UniqueName: \"kubernetes.io/projected/4d198470-13aa-4025-a865-7d1f9f8ca2cc-kube-api-access-v2b4v\") pod \"prometheus-operator-6f4946db56-djr8f\" (UID: \"4d198470-13aa-4025-a865-7d1f9f8ca2cc\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-djr8f" Sep 07 10:09:58.488485 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.488439 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-djr8f" Sep 07 10:09:58.755342 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:58.755315 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-djr8f"] Sep 07 10:09:58.757533 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:09:58.757506 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d198470_13aa_4025_a865_7d1f9f8ca2cc.slice/crio-83947379827f73b8c90ec65b3c5d8bd5154ac3a5b1a1f26de057465b07a2e3d9 WatchSource:0}: Error finding container 83947379827f73b8c90ec65b3c5d8bd5154ac3a5b1a1f26de057465b07a2e3d9: Status 404 returned error can't find the container with id 83947379827f73b8c90ec65b3c5d8bd5154ac3a5b1a1f26de057465b07a2e3d9 Sep 07 10:09:59.298870 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:09:59.298833 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-djr8f" event={"ID":"4d198470-13aa-4025-a865-7d1f9f8ca2cc","Type":"ContainerStarted","Data":"83947379827f73b8c90ec65b3c5d8bd5154ac3a5b1a1f26de057465b07a2e3d9"} Sep 07 10:10:00.306413 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:00.306292 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-djr8f" event={"ID":"4d198470-13aa-4025-a865-7d1f9f8ca2cc","Type":"ContainerStarted","Data":"21b2cd4c8b622c5de7bfccb998fc0bdf38a3c8dbec59875ef04eecb66ae87579"} Sep 07 10:10:00.306413 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:00.306360 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-djr8f" event={"ID":"4d198470-13aa-4025-a865-7d1f9f8ca2cc","Type":"ContainerStarted","Data":"6f2a682682f74eaf55f26fcd0c97d760689429b0e336a03dab0958dda028ff0b"} Sep 07 10:10:00.323774 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:00.322975 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-djr8f" podStartSLOduration=0.970821888 podStartE2EDuration="2.322950977s" totalImagesPullingTime="1.352129089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:10:00.321988165 +0000 UTC m=+118.034230123" watchObservedRunningTime="2026-09-07 10:10:00.322950977 +0000 UTC m=+118.035192920" Sep 07 10:10:02.536447 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.536409 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-975lp"] Sep 07 10:10:02.543555 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.543518 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6ctzx"] Sep 07 10:10:02.543711 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.543647 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-975lp" Sep 07 10:10:02.546030 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.546004 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 07 10:10:02.546202 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.546135 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-zczs4\"" Sep 07 10:10:02.546202 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.546167 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 07 10:10:02.547157 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.547133 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.549556 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.549536 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 07 10:10:02.549922 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.549906 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8phjk\"" Sep 07 10:10:02.550143 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.550128 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 07 10:10:02.550452 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.550438 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 07 10:10:02.553278 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.553256 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-975lp"] Sep 07 10:10:02.702931 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.702891 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fa50d828-c196-4e48-9742-8f6aaea0b9df-node-exporter-wtmp\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.702931 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.702932 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fa50d828-c196-4e48-9742-8f6aaea0b9df-node-exporter-accelerators-collector-config\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.703150 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.702962 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fa50d828-c196-4e48-9742-8f6aaea0b9df-root\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.703150 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.703015 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fa50d828-c196-4e48-9742-8f6aaea0b9df-node-exporter-textfile\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.703150 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.703070 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ce40ab0-a188-4403-b608-575dcef41060-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-975lp\" (UID: \"8ce40ab0-a188-4403-b608-575dcef41060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-975lp" Sep 07 10:10:02.703150 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.703092 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd2ss\" (UniqueName: \"kubernetes.io/projected/8ce40ab0-a188-4403-b608-575dcef41060-kube-api-access-wd2ss\") pod \"openshift-state-metrics-657fd6c54-975lp\" (UID: \"8ce40ab0-a188-4403-b608-575dcef41060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-975lp" Sep 07 10:10:02.703150 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.703135 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fa50d828-c196-4e48-9742-8f6aaea0b9df-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.703300 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.703199 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ce40ab0-a188-4403-b608-575dcef41060-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-975lp\" (UID: \"8ce40ab0-a188-4403-b608-575dcef41060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-975lp" Sep 07 10:10:02.703300 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.703233 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km24z\" (UniqueName: \"kubernetes.io/projected/fa50d828-c196-4e48-9742-8f6aaea0b9df-kube-api-access-km24z\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.703300 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.703281 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa50d828-c196-4e48-9742-8f6aaea0b9df-sys\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.703385 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.703313 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fa50d828-c196-4e48-9742-8f6aaea0b9df-node-exporter-tls\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.703385 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.703351 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ce40ab0-a188-4403-b608-575dcef41060-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-975lp\" (UID: \"8ce40ab0-a188-4403-b608-575dcef41060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-975lp" Sep 07 10:10:02.703385 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.703367 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa50d828-c196-4e48-9742-8f6aaea0b9df-metrics-client-ca\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.803949 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.803856 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ce40ab0-a188-4403-b608-575dcef41060-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-975lp\" (UID: \"8ce40ab0-a188-4403-b608-575dcef41060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-975lp" Sep 07 10:10:02.803949 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.803911 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km24z\" (UniqueName: \"kubernetes.io/projected/fa50d828-c196-4e48-9742-8f6aaea0b9df-kube-api-access-km24z\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.804183 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.803956 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa50d828-c196-4e48-9742-8f6aaea0b9df-sys\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.804183 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.803986 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fa50d828-c196-4e48-9742-8f6aaea0b9df-node-exporter-tls\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.804183 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.804012 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ce40ab0-a188-4403-b608-575dcef41060-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-975lp\" (UID: \"8ce40ab0-a188-4403-b608-575dcef41060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-975lp" Sep 07 10:10:02.804183 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.804058 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa50d828-c196-4e48-9742-8f6aaea0b9df-sys\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.804183 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.804127 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa50d828-c196-4e48-9742-8f6aaea0b9df-metrics-client-ca\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.804183 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.804180 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fa50d828-c196-4e48-9742-8f6aaea0b9df-node-exporter-wtmp\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.804470 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.804201 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fa50d828-c196-4e48-9742-8f6aaea0b9df-node-exporter-accelerators-collector-config\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.804470 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.804224 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fa50d828-c196-4e48-9742-8f6aaea0b9df-root\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.804470 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.804269 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fa50d828-c196-4e48-9742-8f6aaea0b9df-node-exporter-textfile\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.804470 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.804284 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fa50d828-c196-4e48-9742-8f6aaea0b9df-root\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.804470 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.804324 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ce40ab0-a188-4403-b608-575dcef41060-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-975lp\" (UID: \"8ce40ab0-a188-4403-b608-575dcef41060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-975lp" Sep 07 10:10:02.804470 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.804352 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd2ss\" (UniqueName: \"kubernetes.io/projected/8ce40ab0-a188-4403-b608-575dcef41060-kube-api-access-wd2ss\") pod \"openshift-state-metrics-657fd6c54-975lp\" (UID: \"8ce40ab0-a188-4403-b608-575dcef41060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-975lp" Sep 07 10:10:02.804470 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.804355 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fa50d828-c196-4e48-9742-8f6aaea0b9df-node-exporter-wtmp\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.804470 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.804381 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fa50d828-c196-4e48-9742-8f6aaea0b9df-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.804737 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.804591 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fa50d828-c196-4e48-9742-8f6aaea0b9df-node-exporter-textfile\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.804778 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.804761 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa50d828-c196-4e48-9742-8f6aaea0b9df-metrics-client-ca\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.804963 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.804946 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ce40ab0-a188-4403-b608-575dcef41060-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-975lp\" (UID: \"8ce40ab0-a188-4403-b608-575dcef41060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-975lp" Sep 07 10:10:02.806384 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.806355 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 07 10:10:02.806523 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.806491 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 07 10:10:02.806621 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.806602 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 07 10:10:02.806666 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.806626 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 07 10:10:02.806666 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.806628 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 07 10:10:02.812036 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.812014 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-km24z\" (UniqueName: \"kubernetes.io/projected/fa50d828-c196-4e48-9742-8f6aaea0b9df-kube-api-access-km24z\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.812173 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.812157 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd2ss\" (UniqueName: \"kubernetes.io/projected/8ce40ab0-a188-4403-b608-575dcef41060-kube-api-access-wd2ss\") pod \"openshift-state-metrics-657fd6c54-975lp\" (UID: \"8ce40ab0-a188-4403-b608-575dcef41060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-975lp" Sep 07 10:10:02.814980 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:10:02.814960 2575 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 07 10:10:02.815077 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:10:02.814971 2575 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Sep 07 10:10:02.815077 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:10:02.815034 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa50d828-c196-4e48-9742-8f6aaea0b9df-node-exporter-tls podName:fa50d828-c196-4e48-9742-8f6aaea0b9df nodeName:}" failed. No retries permitted until 2026-09-07 10:10:03.315012779 +0000 UTC m=+121.027254720 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/fa50d828-c196-4e48-9742-8f6aaea0b9df-node-exporter-tls") pod "node-exporter-6ctzx" (UID: "fa50d828-c196-4e48-9742-8f6aaea0b9df") : secret "node-exporter-tls" not found Sep 07 10:10:02.815183 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:10:02.815085 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ce40ab0-a188-4403-b608-575dcef41060-openshift-state-metrics-tls podName:8ce40ab0-a188-4403-b608-575dcef41060 nodeName:}" failed. No retries permitted until 2026-09-07 10:10:03.315068353 +0000 UTC m=+121.027310289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/8ce40ab0-a188-4403-b608-575dcef41060-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-975lp" (UID: "8ce40ab0-a188-4403-b608-575dcef41060") : secret "openshift-state-metrics-tls" not found Sep 07 10:10:02.815378 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.815358 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fa50d828-c196-4e48-9742-8f6aaea0b9df-node-exporter-accelerators-collector-config\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.817010 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.816986 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fa50d828-c196-4e48-9742-8f6aaea0b9df-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:02.817114 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:02.817092 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ce40ab0-a188-4403-b608-575dcef41060-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-975lp\" (UID: \"8ce40ab0-a188-4403-b608-575dcef41060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-975lp" Sep 07 10:10:03.411001 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:03.410958 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fa50d828-c196-4e48-9742-8f6aaea0b9df-node-exporter-tls\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:03.411001 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:03.411015 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ce40ab0-a188-4403-b608-575dcef41060-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-975lp\" (UID: \"8ce40ab0-a188-4403-b608-575dcef41060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-975lp" Sep 07 10:10:03.413371 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:03.413346 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fa50d828-c196-4e48-9742-8f6aaea0b9df-node-exporter-tls\") pod \"node-exporter-6ctzx\" (UID: \"fa50d828-c196-4e48-9742-8f6aaea0b9df\") " pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:03.413485 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:03.413473 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ce40ab0-a188-4403-b608-575dcef41060-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-975lp\" (UID: \"8ce40ab0-a188-4403-b608-575dcef41060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-975lp" Sep 07 10:10:03.457755 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:03.457726 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-zczs4\"" Sep 07 10:10:03.462693 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:03.462668 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8phjk\"" Sep 07 10:10:03.466550 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:03.466528 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-975lp" Sep 07 10:10:03.470885 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:03.470860 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6ctzx" Sep 07 10:10:03.488094 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:10:03.488051 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa50d828_c196_4e48_9742_8f6aaea0b9df.slice/crio-defddbaa414b10db599b9964a88ca2f08a043fb2fd230605371eb9305d23ed58 WatchSource:0}: Error finding container defddbaa414b10db599b9964a88ca2f08a043fb2fd230605371eb9305d23ed58: Status 404 returned error can't find the container with id defddbaa414b10db599b9964a88ca2f08a043fb2fd230605371eb9305d23ed58 Sep 07 10:10:03.772911 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:03.772824 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-975lp"] Sep 07 10:10:03.776858 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:10:03.776825 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ce40ab0_a188_4403_b608_575dcef41060.slice/crio-49c38a848c0c874e38151224dc2097583c65d036ddb33723832a8d7c9031d3da WatchSource:0}: Error finding container 49c38a848c0c874e38151224dc2097583c65d036ddb33723832a8d7c9031d3da: Status 404 returned error can't find the container with id 49c38a848c0c874e38151224dc2097583c65d036ddb33723832a8d7c9031d3da Sep 07 10:10:04.322786 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.322739 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-975lp" event={"ID":"8ce40ab0-a188-4403-b608-575dcef41060","Type":"ContainerStarted","Data":"c23f792ce89dc268db7d3d96366c49ec00e762caad31f0df9c0c304180d31b9d"} Sep 07 10:10:04.322975 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.322838 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-975lp" event={"ID":"8ce40ab0-a188-4403-b608-575dcef41060","Type":"ContainerStarted","Data":"74ea81b3c63aee1e4010e449478cf15978f4b64c2653bdb421a73679510c486c"} Sep 07 10:10:04.322975 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.322872 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-975lp" event={"ID":"8ce40ab0-a188-4403-b608-575dcef41060","Type":"ContainerStarted","Data":"49c38a848c0c874e38151224dc2097583c65d036ddb33723832a8d7c9031d3da"} Sep 07 10:10:04.324254 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.324209 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6ctzx" event={"ID":"fa50d828-c196-4e48-9742-8f6aaea0b9df","Type":"ContainerStarted","Data":"defddbaa414b10db599b9964a88ca2f08a043fb2fd230605371eb9305d23ed58"} Sep 07 10:10:04.515720 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.514859 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-b4454c4f5-9svvk"] Sep 07 10:10:04.519199 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.519176 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.521555 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.521536 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 07 10:10:04.521555 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.521549 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-9r2tmm2chqnpu\"" Sep 07 10:10:04.521884 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.521865 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-rn2qv\"" Sep 07 10:10:04.521991 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.521901 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 07 10:10:04.521991 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.521984 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 07 10:10:04.522075 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.521986 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 07 10:10:04.522282 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.522270 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 07 10:10:04.527420 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.527399 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b4454c4f5-9svvk"] Sep 07 10:10:04.622503 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.622479 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6268e89a-3997-4ad1-bcbb-ac4496d1997b-secret-grpc-tls\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.622634 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.622532 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6268e89a-3997-4ad1-bcbb-ac4496d1997b-secret-thanos-querier-tls\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.622634 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.622602 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6268e89a-3997-4ad1-bcbb-ac4496d1997b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.622634 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.622629 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6268e89a-3997-4ad1-bcbb-ac4496d1997b-metrics-client-ca\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.622839 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.622659 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6268e89a-3997-4ad1-bcbb-ac4496d1997b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.622839 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.622682 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6268e89a-3997-4ad1-bcbb-ac4496d1997b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.622839 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.622699 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz8tv\" (UniqueName: \"kubernetes.io/projected/6268e89a-3997-4ad1-bcbb-ac4496d1997b-kube-api-access-wz8tv\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.622839 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.622720 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6268e89a-3997-4ad1-bcbb-ac4496d1997b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.723854 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.723818 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6268e89a-3997-4ad1-bcbb-ac4496d1997b-secret-thanos-querier-tls\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.724009 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.723906 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6268e89a-3997-4ad1-bcbb-ac4496d1997b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.724009 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.723952 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6268e89a-3997-4ad1-bcbb-ac4496d1997b-metrics-client-ca\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.724009 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.723987 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6268e89a-3997-4ad1-bcbb-ac4496d1997b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.724181 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.724011 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6268e89a-3997-4ad1-bcbb-ac4496d1997b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.724181 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.724035 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz8tv\" (UniqueName: \"kubernetes.io/projected/6268e89a-3997-4ad1-bcbb-ac4496d1997b-kube-api-access-wz8tv\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.724285 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.724203 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6268e89a-3997-4ad1-bcbb-ac4496d1997b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.724285 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.724255 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6268e89a-3997-4ad1-bcbb-ac4496d1997b-secret-grpc-tls\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.724829 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.724767 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6268e89a-3997-4ad1-bcbb-ac4496d1997b-metrics-client-ca\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.726899 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.726857 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6268e89a-3997-4ad1-bcbb-ac4496d1997b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.726899 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.726885 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6268e89a-3997-4ad1-bcbb-ac4496d1997b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.727071 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.727019 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6268e89a-3997-4ad1-bcbb-ac4496d1997b-secret-thanos-querier-tls\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.727209 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.727183 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6268e89a-3997-4ad1-bcbb-ac4496d1997b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.727363 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.727345 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6268e89a-3997-4ad1-bcbb-ac4496d1997b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.727788 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.727768 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6268e89a-3997-4ad1-bcbb-ac4496d1997b-secret-grpc-tls\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.732378 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.732358 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz8tv\" (UniqueName: \"kubernetes.io/projected/6268e89a-3997-4ad1-bcbb-ac4496d1997b-kube-api-access-wz8tv\") pod \"thanos-querier-b4454c4f5-9svvk\" (UID: \"6268e89a-3997-4ad1-bcbb-ac4496d1997b\") " pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:04.829110 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:04.829018 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:05.111619 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:05.111532 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b4454c4f5-9svvk"] Sep 07 10:10:05.129644 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:10:05.129613 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6268e89a_3997_4ad1_bcbb_ac4496d1997b.slice/crio-ab2c32e74159af291fd7a31b0ef76ffdfdbcd52506b547b9a732ac556cd7f156 WatchSource:0}: Error finding container ab2c32e74159af291fd7a31b0ef76ffdfdbcd52506b547b9a732ac556cd7f156: Status 404 returned error can't find the container with id ab2c32e74159af291fd7a31b0ef76ffdfdbcd52506b547b9a732ac556cd7f156 Sep 07 10:10:05.329438 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:05.329399 2575 generic.go:356] "Generic (PLEG): container finished" podID="fa50d828-c196-4e48-9742-8f6aaea0b9df" containerID="b3afb1762f588bbb566adb14f9a65b834a84bdf392108cd5bfa95d847c61f6c1" exitCode=0 Sep 07 10:10:05.329607 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:05.329439 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6ctzx" event={"ID":"fa50d828-c196-4e48-9742-8f6aaea0b9df","Type":"ContainerDied","Data":"b3afb1762f588bbb566adb14f9a65b834a84bdf392108cd5bfa95d847c61f6c1"} Sep 07 10:10:05.331779 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:05.331754 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-975lp" event={"ID":"8ce40ab0-a188-4403-b608-575dcef41060","Type":"ContainerStarted","Data":"10e145c3d2180fc9c14a41f63845f66dc487f4bddfd15c47cc25eb78f1501476"} Sep 07 10:10:05.333137 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:05.333114 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" event={"ID":"6268e89a-3997-4ad1-bcbb-ac4496d1997b","Type":"ContainerStarted","Data":"ab2c32e74159af291fd7a31b0ef76ffdfdbcd52506b547b9a732ac556cd7f156"} Sep 07 10:10:05.361705 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:05.361618 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-975lp" podStartSLOduration=2.111894316 podStartE2EDuration="3.361603309s" totalImagesPullingTime="1.249708993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:10:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:10:05.36040742 +0000 UTC m=+123.072649367" watchObservedRunningTime="2026-09-07 10:10:05.361603309 +0000 UTC m=+123.073845253" Sep 07 10:10:06.340704 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:06.340660 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6ctzx" event={"ID":"fa50d828-c196-4e48-9742-8f6aaea0b9df","Type":"ContainerStarted","Data":"71289f7ee0e18b40949ae45794948aab4e34c5308a9cecf8968ca69bb5f485bc"} Sep 07 10:10:06.341129 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:06.340735 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6ctzx" event={"ID":"fa50d828-c196-4e48-9742-8f6aaea0b9df","Type":"ContainerStarted","Data":"ada1194873b8f7c2ed3f65132a32323124b0eef585d609e286959335e7f63452"} Sep 07 10:10:06.358827 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:06.358747 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6ctzx" podStartSLOduration=3.219833474 podStartE2EDuration="4.358726035s" totalImagesPullingTime="1.076935224s" totalInitContainerRuntime="61.957337ms" isStatefulPod=true podCreationTimestamp="2026-09-07 10:10:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:10:06.357822137 +0000 UTC m=+124.070064082" watchObservedRunningTime="2026-09-07 10:10:06.358726035 +0000 UTC m=+124.070967981" Sep 07 10:10:06.947866 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:06.947832 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7f5864f4f-6d2qt"] Sep 07 10:10:06.952620 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:06.952586 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:06.955159 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:06.955133 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 07 10:10:06.956004 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:06.955982 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4fq3r4plgh66t\"" Sep 07 10:10:06.956134 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:06.956010 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 07 10:10:06.956134 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:06.955984 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-4tg5s\"" Sep 07 10:10:06.956134 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:06.956050 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 07 10:10:06.956134 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:06.956061 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 07 10:10:06.958650 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:06.958626 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7f5864f4f-6d2qt"] Sep 07 10:10:07.045155 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.045103 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ff48b6c0-a453-4d1d-8851-ded97f4ca8ed-audit-log\") pod \"metrics-server-7f5864f4f-6d2qt\" (UID: \"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed\") " pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.045155 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.045163 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ff48b6c0-a453-4d1d-8851-ded97f4ca8ed-client-ca-bundle\") pod \"metrics-server-7f5864f4f-6d2qt\" (UID: \"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed\") " pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.045424 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.045279 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbpjq\" (UniqueName: \"kubernetes.io/projected/ff48b6c0-a453-4d1d-8851-ded97f4ca8ed-kube-api-access-dbpjq\") pod \"metrics-server-7f5864f4f-6d2qt\" (UID: \"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed\") " pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.045424 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.045336 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ff48b6c0-a453-4d1d-8851-ded97f4ca8ed-secret-metrics-server-tls\") pod \"metrics-server-7f5864f4f-6d2qt\" (UID: \"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed\") " pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.045424 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.045404 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff48b6c0-a453-4d1d-8851-ded97f4ca8ed-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f5864f4f-6d2qt\" (UID: \"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed\") " pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.045565 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.045443 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ff48b6c0-a453-4d1d-8851-ded97f4ca8ed-secret-metrics-server-client-certs\") pod \"metrics-server-7f5864f4f-6d2qt\" (UID: \"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed\") " pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.045565 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.045470 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ff48b6c0-a453-4d1d-8851-ded97f4ca8ed-metrics-server-audit-profiles\") pod \"metrics-server-7f5864f4f-6d2qt\" (UID: \"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed\") " pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.145841 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.145803 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ff48b6c0-a453-4d1d-8851-ded97f4ca8ed-audit-log\") pod \"metrics-server-7f5864f4f-6d2qt\" (UID: \"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed\") " pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.146005 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.145855 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ff48b6c0-a453-4d1d-8851-ded97f4ca8ed-client-ca-bundle\") pod \"metrics-server-7f5864f4f-6d2qt\" (UID: \"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed\") " pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.146005 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.145891 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbpjq\" (UniqueName: \"kubernetes.io/projected/ff48b6c0-a453-4d1d-8851-ded97f4ca8ed-kube-api-access-dbpjq\") pod \"metrics-server-7f5864f4f-6d2qt\" (UID: \"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed\") " pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.146005 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.145922 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ff48b6c0-a453-4d1d-8851-ded97f4ca8ed-secret-metrics-server-tls\") pod \"metrics-server-7f5864f4f-6d2qt\" (UID: \"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed\") " pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.146005 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.145958 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff48b6c0-a453-4d1d-8851-ded97f4ca8ed-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f5864f4f-6d2qt\" (UID: \"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed\") " pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.146005 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.145995 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ff48b6c0-a453-4d1d-8851-ded97f4ca8ed-secret-metrics-server-client-certs\") pod \"metrics-server-7f5864f4f-6d2qt\" (UID: \"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed\") " pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.146260 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.146022 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ff48b6c0-a453-4d1d-8851-ded97f4ca8ed-metrics-server-audit-profiles\") pod \"metrics-server-7f5864f4f-6d2qt\" (UID: \"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed\") " pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.146260 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.146215 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ff48b6c0-a453-4d1d-8851-ded97f4ca8ed-audit-log\") pod \"metrics-server-7f5864f4f-6d2qt\" (UID: \"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed\") " pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.146782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.146751 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff48b6c0-a453-4d1d-8851-ded97f4ca8ed-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f5864f4f-6d2qt\" (UID: \"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed\") " pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.146947 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.146921 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ff48b6c0-a453-4d1d-8851-ded97f4ca8ed-metrics-server-audit-profiles\") pod \"metrics-server-7f5864f4f-6d2qt\" (UID: \"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed\") " pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.148394 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.148367 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ff48b6c0-a453-4d1d-8851-ded97f4ca8ed-client-ca-bundle\") pod \"metrics-server-7f5864f4f-6d2qt\" (UID: \"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed\") " pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.148503 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.148433 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ff48b6c0-a453-4d1d-8851-ded97f4ca8ed-secret-metrics-server-client-certs\") pod \"metrics-server-7f5864f4f-6d2qt\" (UID: \"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed\") " pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.148503 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.148498 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ff48b6c0-a453-4d1d-8851-ded97f4ca8ed-secret-metrics-server-tls\") pod \"metrics-server-7f5864f4f-6d2qt\" (UID: \"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed\") " pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.153663 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.153640 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbpjq\" (UniqueName: \"kubernetes.io/projected/ff48b6c0-a453-4d1d-8851-ded97f4ca8ed-kube-api-access-dbpjq\") pod \"metrics-server-7f5864f4f-6d2qt\" (UID: \"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed\") " pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.264888 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.264861 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:07.348693 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.348657 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" event={"ID":"6268e89a-3997-4ad1-bcbb-ac4496d1997b","Type":"ContainerStarted","Data":"5f5d87fcb20c7acd2a025d0c26abf6013b3088dd52e47049482efcc85c3f6d86"} Sep 07 10:10:07.349215 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.348745 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" event={"ID":"6268e89a-3997-4ad1-bcbb-ac4496d1997b","Type":"ContainerStarted","Data":"4d19fffc13b0e3b624a4f77f8964d893270e85d7fc51f243ec3cd14821cdfbfc"} Sep 07 10:10:07.360773 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.360746 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-kfqn5"] Sep 07 10:10:07.365661 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.365062 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kfqn5" Sep 07 10:10:07.368843 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.368817 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-sh427\"" Sep 07 10:10:07.368972 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.368791 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 07 10:10:07.380467 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.380447 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-kfqn5"] Sep 07 10:10:07.447856 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.447791 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hl48\" (UniqueName: \"kubernetes.io/projected/b37db823-b8fb-448c-9556-a28cd0896239-kube-api-access-7hl48\") pod \"monitoring-plugin-85bbbbbf4-kfqn5\" (UID: \"b37db823-b8fb-448c-9556-a28cd0896239\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kfqn5" Sep 07 10:10:07.447856 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.447862 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b37db823-b8fb-448c-9556-a28cd0896239-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-kfqn5\" (UID: \"b37db823-b8fb-448c-9556-a28cd0896239\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kfqn5" Sep 07 10:10:07.540756 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.540672 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7f5864f4f-6d2qt"] Sep 07 10:10:07.543475 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:10:07.543450 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff48b6c0_a453_4d1d_8851_ded97f4ca8ed.slice/crio-b83e67f62a2a3b62832a14934f327879b6813cceda07c5c7db244c8c03ad4511 WatchSource:0}: Error finding container b83e67f62a2a3b62832a14934f327879b6813cceda07c5c7db244c8c03ad4511: Status 404 returned error can't find the container with id b83e67f62a2a3b62832a14934f327879b6813cceda07c5c7db244c8c03ad4511 Sep 07 10:10:07.548308 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.548286 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hl48\" (UniqueName: \"kubernetes.io/projected/b37db823-b8fb-448c-9556-a28cd0896239-kube-api-access-7hl48\") pod \"monitoring-plugin-85bbbbbf4-kfqn5\" (UID: \"b37db823-b8fb-448c-9556-a28cd0896239\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kfqn5" Sep 07 10:10:07.548378 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.548319 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b37db823-b8fb-448c-9556-a28cd0896239-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-kfqn5\" (UID: \"b37db823-b8fb-448c-9556-a28cd0896239\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kfqn5" Sep 07 10:10:07.548470 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:10:07.548457 2575 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 07 10:10:07.548523 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:10:07.548511 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b37db823-b8fb-448c-9556-a28cd0896239-monitoring-plugin-cert podName:b37db823-b8fb-448c-9556-a28cd0896239 nodeName:}" failed. No retries permitted until 2026-09-07 10:10:08.048493761 +0000 UTC m=+125.760735685 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/b37db823-b8fb-448c-9556-a28cd0896239-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-kfqn5" (UID: "b37db823-b8fb-448c-9556-a28cd0896239") : secret "monitoring-plugin-cert" not found Sep 07 10:10:07.557950 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:07.557928 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hl48\" (UniqueName: \"kubernetes.io/projected/b37db823-b8fb-448c-9556-a28cd0896239-kube-api-access-7hl48\") pod \"monitoring-plugin-85bbbbbf4-kfqn5\" (UID: \"b37db823-b8fb-448c-9556-a28cd0896239\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kfqn5" Sep 07 10:10:08.051526 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:08.051483 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b37db823-b8fb-448c-9556-a28cd0896239-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-kfqn5\" (UID: \"b37db823-b8fb-448c-9556-a28cd0896239\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kfqn5" Sep 07 10:10:08.054264 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:08.054231 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b37db823-b8fb-448c-9556-a28cd0896239-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-kfqn5\" (UID: \"b37db823-b8fb-448c-9556-a28cd0896239\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kfqn5" Sep 07 10:10:08.274587 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:08.274477 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kfqn5" Sep 07 10:10:08.360175 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:08.360116 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" event={"ID":"6268e89a-3997-4ad1-bcbb-ac4496d1997b","Type":"ContainerStarted","Data":"5021921237a4bd53f5bc6e6362c799585af2db517f1115891cfc30cec67b68ba"} Sep 07 10:10:08.360573 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:08.360224 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" event={"ID":"6268e89a-3997-4ad1-bcbb-ac4496d1997b","Type":"ContainerStarted","Data":"c3ae881948d8a9f5179a20e6bb4e36f62e12ad0cbacb941900c4c6f637ae72ab"} Sep 07 10:10:08.360573 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:08.360267 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" event={"ID":"6268e89a-3997-4ad1-bcbb-ac4496d1997b","Type":"ContainerStarted","Data":"82654afd15b2ae8e66fd5761f00abc991d9de6e2ad39a25ff84f94892ea22f4d"} Sep 07 10:10:08.362084 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:08.362051 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" event={"ID":"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed","Type":"ContainerStarted","Data":"b83e67f62a2a3b62832a14934f327879b6813cceda07c5c7db244c8c03ad4511"} Sep 07 10:10:08.557225 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:08.557188 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-kfqn5"] Sep 07 10:10:08.561529 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:10:08.561500 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb37db823_b8fb_448c_9556_a28cd0896239.slice/crio-439b57e03666bbcbdb858641b71c47e5cd7d43493781bef5f76096d599dfec6c WatchSource:0}: Error finding container 439b57e03666bbcbdb858641b71c47e5cd7d43493781bef5f76096d599dfec6c: Status 404 returned error can't find the container with id 439b57e03666bbcbdb858641b71c47e5cd7d43493781bef5f76096d599dfec6c Sep 07 10:10:09.369864 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:09.369821 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" event={"ID":"6268e89a-3997-4ad1-bcbb-ac4496d1997b","Type":"ContainerStarted","Data":"db04736fac48a4ef2ef5453db39b4dbd25d331fad57c53bc765d521b8d9697a8"} Sep 07 10:10:09.370306 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:09.370147 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:09.371853 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:09.371826 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" event={"ID":"ff48b6c0-a453-4d1d-8851-ded97f4ca8ed","Type":"ContainerStarted","Data":"d94914125efbbbb7c3409f93147af740c47db7fa1908838bee1629befa9c5ab4"} Sep 07 10:10:09.373403 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:09.373380 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kfqn5" event={"ID":"b37db823-b8fb-448c-9556-a28cd0896239","Type":"ContainerStarted","Data":"439b57e03666bbcbdb858641b71c47e5cd7d43493781bef5f76096d599dfec6c"} Sep 07 10:10:09.391396 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:09.391346 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" podStartSLOduration=2.581794435 podStartE2EDuration="5.391333597s" totalImagesPullingTime="2.809539162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:10:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:10:09.389991954 +0000 UTC m=+127.102233924" watchObservedRunningTime="2026-09-07 10:10:09.391333597 +0000 UTC m=+127.103575575" Sep 07 10:10:09.405984 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:09.405887 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" podStartSLOduration=1.7961052610000001 podStartE2EDuration="3.405869635s" totalImagesPullingTime="1.609764374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:10:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:10:09.405371573 +0000 UTC m=+127.117613520" watchObservedRunningTime="2026-09-07 10:10:09.405869635 +0000 UTC m=+127.118111561" Sep 07 10:10:10.378251 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:10.378148 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kfqn5" event={"ID":"b37db823-b8fb-448c-9556-a28cd0896239","Type":"ContainerStarted","Data":"524fffb8bab0b99d3bcb31ca2c2304ee1fd49ec4055bba9ba9f616070e3235e4"} Sep 07 10:10:10.378686 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:10.378499 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kfqn5" Sep 07 10:10:10.382753 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:10.382731 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kfqn5" Sep 07 10:10:10.394236 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:10.394197 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-kfqn5" podStartSLOduration=1.8619447789999999 podStartE2EDuration="3.394183978s" totalImagesPullingTime="1.532239199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:10:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:10:10.393913958 +0000 UTC m=+128.106155906" watchObservedRunningTime="2026-09-07 10:10:10.394183978 +0000 UTC m=+128.106425957" Sep 07 10:10:12.692721 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.692677 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs\") pod \"network-metrics-daemon-4wfhx\" (UID: \"d11a5179-d3c5-48d5-a7c6-a7c367fd2584\") " pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:10:12.695016 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.694989 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d11a5179-d3c5-48d5-a7c6-a7c367fd2584-metrics-certs\") pod \"network-metrics-daemon-4wfhx\" (UID: \"d11a5179-d3c5-48d5-a7c6-a7c367fd2584\") " pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:10:12.767175 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.767141 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b946bc75-m5t7g"] Sep 07 10:10:12.770743 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.770725 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:12.773645 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.773620 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 07 10:10:12.773645 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.773635 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 07 10:10:12.774487 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.774469 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 07 10:10:12.774600 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.774493 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 07 10:10:12.774893 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.774861 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 07 10:10:12.774893 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.774867 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 07 10:10:12.775044 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.774919 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-96cg8\"" Sep 07 10:10:12.775044 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.774933 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 07 10:10:12.779341 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.779293 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 07 10:10:12.780022 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.780004 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b946bc75-m5t7g"] Sep 07 10:10:12.793166 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.793139 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-console-config\") pod \"console-b946bc75-m5t7g\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:12.793292 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.793181 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-console-oauth-config\") pod \"console-b946bc75-m5t7g\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:12.793292 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.793237 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-oauth-serving-cert\") pod \"console-b946bc75-m5t7g\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:12.793376 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.793296 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-trusted-ca-bundle\") pod \"console-b946bc75-m5t7g\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:12.793376 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.793319 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-console-serving-cert\") pod \"console-b946bc75-m5t7g\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:12.793451 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.793375 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzksj\" (UniqueName: \"kubernetes.io/projected/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-kube-api-access-nzksj\") pod \"console-b946bc75-m5t7g\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:12.793451 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.793443 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-service-ca\") pod \"console-b946bc75-m5t7g\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:12.846566 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.846535 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2frx2\"" Sep 07 10:10:12.855247 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.855213 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wfhx" Sep 07 10:10:12.894171 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.894013 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-oauth-serving-cert\") pod \"console-b946bc75-m5t7g\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:12.894171 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.894085 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-trusted-ca-bundle\") pod \"console-b946bc75-m5t7g\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:12.894171 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.894123 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-console-serving-cert\") pod \"console-b946bc75-m5t7g\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:12.894171 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.894156 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzksj\" (UniqueName: \"kubernetes.io/projected/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-kube-api-access-nzksj\") pod \"console-b946bc75-m5t7g\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:12.894512 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.894199 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-service-ca\") pod \"console-b946bc75-m5t7g\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:12.894512 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.894233 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-console-config\") pod \"console-b946bc75-m5t7g\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:12.894512 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.894269 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-console-oauth-config\") pod \"console-b946bc75-m5t7g\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:12.895788 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.895694 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-console-config\") pod \"console-b946bc75-m5t7g\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:12.895788 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.895704 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-oauth-serving-cert\") pod \"console-b946bc75-m5t7g\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:12.895788 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.895725 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-trusted-ca-bundle\") pod \"console-b946bc75-m5t7g\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:12.896040 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.895992 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-service-ca\") pod \"console-b946bc75-m5t7g\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:12.896946 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.896923 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-console-oauth-config\") pod \"console-b946bc75-m5t7g\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:12.897280 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.897264 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-console-serving-cert\") pod \"console-b946bc75-m5t7g\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:12.903417 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:12.903371 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzksj\" (UniqueName: \"kubernetes.io/projected/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-kube-api-access-nzksj\") pod \"console-b946bc75-m5t7g\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:13.080757 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:13.080718 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:13.125895 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:13.125868 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4wfhx"] Sep 07 10:10:13.129202 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:10:13.129129 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd11a5179_d3c5_48d5_a7c6_a7c367fd2584.slice/crio-a408cd57366c423706a763ee9e24a63d90c313a610a2c98055cf1b94d2ba611c WatchSource:0}: Error finding container a408cd57366c423706a763ee9e24a63d90c313a610a2c98055cf1b94d2ba611c: Status 404 returned error can't find the container with id a408cd57366c423706a763ee9e24a63d90c313a610a2c98055cf1b94d2ba611c Sep 07 10:10:13.351167 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:13.351138 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b946bc75-m5t7g"] Sep 07 10:10:13.353622 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:10:13.353596 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda58c35c0_ff40_4a2a_aa0a_6bbbeaa610b1.slice/crio-3170a53353dba818bc5e76812805c1753fdd39463f26a4d2033ce02f4e3682cb WatchSource:0}: Error finding container 3170a53353dba818bc5e76812805c1753fdd39463f26a4d2033ce02f4e3682cb: Status 404 returned error can't find the container with id 3170a53353dba818bc5e76812805c1753fdd39463f26a4d2033ce02f4e3682cb Sep 07 10:10:13.389582 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:13.389546 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4wfhx" event={"ID":"d11a5179-d3c5-48d5-a7c6-a7c367fd2584","Type":"ContainerStarted","Data":"a408cd57366c423706a763ee9e24a63d90c313a610a2c98055cf1b94d2ba611c"} Sep 07 10:10:13.390746 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:13.390724 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b946bc75-m5t7g" event={"ID":"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1","Type":"ContainerStarted","Data":"3170a53353dba818bc5e76812805c1753fdd39463f26a4d2033ce02f4e3682cb"} Sep 07 10:10:14.398744 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:14.398705 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4wfhx" event={"ID":"d11a5179-d3c5-48d5-a7c6-a7c367fd2584","Type":"ContainerStarted","Data":"b4bc2898fa94007d8666b8a7a09e1568a4b59a96b41886e69735c2d3e419b3f2"} Sep 07 10:10:15.386243 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:15.386207 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-b4454c4f5-9svvk" Sep 07 10:10:15.404777 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:15.404738 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4wfhx" event={"ID":"d11a5179-d3c5-48d5-a7c6-a7c367fd2584","Type":"ContainerStarted","Data":"b2f61fbe92ed483c9ecdd318d42e0d668b05736ae711c6905f9d4b099fe1b0e6"} Sep 07 10:10:15.424767 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:15.424705 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4wfhx" podStartSLOduration=131.324854301 podStartE2EDuration="2m12.424683454s" totalImagesPullingTime="1.099829153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:10:15.423878986 +0000 UTC m=+133.136120944" watchObservedRunningTime="2026-09-07 10:10:15.424683454 +0000 UTC m=+133.136925419" Sep 07 10:10:15.562074 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:15.562029 2575 patch_prober.go:28] interesting pod/image-registry-86f7794bb5-pxqrg container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 07 10:10:15.562236 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:15.562135 2575 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" podUID="a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 07 10:10:16.410028 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:16.409990 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b946bc75-m5t7g" event={"ID":"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1","Type":"ContainerStarted","Data":"677672afa4399c74125a8b1fc69af1e3d0254eeab62978222089853cb2a0b67d"} Sep 07 10:10:16.427006 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:16.426953 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b946bc75-m5t7g" podStartSLOduration=1.548914685 podStartE2EDuration="4.426931312s" totalImagesPullingTime="2.878016627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:10:16.425702679 +0000 UTC m=+134.137944640" watchObservedRunningTime="2026-09-07 10:10:16.426931312 +0000 UTC m=+134.139173256" Sep 07 10:10:17.294180 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:17.294145 2575 patch_prober.go:28] interesting pod/image-registry-86f7794bb5-pxqrg container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 07 10:10:17.294339 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:17.294197 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" podUID="a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 07 10:10:23.081284 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:23.081244 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:23.081737 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:23.081520 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:23.082730 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:23.082707 2575 patch_prober.go:28] interesting pod/console-b946bc75-m5t7g container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.20:8443/health\": dial tcp 10.133.0.20:8443: connect: connection refused" start-of-body= Sep 07 10:10:23.082844 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:23.082754 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-b946bc75-m5t7g" podUID="a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1" containerName="console" probeResult="failure" output="Get \"https://10.133.0.20:8443/health\": dial tcp 10.133.0.20:8443: connect: connection refused" Sep 07 10:10:25.560383 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:25.560342 2575 patch_prober.go:28] interesting pod/image-registry-86f7794bb5-pxqrg container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 07 10:10:25.560781 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:25.560395 2575 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" podUID="a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 07 10:10:27.265566 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:27.265526 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:27.265981 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:27.265620 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:27.293999 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:27.293967 2575 patch_prober.go:28] interesting pod/image-registry-86f7794bb5-pxqrg container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 07 10:10:27.294123 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:27.294018 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" podUID="a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 07 10:10:33.082052 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:33.082007 2575 patch_prober.go:28] interesting pod/console-b946bc75-m5t7g container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.20:8443/health\": dial tcp 10.133.0.20:8443: connect: connection refused" start-of-body= Sep 07 10:10:33.082512 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:33.082087 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-b946bc75-m5t7g" podUID="a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1" containerName="console" probeResult="failure" output="Get \"https://10.133.0.20:8443/health\": dial tcp 10.133.0.20:8443: connect: connection refused" Sep 07 10:10:35.560383 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:35.560349 2575 patch_prober.go:28] interesting pod/image-registry-86f7794bb5-pxqrg container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 07 10:10:35.560829 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:35.560400 2575 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" podUID="a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 07 10:10:35.560829 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:35.560437 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:10:35.560929 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:35.560895 2575 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"0bd4f2d75e03d8a14843975819a3432b85a18f7d7c347a5a1619e384c7c51d1b"} pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" containerMessage="Container registry failed liveness probe, will be restarted" Sep 07 10:10:35.564591 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:35.564558 2575 patch_prober.go:28] interesting pod/image-registry-86f7794bb5-pxqrg container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 07 10:10:35.564707 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:35.564618 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" podUID="a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 07 10:10:39.687077 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:10:39.687026 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-nc7j5" podUID="b07b7355-ed97-4a04-958b-c675ec096d60" Sep 07 10:10:40.498935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:40.498855 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nc7j5" Sep 07 10:10:43.086433 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:43.086403 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:43.090188 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:43.090167 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:10:43.104272 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:43.104228 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-dx9lz_502e0127-90b6-4388-860b-5dd7979db869/cluster-monitoring-operator/0.log" Sep 07 10:10:43.902577 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:43.902545 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7f5864f4f-6d2qt_ff48b6c0-a453-4d1d-8851-ded97f4ca8ed/metrics-server/0.log" Sep 07 10:10:44.102841 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:44.102747 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-kfqn5_b37db823-b8fb-448c-9556-a28cd0896239/monitoring-plugin/0.log" Sep 07 10:10:44.302949 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:44.302876 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6ctzx_fa50d828-c196-4e48-9742-8f6aaea0b9df/init-textfile/0.log" Sep 07 10:10:44.502956 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:44.502928 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6ctzx_fa50d828-c196-4e48-9742-8f6aaea0b9df/node-exporter/0.log" Sep 07 10:10:44.582186 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:44.582157 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls\") pod \"dns-default-nc7j5\" (UID: \"b07b7355-ed97-4a04-958b-c675ec096d60\") " pod="openshift-dns/dns-default-nc7j5" Sep 07 10:10:44.584483 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:44.584459 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07b7355-ed97-4a04-958b-c675ec096d60-metrics-tls\") pod \"dns-default-nc7j5\" (UID: \"b07b7355-ed97-4a04-958b-c675ec096d60\") " pod="openshift-dns/dns-default-nc7j5" Sep 07 10:10:44.702257 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:44.702223 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5dlqq\"" Sep 07 10:10:44.702712 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:44.702695 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6ctzx_fa50d828-c196-4e48-9742-8f6aaea0b9df/kube-rbac-proxy/0.log" Sep 07 10:10:44.710577 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:44.710563 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nc7j5" Sep 07 10:10:44.982913 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:44.982827 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nc7j5"] Sep 07 10:10:44.986447 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:10:44.986422 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb07b7355_ed97_4a04_958b_c675ec096d60.slice/crio-1923fbcfe87a121a4fd59026b2b9d0950674e9f25c73685c8cf76932a81d6bda WatchSource:0}: Error finding container 1923fbcfe87a121a4fd59026b2b9d0950674e9f25c73685c8cf76932a81d6bda: Status 404 returned error can't find the container with id 1923fbcfe87a121a4fd59026b2b9d0950674e9f25c73685c8cf76932a81d6bda Sep 07 10:10:45.519541 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:45.519503 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nc7j5" event={"ID":"b07b7355-ed97-4a04-958b-c675ec096d60","Type":"ContainerStarted","Data":"1923fbcfe87a121a4fd59026b2b9d0950674e9f25c73685c8cf76932a81d6bda"} Sep 07 10:10:45.565432 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:45.565394 2575 patch_prober.go:28] interesting pod/image-registry-86f7794bb5-pxqrg container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 07 10:10:45.565625 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:45.565461 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" podUID="a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 07 10:10:46.102524 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:46.102444 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-975lp_8ce40ab0-a188-4403-b608-575dcef41060/kube-rbac-proxy-main/0.log" Sep 07 10:10:46.303481 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:46.303452 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-975lp_8ce40ab0-a188-4403-b608-575dcef41060/kube-rbac-proxy-self/0.log" Sep 07 10:10:46.502445 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:46.502353 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-975lp_8ce40ab0-a188-4403-b608-575dcef41060/openshift-state-metrics/0.log" Sep 07 10:10:47.270595 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:47.270571 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:47.274609 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:47.274586 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7f5864f4f-6d2qt" Sep 07 10:10:47.529677 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:47.529634 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nc7j5" event={"ID":"b07b7355-ed97-4a04-958b-c675ec096d60","Type":"ContainerStarted","Data":"8a5e12b20a7c92fec768c70dee74d0b82ac8276c213025e54f260fd6b50e182b"} Sep 07 10:10:47.529875 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:47.529718 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nc7j5" event={"ID":"b07b7355-ed97-4a04-958b-c675ec096d60","Type":"ContainerStarted","Data":"531efc8d6cd27e46095ccc2be5389f985b85ac707135586b4bb635e10662a376"} Sep 07 10:10:47.546529 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:47.546474 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nc7j5" podStartSLOduration=129.395397885 podStartE2EDuration="2m11.546458189s" totalImagesPullingTime="2.151060304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:08:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:10:47.54469046 +0000 UTC m=+165.256932407" watchObservedRunningTime="2026-09-07 10:10:47.546458189 +0000 UTC m=+165.258700134" Sep 07 10:10:48.103157 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:48.103128 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-djr8f_4d198470-13aa-4025-a865-7d1f9f8ca2cc/prometheus-operator/0.log" Sep 07 10:10:48.302209 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:48.302182 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-djr8f_4d198470-13aa-4025-a865-7d1f9f8ca2cc/kube-rbac-proxy/0.log" Sep 07 10:10:48.533483 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:48.533387 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nc7j5" Sep 07 10:10:48.703582 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:48.703552 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b4454c4f5-9svvk_6268e89a-3997-4ad1-bcbb-ac4496d1997b/thanos-query/0.log" Sep 07 10:10:48.902730 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:48.902703 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b4454c4f5-9svvk_6268e89a-3997-4ad1-bcbb-ac4496d1997b/kube-rbac-proxy-web/0.log" Sep 07 10:10:49.102523 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:49.102484 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b4454c4f5-9svvk_6268e89a-3997-4ad1-bcbb-ac4496d1997b/kube-rbac-proxy/0.log" Sep 07 10:10:49.302312 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:49.302236 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b4454c4f5-9svvk_6268e89a-3997-4ad1-bcbb-ac4496d1997b/prom-label-proxy/0.log" Sep 07 10:10:49.502647 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:49.502620 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b4454c4f5-9svvk_6268e89a-3997-4ad1-bcbb-ac4496d1997b/kube-rbac-proxy-rules/0.log" Sep 07 10:10:49.702073 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:49.702041 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b4454c4f5-9svvk_6268e89a-3997-4ad1-bcbb-ac4496d1997b/kube-rbac-proxy-metrics/0.log" Sep 07 10:10:50.102215 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:50.102186 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-m6mmt_6aad73e9-87ee-49b0-8651-d3fd8390e4e4/console-operator/1.log" Sep 07 10:10:50.303121 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:50.303095 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-m6mmt_6aad73e9-87ee-49b0-8651-d3fd8390e4e4/console-operator/2.log" Sep 07 10:10:50.902911 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:50.902884 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-b946bc75-m5t7g_a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1/console/0.log" Sep 07 10:10:51.702498 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:51.702470 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nc7j5_b07b7355-ed97-4a04-958b-c675ec096d60/dns/0.log" Sep 07 10:10:51.905102 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:51.905075 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nc7j5_b07b7355-ed97-4a04-958b-c675ec096d60/kube-rbac-proxy/0.log" Sep 07 10:10:52.901970 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:52.901936 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w5w2p_19524835-4647-4786-bc0f-c222dd09c075/dns-node-resolver/0.log" Sep 07 10:10:53.304355 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:53.304282 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-86f7794bb5-pxqrg_a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc/registry/0.log" Sep 07 10:10:53.702357 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:53.702324 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t6ztt_a6a30db9-93e6-41b1-a0e6-28e5088df925/node-ca/0.log" Sep 07 10:10:54.103158 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:54.103131 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c558849d-hzpf7_c3c51a39-edc3-44e7-a0ed-843ac7ec5059/router/0.log" Sep 07 10:10:54.502746 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:54.502670 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jfq75_1352b671-2ab8-4c45-8f10-f82960e1ff1c/serve-healthcheck-canary/0.log" Sep 07 10:10:54.557370 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:54.557337 2575 generic.go:356] "Generic (PLEG): container finished" podID="70c229fe-b5da-413c-b231-625155198200" containerID="73e7bcbc253c28f6acf091a7e9366e080f0d2eef3574adbc1ef40fba58debd73" exitCode=0 Sep 07 10:10:54.557506 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:54.557411 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8bl4l" event={"ID":"70c229fe-b5da-413c-b231-625155198200","Type":"ContainerDied","Data":"73e7bcbc253c28f6acf091a7e9366e080f0d2eef3574adbc1ef40fba58debd73"} Sep 07 10:10:54.557755 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:54.557740 2575 scope.go:117] "RemoveContainer" containerID="73e7bcbc253c28f6acf091a7e9366e080f0d2eef3574adbc1ef40fba58debd73" Sep 07 10:10:55.563778 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:55.563745 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8bl4l" event={"ID":"70c229fe-b5da-413c-b231-625155198200","Type":"ContainerStarted","Data":"ea86a5ea1ea6f3d4d095a948cf16998e4a550ec2fe9e3c23b6b65028f78c18ac"} Sep 07 10:10:55.566076 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:55.566050 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:10:58.540279 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:10:58.540237 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nc7j5" Sep 07 10:11:00.580138 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:11:00.580095 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" podUID="a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc" containerName="registry" containerID="cri-o://0bd4f2d75e03d8a14843975819a3432b85a18f7d7c347a5a1619e384c7c51d1b" gracePeriod=30 Sep 07 10:11:02.596180 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:11:02.596136 2575 generic.go:356] "Generic (PLEG): container finished" podID="a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc" containerID="0bd4f2d75e03d8a14843975819a3432b85a18f7d7c347a5a1619e384c7c51d1b" exitCode=0 Sep 07 10:11:02.596649 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:11:02.596218 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" event={"ID":"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc","Type":"ContainerDied","Data":"0bd4f2d75e03d8a14843975819a3432b85a18f7d7c347a5a1619e384c7c51d1b"} Sep 07 10:11:02.596649 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:11:02.596273 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:11:02.596649 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:11:02.596288 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" event={"ID":"a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc","Type":"ContainerStarted","Data":"dd74f7db4adbb5f4dce0dc072665d767f00e548df8a617a93b7a872d132110f1"} Sep 07 10:11:23.605587 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:11:23.605553 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-86f7794bb5-pxqrg" Sep 07 10:11:41.028849 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:11:41.028812 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b946bc75-m5t7g"] Sep 07 10:12:06.050694 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.050633 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-b946bc75-m5t7g" podUID="a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1" containerName="console" containerID="cri-o://677672afa4399c74125a8b1fc69af1e3d0254eeab62978222089853cb2a0b67d" gracePeriod=15 Sep 07 10:12:06.308735 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.308674 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:12:06.388190 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.388145 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-oauth-serving-cert\") pod \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " Sep 07 10:12:06.388362 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.388246 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-console-config\") pod \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " Sep 07 10:12:06.388362 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.388306 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-console-oauth-config\") pod \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " Sep 07 10:12:06.388362 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.388324 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-service-ca\") pod \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " Sep 07 10:12:06.388362 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.388350 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzksj\" (UniqueName: \"kubernetes.io/projected/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-kube-api-access-nzksj\") pod \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " Sep 07 10:12:06.388552 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.388370 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-trusted-ca-bundle\") pod \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " Sep 07 10:12:06.388552 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.388393 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-console-serving-cert\") pod \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\" (UID: \"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1\") " Sep 07 10:12:06.388688 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.388664 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1" (UID: "a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 10:12:06.388754 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.388667 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-console-config" (OuterVolumeSpecName: "console-config") pod "a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1" (UID: "a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 10:12:06.388826 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.388786 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-service-ca" (OuterVolumeSpecName: "service-ca") pod "a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1" (UID: "a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 10:12:06.388882 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.388854 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1" (UID: "a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 10:12:06.390719 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.390696 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1" (UID: "a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:12:06.390819 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.390754 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1" (UID: "a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:12:06.390883 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.390833 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-kube-api-access-nzksj" (OuterVolumeSpecName: "kube-api-access-nzksj") pod "a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1" (UID: "a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1"). InnerVolumeSpecName "kube-api-access-nzksj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:12:06.489928 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.489887 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzksj\" (UniqueName: \"kubernetes.io/projected/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-kube-api-access-nzksj\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:12:06.489928 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.489920 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-trusted-ca-bundle\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:12:06.489928 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.489931 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-console-serving-cert\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:12:06.490151 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.489941 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-oauth-serving-cert\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:12:06.490151 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.489951 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-console-config\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:12:06.490151 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.489960 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-console-oauth-config\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:12:06.490151 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.489969 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1-service-ca\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:12:06.839476 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.839442 2575 generic.go:356] "Generic (PLEG): container finished" podID="a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1" containerID="677672afa4399c74125a8b1fc69af1e3d0254eeab62978222089853cb2a0b67d" exitCode=0 Sep 07 10:12:06.839600 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.839490 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b946bc75-m5t7g" Sep 07 10:12:06.839600 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.839496 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b946bc75-m5t7g" event={"ID":"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1","Type":"ContainerDied","Data":"677672afa4399c74125a8b1fc69af1e3d0254eeab62978222089853cb2a0b67d"} Sep 07 10:12:06.839600 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.839545 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b946bc75-m5t7g" event={"ID":"a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1","Type":"ContainerDied","Data":"3170a53353dba818bc5e76812805c1753fdd39463f26a4d2033ce02f4e3682cb"} Sep 07 10:12:06.839600 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.839574 2575 scope.go:117] "RemoveContainer" containerID="677672afa4399c74125a8b1fc69af1e3d0254eeab62978222089853cb2a0b67d" Sep 07 10:12:06.848361 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.848341 2575 scope.go:117] "RemoveContainer" containerID="677672afa4399c74125a8b1fc69af1e3d0254eeab62978222089853cb2a0b67d" Sep 07 10:12:06.848648 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:12:06.848627 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"677672afa4399c74125a8b1fc69af1e3d0254eeab62978222089853cb2a0b67d\": container with ID starting with 677672afa4399c74125a8b1fc69af1e3d0254eeab62978222089853cb2a0b67d not found: ID does not exist" containerID="677672afa4399c74125a8b1fc69af1e3d0254eeab62978222089853cb2a0b67d" Sep 07 10:12:06.848716 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.848661 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"677672afa4399c74125a8b1fc69af1e3d0254eeab62978222089853cb2a0b67d"} err="failed to get container status \"677672afa4399c74125a8b1fc69af1e3d0254eeab62978222089853cb2a0b67d\": rpc error: code = NotFound desc = could not find container \"677672afa4399c74125a8b1fc69af1e3d0254eeab62978222089853cb2a0b67d\": container with ID starting with 677672afa4399c74125a8b1fc69af1e3d0254eeab62978222089853cb2a0b67d not found: ID does not exist" Sep 07 10:12:06.857519 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.857489 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b946bc75-m5t7g"] Sep 07 10:12:06.862057 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:06.862034 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b946bc75-m5t7g"] Sep 07 10:12:08.834191 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:08.834155 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1" path="/var/lib/kubelet/pods/a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1/volumes" Sep 07 10:12:52.651731 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.651691 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5455ff9d57-tbcqj"] Sep 07 10:12:52.652200 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.652036 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1" containerName="console" Sep 07 10:12:52.652200 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.652051 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1" containerName="console" Sep 07 10:12:52.652200 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.652117 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a58c35c0-ff40-4a2a-aa0a-6bbbeaa610b1" containerName="console" Sep 07 10:12:52.655110 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.655092 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.657936 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.657915 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 07 10:12:52.658829 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.658809 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-96cg8\"" Sep 07 10:12:52.659872 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.659853 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 07 10:12:52.660590 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.660569 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 07 10:12:52.661981 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.661964 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 07 10:12:52.662059 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.661991 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 07 10:12:52.662244 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.662230 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 07 10:12:52.671323 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.671305 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 07 10:12:52.671900 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.671884 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 07 10:12:52.675652 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.675631 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5455ff9d57-tbcqj"] Sep 07 10:12:52.775867 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.775832 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-console-oauth-config\") pod \"console-5455ff9d57-tbcqj\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.775867 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.775867 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-trusted-ca-bundle\") pod \"console-5455ff9d57-tbcqj\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.776068 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.775890 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-console-config\") pod \"console-5455ff9d57-tbcqj\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.776068 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.775990 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-service-ca\") pod \"console-5455ff9d57-tbcqj\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.776068 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.776021 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-594p6\" (UniqueName: \"kubernetes.io/projected/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-kube-api-access-594p6\") pod \"console-5455ff9d57-tbcqj\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.776189 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.776092 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-oauth-serving-cert\") pod \"console-5455ff9d57-tbcqj\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.776189 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.776111 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-console-serving-cert\") pod \"console-5455ff9d57-tbcqj\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.877170 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.877133 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-oauth-serving-cert\") pod \"console-5455ff9d57-tbcqj\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.877170 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.877171 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-console-serving-cert\") pod \"console-5455ff9d57-tbcqj\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.877426 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.877220 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-console-oauth-config\") pod \"console-5455ff9d57-tbcqj\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.877426 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.877243 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-trusted-ca-bundle\") pod \"console-5455ff9d57-tbcqj\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.877426 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.877274 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-console-config\") pod \"console-5455ff9d57-tbcqj\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.877426 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.877311 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-service-ca\") pod \"console-5455ff9d57-tbcqj\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.877426 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.877330 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-594p6\" (UniqueName: \"kubernetes.io/projected/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-kube-api-access-594p6\") pod \"console-5455ff9d57-tbcqj\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.877979 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.877958 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-oauth-serving-cert\") pod \"console-5455ff9d57-tbcqj\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.878098 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.878076 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-console-config\") pod \"console-5455ff9d57-tbcqj\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.878164 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.878148 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-service-ca\") pod \"console-5455ff9d57-tbcqj\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.878271 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.878249 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-trusted-ca-bundle\") pod \"console-5455ff9d57-tbcqj\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.879760 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.879739 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-console-serving-cert\") pod \"console-5455ff9d57-tbcqj\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.879905 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.879885 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-console-oauth-config\") pod \"console-5455ff9d57-tbcqj\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.885865 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.885844 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-594p6\" (UniqueName: \"kubernetes.io/projected/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-kube-api-access-594p6\") pod \"console-5455ff9d57-tbcqj\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:52.964017 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:52.963933 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:12:53.233708 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:53.233460 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5455ff9d57-tbcqj"] Sep 07 10:12:53.236535 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:12:53.236504 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ce0aabd_47c5_42f1_9aa9_dfd7f17cbffb.slice/crio-27281a422a404f95e62485ffd0564a0b98c1bf674ff60e90e1097119c29db174 WatchSource:0}: Error finding container 27281a422a404f95e62485ffd0564a0b98c1bf674ff60e90e1097119c29db174: Status 404 returned error can't find the container with id 27281a422a404f95e62485ffd0564a0b98c1bf674ff60e90e1097119c29db174 Sep 07 10:12:54.019256 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:54.019219 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5455ff9d57-tbcqj" event={"ID":"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb","Type":"ContainerStarted","Data":"534104bd91b785c571cb0c8c68cf83266c6e558e911ceef05d8a2aebc0e674ab"} Sep 07 10:12:54.019604 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:54.019267 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5455ff9d57-tbcqj" event={"ID":"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb","Type":"ContainerStarted","Data":"27281a422a404f95e62485ffd0564a0b98c1bf674ff60e90e1097119c29db174"} Sep 07 10:12:54.038225 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:12:54.038177 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5455ff9d57-tbcqj" podStartSLOduration=2.038161607 podStartE2EDuration="2.038161607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:12:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:12:54.036323936 +0000 UTC m=+291.748565880" watchObservedRunningTime="2026-09-07 10:12:54.038161607 +0000 UTC m=+291.750403603" Sep 07 10:13:02.700637 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:13:02.700604 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-m6mmt_6aad73e9-87ee-49b0-8651-d3fd8390e4e4/console-operator/1.log" Sep 07 10:13:02.706065 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:13:02.706037 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-m6mmt_6aad73e9-87ee-49b0-8651-d3fd8390e4e4/console-operator/1.log" Sep 07 10:13:02.708735 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:13:02.708703 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/ovn-acl-logging/0.log" Sep 07 10:13:02.713058 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:13:02.713025 2575 kubelet.go:1648] "Image garbage collection succeeded" Sep 07 10:13:02.715291 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:13:02.715260 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/ovn-acl-logging/0.log" Sep 07 10:13:02.964480 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:13:02.964380 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:13:02.964480 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:13:02.964448 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:13:02.969303 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:13:02.969280 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:13:03.062424 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:13:03.062394 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:15:13.865275 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:13.865240 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5455ff9d57-tbcqj"] Sep 07 10:15:38.886332 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:38.886214 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5455ff9d57-tbcqj" podUID="6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb" containerName="console" containerID="cri-o://534104bd91b785c571cb0c8c68cf83266c6e558e911ceef05d8a2aebc0e674ab" gracePeriod=15 Sep 07 10:15:39.132471 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.132444 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:15:39.258013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.257922 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-console-oauth-config\") pod \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " Sep 07 10:15:39.258013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.257963 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-console-serving-cert\") pod \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " Sep 07 10:15:39.258013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.257996 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-oauth-serving-cert\") pod \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " Sep 07 10:15:39.258291 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.258031 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-console-config\") pod \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " Sep 07 10:15:39.258291 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.258079 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-trusted-ca-bundle\") pod \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " Sep 07 10:15:39.258291 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.258112 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-594p6\" (UniqueName: \"kubernetes.io/projected/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-kube-api-access-594p6\") pod \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " Sep 07 10:15:39.258291 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.258138 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-service-ca\") pod \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\" (UID: \"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb\") " Sep 07 10:15:39.258512 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.258454 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb" (UID: "6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 10:15:39.258599 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.258568 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-console-config" (OuterVolumeSpecName: "console-config") pod "6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb" (UID: "6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 10:15:39.258650 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.258591 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb" (UID: "6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 10:15:39.258650 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.258609 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-service-ca" (OuterVolumeSpecName: "service-ca") pod "6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb" (UID: "6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 10:15:39.260399 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.260372 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb" (UID: "6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:15:39.260503 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.260404 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb" (UID: "6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:15:39.260503 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.260423 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-kube-api-access-594p6" (OuterVolumeSpecName: "kube-api-access-594p6") pod "6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb" (UID: "6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb"). InnerVolumeSpecName "kube-api-access-594p6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:15:39.359075 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.359016 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-console-config\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:15:39.359075 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.359067 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-trusted-ca-bundle\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:15:39.359075 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.359079 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-594p6\" (UniqueName: \"kubernetes.io/projected/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-kube-api-access-594p6\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:15:39.359075 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.359088 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-service-ca\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:15:39.359075 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.359097 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-console-oauth-config\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:15:39.359368 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.359105 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-console-serving-cert\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:15:39.359368 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.359114 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb-oauth-serving-cert\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:15:39.674610 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.674578 2575 generic.go:356] "Generic (PLEG): container finished" podID="6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb" containerID="534104bd91b785c571cb0c8c68cf83266c6e558e911ceef05d8a2aebc0e674ab" exitCode=0 Sep 07 10:15:39.674785 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.674619 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5455ff9d57-tbcqj" event={"ID":"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb","Type":"ContainerDied","Data":"534104bd91b785c571cb0c8c68cf83266c6e558e911ceef05d8a2aebc0e674ab"} Sep 07 10:15:39.674785 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.674627 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5455ff9d57-tbcqj" Sep 07 10:15:39.674785 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.674655 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5455ff9d57-tbcqj" event={"ID":"6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb","Type":"ContainerDied","Data":"27281a422a404f95e62485ffd0564a0b98c1bf674ff60e90e1097119c29db174"} Sep 07 10:15:39.674785 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.674675 2575 scope.go:117] "RemoveContainer" containerID="534104bd91b785c571cb0c8c68cf83266c6e558e911ceef05d8a2aebc0e674ab" Sep 07 10:15:39.683377 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.683359 2575 scope.go:117] "RemoveContainer" containerID="534104bd91b785c571cb0c8c68cf83266c6e558e911ceef05d8a2aebc0e674ab" Sep 07 10:15:39.683681 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:15:39.683660 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"534104bd91b785c571cb0c8c68cf83266c6e558e911ceef05d8a2aebc0e674ab\": container with ID starting with 534104bd91b785c571cb0c8c68cf83266c6e558e911ceef05d8a2aebc0e674ab not found: ID does not exist" containerID="534104bd91b785c571cb0c8c68cf83266c6e558e911ceef05d8a2aebc0e674ab" Sep 07 10:15:39.683724 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.683691 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"534104bd91b785c571cb0c8c68cf83266c6e558e911ceef05d8a2aebc0e674ab"} err="failed to get container status \"534104bd91b785c571cb0c8c68cf83266c6e558e911ceef05d8a2aebc0e674ab\": rpc error: code = NotFound desc = could not find container \"534104bd91b785c571cb0c8c68cf83266c6e558e911ceef05d8a2aebc0e674ab\": container with ID starting with 534104bd91b785c571cb0c8c68cf83266c6e558e911ceef05d8a2aebc0e674ab not found: ID does not exist" Sep 07 10:15:39.696310 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.696278 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5455ff9d57-tbcqj"] Sep 07 10:15:39.701534 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:39.701505 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5455ff9d57-tbcqj"] Sep 07 10:15:40.830986 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:15:40.830944 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb" path="/var/lib/kubelet/pods/6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb/volumes" Sep 07 10:18:02.727954 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:18:02.727926 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-m6mmt_6aad73e9-87ee-49b0-8651-d3fd8390e4e4/console-operator/1.log" Sep 07 10:18:02.731378 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:18:02.731353 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-m6mmt_6aad73e9-87ee-49b0-8651-d3fd8390e4e4/console-operator/1.log" Sep 07 10:18:02.735570 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:18:02.735546 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/ovn-acl-logging/0.log" Sep 07 10:18:02.738656 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:18:02.738636 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/ovn-acl-logging/0.log" Sep 07 10:21:49.612514 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:21:49.612477 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-test-apply-7wc44"] Sep 07 10:21:49.613010 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:21:49.612848 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb" containerName="console" Sep 07 10:21:49.613010 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:21:49.612861 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb" containerName="console" Sep 07 10:21:49.613010 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:21:49.612925 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ce0aabd-47c5-42f1-9aa9-dfd7f17cbffb" containerName="console" Sep 07 10:21:49.615948 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:21:49.615931 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-7wc44" Sep 07 10:21:49.618213 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:21:49.618184 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 07 10:21:49.618354 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:21:49.618324 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 07 10:21:49.619008 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:21:49.618992 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 07 10:21:49.619075 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:21:49.619012 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-cronjob-dockercfg-lz4qf\"" Sep 07 10:21:49.624043 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:21:49.624014 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-7wc44"] Sep 07 10:21:49.724444 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:21:49.724409 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn8dh\" (UniqueName: \"kubernetes.io/projected/309e7ad6-1dde-4a0e-9c27-fc6b5766cd02-kube-api-access-wn8dh\") pod \"feast-test-apply-7wc44\" (UID: \"309e7ad6-1dde-4a0e-9c27-fc6b5766cd02\") " pod="test-ns-feast/feast-test-apply-7wc44" Sep 07 10:21:49.825850 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:21:49.825779 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn8dh\" (UniqueName: \"kubernetes.io/projected/309e7ad6-1dde-4a0e-9c27-fc6b5766cd02-kube-api-access-wn8dh\") pod \"feast-test-apply-7wc44\" (UID: \"309e7ad6-1dde-4a0e-9c27-fc6b5766cd02\") " pod="test-ns-feast/feast-test-apply-7wc44" Sep 07 10:21:49.833398 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:21:49.833360 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn8dh\" (UniqueName: \"kubernetes.io/projected/309e7ad6-1dde-4a0e-9c27-fc6b5766cd02-kube-api-access-wn8dh\") pod \"feast-test-apply-7wc44\" (UID: \"309e7ad6-1dde-4a0e-9c27-fc6b5766cd02\") " pod="test-ns-feast/feast-test-apply-7wc44" Sep 07 10:21:49.925700 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:21:49.925603 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-7wc44" Sep 07 10:21:50.193851 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:21:50.193593 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-7wc44"] Sep 07 10:21:50.196459 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:21:50.196422 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod309e7ad6_1dde_4a0e_9c27_fc6b5766cd02.slice/crio-5ba1e39c80d3b4e9ccd8f0a7b2d403dc5c2477fc9fb6e067733d4e9613de497f WatchSource:0}: Error finding container 5ba1e39c80d3b4e9ccd8f0a7b2d403dc5c2477fc9fb6e067733d4e9613de497f: Status 404 returned error can't find the container with id 5ba1e39c80d3b4e9ccd8f0a7b2d403dc5c2477fc9fb6e067733d4e9613de497f Sep 07 10:21:50.198194 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:21:50.198174 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 10:21:51.087418 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:21:51.087334 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-7wc44" event={"ID":"309e7ad6-1dde-4a0e-9c27-fc6b5766cd02","Type":"ContainerStarted","Data":"5ba1e39c80d3b4e9ccd8f0a7b2d403dc5c2477fc9fb6e067733d4e9613de497f"} Sep 07 10:21:55.105018 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:21:55.104981 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-7wc44" event={"ID":"309e7ad6-1dde-4a0e-9c27-fc6b5766cd02","Type":"ContainerStarted","Data":"754334426a63a0c4187f4e3d851c58aa32a0888fe52e5e584174f7cce3b212fc"} Sep 07 10:22:02.133483 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:22:02.133443 2575 generic.go:356] "Generic (PLEG): container finished" podID="309e7ad6-1dde-4a0e-9c27-fc6b5766cd02" containerID="754334426a63a0c4187f4e3d851c58aa32a0888fe52e5e584174f7cce3b212fc" exitCode=0 Sep 07 10:22:02.133899 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:22:02.133492 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-7wc44" event={"ID":"309e7ad6-1dde-4a0e-9c27-fc6b5766cd02","Type":"ContainerDied","Data":"754334426a63a0c4187f4e3d851c58aa32a0888fe52e5e584174f7cce3b212fc"} Sep 07 10:22:03.139147 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:22:03.139108 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-7wc44" event={"ID":"309e7ad6-1dde-4a0e-9c27-fc6b5766cd02","Type":"ContainerStarted","Data":"3d062dcc1dbe75da9f0a85eda0f4cd75e453f4c871e5575d404b34873493fe96"} Sep 07 10:22:03.154523 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:22:03.154470 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-test-apply-7wc44" podStartSLOduration=2.843594602 podStartE2EDuration="14.154452122s" totalImagesPullingTime="4.550646136s" totalInitContainerRuntime="6.760211384s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:21:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:22:03.153149375 +0000 UTC m=+840.865391320" watchObservedRunningTime="2026-09-07 10:22:03.154452122 +0000 UTC m=+840.866694068" Sep 07 10:22:12.177091 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:22:12.177059 2575 generic.go:356] "Generic (PLEG): container finished" podID="309e7ad6-1dde-4a0e-9c27-fc6b5766cd02" containerID="3d062dcc1dbe75da9f0a85eda0f4cd75e453f4c871e5575d404b34873493fe96" exitCode=0 Sep 07 10:22:12.177521 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:22:12.177140 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-7wc44" event={"ID":"309e7ad6-1dde-4a0e-9c27-fc6b5766cd02","Type":"ContainerDied","Data":"3d062dcc1dbe75da9f0a85eda0f4cd75e453f4c871e5575d404b34873493fe96"} Sep 07 10:22:13.318069 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:22:13.318042 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-7wc44" Sep 07 10:22:13.417663 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:22:13.417624 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wn8dh\" (UniqueName: \"kubernetes.io/projected/309e7ad6-1dde-4a0e-9c27-fc6b5766cd02-kube-api-access-wn8dh\") pod \"309e7ad6-1dde-4a0e-9c27-fc6b5766cd02\" (UID: \"309e7ad6-1dde-4a0e-9c27-fc6b5766cd02\") " Sep 07 10:22:13.419930 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:22:13.419900 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/309e7ad6-1dde-4a0e-9c27-fc6b5766cd02-kube-api-access-wn8dh" (OuterVolumeSpecName: "kube-api-access-wn8dh") pod "309e7ad6-1dde-4a0e-9c27-fc6b5766cd02" (UID: "309e7ad6-1dde-4a0e-9c27-fc6b5766cd02"). InnerVolumeSpecName "kube-api-access-wn8dh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:22:13.518562 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:22:13.518459 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wn8dh\" (UniqueName: \"kubernetes.io/projected/309e7ad6-1dde-4a0e-9c27-fc6b5766cd02-kube-api-access-wn8dh\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:22:14.189115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:22:14.189090 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-7wc44" Sep 07 10:22:14.189289 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:22:14.189118 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-7wc44" event={"ID":"309e7ad6-1dde-4a0e-9c27-fc6b5766cd02","Type":"ContainerDied","Data":"5ba1e39c80d3b4e9ccd8f0a7b2d403dc5c2477fc9fb6e067733d4e9613de497f"} Sep 07 10:22:14.189289 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:22:14.189154 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ba1e39c80d3b4e9ccd8f0a7b2d403dc5c2477fc9fb6e067733d4e9613de497f" Sep 07 10:22:14.365188 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:22:14.365146 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-7wc44_309e7ad6-1dde-4a0e-9c27-fc6b5766cd02/feast-1/0.log" Sep 07 10:22:14.370698 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:22:14.370678 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-7wc44_309e7ad6-1dde-4a0e-9c27-fc6b5766cd02/feast-0/0.log" Sep 07 10:22:52.479947 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:22:52.479907 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-test-apply-7wc44"] Sep 07 10:22:52.487582 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:22:52.487554 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-test-apply-7wc44"] Sep 07 10:22:52.830738 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:22:52.830705 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="309e7ad6-1dde-4a0e-9c27-fc6b5766cd02" path="/var/lib/kubelet/pods/309e7ad6-1dde-4a0e-9c27-fc6b5766cd02/volumes" Sep 07 10:23:02.753438 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:23:02.753398 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-m6mmt_6aad73e9-87ee-49b0-8651-d3fd8390e4e4/console-operator/1.log" Sep 07 10:23:02.754415 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:23:02.754390 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-m6mmt_6aad73e9-87ee-49b0-8651-d3fd8390e4e4/console-operator/1.log" Sep 07 10:23:02.760296 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:23:02.760273 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/ovn-acl-logging/0.log" Sep 07 10:23:02.761175 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:23:02.761152 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/ovn-acl-logging/0.log" Sep 07 10:25:30.930644 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:25:30.930610 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-fhzgm"] Sep 07 10:25:30.931076 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:25:30.931058 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="309e7ad6-1dde-4a0e-9c27-fc6b5766cd02" containerName="feast-0" Sep 07 10:25:30.931121 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:25:30.931081 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="309e7ad6-1dde-4a0e-9c27-fc6b5766cd02" containerName="feast-0" Sep 07 10:25:30.931121 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:25:30.931117 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="309e7ad6-1dde-4a0e-9c27-fc6b5766cd02" containerName="feast-1" Sep 07 10:25:30.931182 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:25:30.931125 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="309e7ad6-1dde-4a0e-9c27-fc6b5766cd02" containerName="feast-1" Sep 07 10:25:30.931213 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:25:30.931207 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="309e7ad6-1dde-4a0e-9c27-fc6b5766cd02" containerName="feast-1" Sep 07 10:25:30.934264 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:25:30.934241 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-fhzgm" Sep 07 10:25:30.936500 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:25:30.936467 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Sep 07 10:25:30.936500 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:25:30.936483 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-lqvlz\"" Sep 07 10:25:30.937387 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:25:30.937362 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Sep 07 10:25:30.945524 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:25:30.945501 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-fhzgm"] Sep 07 10:25:31.084099 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:25:31.084040 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76gmz\" (UniqueName: \"kubernetes.io/projected/3f9b4a8e-d2de-4f55-8737-66aa6cae47d0-kube-api-access-76gmz\") pod \"redis-69bbd798b6-fhzgm\" (UID: \"3f9b4a8e-d2de-4f55-8737-66aa6cae47d0\") " pod="test-ns-feast-rest/redis-69bbd798b6-fhzgm" Sep 07 10:25:31.185274 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:25:31.185174 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76gmz\" (UniqueName: \"kubernetes.io/projected/3f9b4a8e-d2de-4f55-8737-66aa6cae47d0-kube-api-access-76gmz\") pod \"redis-69bbd798b6-fhzgm\" (UID: \"3f9b4a8e-d2de-4f55-8737-66aa6cae47d0\") " pod="test-ns-feast-rest/redis-69bbd798b6-fhzgm" Sep 07 10:25:31.193430 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:25:31.193400 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76gmz\" (UniqueName: \"kubernetes.io/projected/3f9b4a8e-d2de-4f55-8737-66aa6cae47d0-kube-api-access-76gmz\") pod \"redis-69bbd798b6-fhzgm\" (UID: \"3f9b4a8e-d2de-4f55-8737-66aa6cae47d0\") " pod="test-ns-feast-rest/redis-69bbd798b6-fhzgm" Sep 07 10:25:31.244192 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:25:31.244153 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-fhzgm" Sep 07 10:25:31.510361 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:25:31.510297 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-fhzgm"] Sep 07 10:25:31.513225 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:25:31.513197 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f9b4a8e_d2de_4f55_8737_66aa6cae47d0.slice/crio-5748687053f1041d12db22b9458002ccfef00799c0fd39978e58adfa67433de0 WatchSource:0}: Error finding container 5748687053f1041d12db22b9458002ccfef00799c0fd39978e58adfa67433de0: Status 404 returned error can't find the container with id 5748687053f1041d12db22b9458002ccfef00799c0fd39978e58adfa67433de0 Sep 07 10:25:31.949515 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:25:31.949481 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-fhzgm" event={"ID":"3f9b4a8e-d2de-4f55-8737-66aa6cae47d0","Type":"ContainerStarted","Data":"5748687053f1041d12db22b9458002ccfef00799c0fd39978e58adfa67433de0"} Sep 07 10:25:32.955769 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:25:32.955732 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-fhzgm" event={"ID":"3f9b4a8e-d2de-4f55-8737-66aa6cae47d0","Type":"ContainerStarted","Data":"ade91dd421f080ca2abeb9ddd8f20f8e399762c031097bd8c03c051176201903"} Sep 07 10:25:32.968408 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:25:32.968352 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-fhzgm" podStartSLOduration=1.610746359 podStartE2EDuration="2.968336967s" totalImagesPullingTime="1.357590608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:25:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:25:32.968036132 +0000 UTC m=+1050.680278077" watchObservedRunningTime="2026-09-07 10:25:32.968336967 +0000 UTC m=+1050.680578912" Sep 07 10:27:59.355779 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:27:59.355739 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-fhzgm"] Sep 07 10:27:59.356299 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:27:59.355946 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-fhzgm" podUID="3f9b4a8e-d2de-4f55-8737-66aa6cae47d0" containerName="redis" containerID="cri-o://ade91dd421f080ca2abeb9ddd8f20f8e399762c031097bd8c03c051176201903" gracePeriod=30 Sep 07 10:27:59.515851 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:27:59.515809 2575 generic.go:356] "Generic (PLEG): container finished" podID="3f9b4a8e-d2de-4f55-8737-66aa6cae47d0" containerID="ade91dd421f080ca2abeb9ddd8f20f8e399762c031097bd8c03c051176201903" exitCode=0 Sep 07 10:27:59.515851 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:27:59.515835 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-fhzgm" event={"ID":"3f9b4a8e-d2de-4f55-8737-66aa6cae47d0","Type":"ContainerDied","Data":"ade91dd421f080ca2abeb9ddd8f20f8e399762c031097bd8c03c051176201903"} Sep 07 10:27:59.604325 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:27:59.604246 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-fhzgm" Sep 07 10:27:59.664107 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:27:59.664024 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76gmz\" (UniqueName: \"kubernetes.io/projected/3f9b4a8e-d2de-4f55-8737-66aa6cae47d0-kube-api-access-76gmz\") pod \"3f9b4a8e-d2de-4f55-8737-66aa6cae47d0\" (UID: \"3f9b4a8e-d2de-4f55-8737-66aa6cae47d0\") " Sep 07 10:27:59.666126 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:27:59.666095 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f9b4a8e-d2de-4f55-8737-66aa6cae47d0-kube-api-access-76gmz" (OuterVolumeSpecName: "kube-api-access-76gmz") pod "3f9b4a8e-d2de-4f55-8737-66aa6cae47d0" (UID: "3f9b4a8e-d2de-4f55-8737-66aa6cae47d0"). InnerVolumeSpecName "kube-api-access-76gmz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:27:59.765275 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:27:59.765236 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76gmz\" (UniqueName: \"kubernetes.io/projected/3f9b4a8e-d2de-4f55-8737-66aa6cae47d0-kube-api-access-76gmz\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:28:00.521620 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:00.521583 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-fhzgm" event={"ID":"3f9b4a8e-d2de-4f55-8737-66aa6cae47d0","Type":"ContainerDied","Data":"5748687053f1041d12db22b9458002ccfef00799c0fd39978e58adfa67433de0"} Sep 07 10:28:00.521620 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:00.521617 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-fhzgm" Sep 07 10:28:00.522165 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:00.521636 2575 scope.go:117] "RemoveContainer" containerID="ade91dd421f080ca2abeb9ddd8f20f8e399762c031097bd8c03c051176201903" Sep 07 10:28:00.540415 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:00.540386 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-fhzgm"] Sep 07 10:28:00.544175 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:00.544147 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-fhzgm"] Sep 07 10:28:00.831471 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:00.831428 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f9b4a8e-d2de-4f55-8737-66aa6cae47d0" path="/var/lib/kubelet/pods/3f9b4a8e-d2de-4f55-8737-66aa6cae47d0/volumes" Sep 07 10:28:02.779003 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:02.778973 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-m6mmt_6aad73e9-87ee-49b0-8651-d3fd8390e4e4/console-operator/1.log" Sep 07 10:28:02.779533 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:02.779514 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-m6mmt_6aad73e9-87ee-49b0-8651-d3fd8390e4e4/console-operator/1.log" Sep 07 10:28:02.786548 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:02.786525 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/ovn-acl-logging/0.log" Sep 07 10:28:02.786787 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:02.786761 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/ovn-acl-logging/0.log" Sep 07 10:28:02.816785 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:02.816750 2575 scope.go:117] "RemoveContainer" containerID="3d062dcc1dbe75da9f0a85eda0f4cd75e453f4c871e5575d404b34873493fe96" Sep 07 10:28:02.824654 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:02.824628 2575 scope.go:117] "RemoveContainer" containerID="754334426a63a0c4187f4e3d851c58aa32a0888fe52e5e584174f7cce3b212fc" Sep 07 10:28:07.731109 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:07.731070 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-n8hzm"] Sep 07 10:28:07.731635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:07.731496 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f9b4a8e-d2de-4f55-8737-66aa6cae47d0" containerName="redis" Sep 07 10:28:07.731635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:07.731515 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f9b4a8e-d2de-4f55-8737-66aa6cae47d0" containerName="redis" Sep 07 10:28:07.731635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:07.731611 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f9b4a8e-d2de-4f55-8737-66aa6cae47d0" containerName="redis" Sep 07 10:28:07.735984 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:07.735961 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-n8hzm" Sep 07 10:28:07.739057 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:07.739035 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 07 10:28:07.739751 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:07.739733 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 07 10:28:07.739868 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:07.739753 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-wdvhx\"" Sep 07 10:28:07.751298 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:07.751269 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-n8hzm"] Sep 07 10:28:07.826085 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:07.826039 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k567\" (UniqueName: \"kubernetes.io/projected/c57c18bd-172d-420c-84d5-004cb82b2db1-kube-api-access-7k567\") pod \"feast-operator-controller-manager-689b877b4b-n8hzm\" (UID: \"c57c18bd-172d-420c-84d5-004cb82b2db1\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-n8hzm" Sep 07 10:28:07.926517 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:07.926475 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k567\" (UniqueName: \"kubernetes.io/projected/c57c18bd-172d-420c-84d5-004cb82b2db1-kube-api-access-7k567\") pod \"feast-operator-controller-manager-689b877b4b-n8hzm\" (UID: \"c57c18bd-172d-420c-84d5-004cb82b2db1\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-n8hzm" Sep 07 10:28:07.935764 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:07.935733 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k567\" (UniqueName: \"kubernetes.io/projected/c57c18bd-172d-420c-84d5-004cb82b2db1-kube-api-access-7k567\") pod \"feast-operator-controller-manager-689b877b4b-n8hzm\" (UID: \"c57c18bd-172d-420c-84d5-004cb82b2db1\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-n8hzm" Sep 07 10:28:08.045958 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:08.045870 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-n8hzm" Sep 07 10:28:08.316840 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:08.316635 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-n8hzm"] Sep 07 10:28:08.319427 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:28:08.319398 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc57c18bd_172d_420c_84d5_004cb82b2db1.slice/crio-401f980dda958f79ef8e98a572217e0170229974c8979f3ffb84e4ff84f7bd3d WatchSource:0}: Error finding container 401f980dda958f79ef8e98a572217e0170229974c8979f3ffb84e4ff84f7bd3d: Status 404 returned error can't find the container with id 401f980dda958f79ef8e98a572217e0170229974c8979f3ffb84e4ff84f7bd3d Sep 07 10:28:08.321294 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:08.321275 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 10:28:08.553038 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:08.553005 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-n8hzm" event={"ID":"c57c18bd-172d-420c-84d5-004cb82b2db1","Type":"ContainerStarted","Data":"401f980dda958f79ef8e98a572217e0170229974c8979f3ffb84e4ff84f7bd3d"} Sep 07 10:28:10.564591 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:10.564553 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-n8hzm" event={"ID":"c57c18bd-172d-420c-84d5-004cb82b2db1","Type":"ContainerStarted","Data":"c864bc966dbd172cea3e241b34263abe09162951a2c0a7b83b4d442d565d9fad"} Sep 07 10:28:10.565069 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:10.564690 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-n8hzm" Sep 07 10:28:10.582040 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:10.581991 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-n8hzm" podStartSLOduration=1.8484077380000001 podStartE2EDuration="3.581975067s" totalImagesPullingTime="1.733567329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:28:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:28:10.580490978 +0000 UTC m=+1208.292732923" watchObservedRunningTime="2026-09-07 10:28:10.581975067 +0000 UTC m=+1208.294217013" Sep 07 10:28:21.571414 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:21.571382 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-n8hzm" Sep 07 10:28:22.996623 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:22.996586 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j"] Sep 07 10:28:23.006707 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.006677 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:23.009260 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.009231 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 07 10:28:23.009480 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.009448 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-g6d6d\"" Sep 07 10:28:23.010085 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.010066 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 07 10:28:23.010221 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.010166 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 07 10:28:23.010221 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.010176 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 07 10:28:23.010421 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.010133 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 07 10:28:23.010421 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.010067 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 07 10:28:23.011292 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.011272 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j"] Sep 07 10:28:23.139690 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.139653 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2b7251f6-99ae-4597-a8e7-5accb9db4efa-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-j494j\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:23.139895 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.139706 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j494j\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:23.139895 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.139778 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j494j\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:23.139895 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.139829 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j494j\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:23.139895 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.139864 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j494j\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:23.139895 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.139884 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgzj5\" (UniqueName: \"kubernetes.io/projected/2b7251f6-99ae-4597-a8e7-5accb9db4efa-kube-api-access-tgzj5\") pod \"feast-simple-feast-setup-57576b8fcf-j494j\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:23.240625 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.240589 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2b7251f6-99ae-4597-a8e7-5accb9db4efa-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-j494j\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:23.240832 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.240645 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j494j\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:23.240832 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.240683 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j494j\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:23.240832 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.240706 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j494j\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:23.240832 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.240744 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j494j\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:23.240832 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.240775 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgzj5\" (UniqueName: \"kubernetes.io/projected/2b7251f6-99ae-4597-a8e7-5accb9db4efa-kube-api-access-tgzj5\") pod \"feast-simple-feast-setup-57576b8fcf-j494j\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:23.241093 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.241018 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2b7251f6-99ae-4597-a8e7-5accb9db4efa-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-j494j\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:23.243363 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.243329 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j494j\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:23.243491 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.243452 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j494j\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:23.243541 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.243514 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j494j\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:23.243581 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.243539 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j494j\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:23.248717 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.248659 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgzj5\" (UniqueName: \"kubernetes.io/projected/2b7251f6-99ae-4597-a8e7-5accb9db4efa-kube-api-access-tgzj5\") pod \"feast-simple-feast-setup-57576b8fcf-j494j\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:23.318435 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.318394 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:23.594269 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.594244 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j"] Sep 07 10:28:23.596291 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:28:23.596261 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b7251f6_99ae_4597_a8e7_5accb9db4efa.slice/crio-d4c59fde41a8d8e9c62990b9a8f4eda61fbbdff5407126b133bcbb51c00c97d5 WatchSource:0}: Error finding container d4c59fde41a8d8e9c62990b9a8f4eda61fbbdff5407126b133bcbb51c00c97d5: Status 404 returned error can't find the container with id d4c59fde41a8d8e9c62990b9a8f4eda61fbbdff5407126b133bcbb51c00c97d5 Sep 07 10:28:23.619275 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:23.619245 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" event={"ID":"2b7251f6-99ae-4597-a8e7-5accb9db4efa","Type":"ContainerStarted","Data":"d4c59fde41a8d8e9c62990b9a8f4eda61fbbdff5407126b133bcbb51c00c97d5"} Sep 07 10:28:36.686555 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:36.686509 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" event={"ID":"2b7251f6-99ae-4597-a8e7-5accb9db4efa","Type":"ContainerStarted","Data":"5d87c96084f14aea8991e11b411b17303922fe290f1b2235a39afc2eea8cd3fa"} Sep 07 10:28:41.708538 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:41.708451 2575 generic.go:356] "Generic (PLEG): container finished" podID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerID="5d87c96084f14aea8991e11b411b17303922fe290f1b2235a39afc2eea8cd3fa" exitCode=0 Sep 07 10:28:41.708538 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:41.708523 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" event={"ID":"2b7251f6-99ae-4597-a8e7-5accb9db4efa","Type":"ContainerDied","Data":"5d87c96084f14aea8991e11b411b17303922fe290f1b2235a39afc2eea8cd3fa"} Sep 07 10:28:42.714021 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:42.713985 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" event={"ID":"2b7251f6-99ae-4597-a8e7-5accb9db4efa","Type":"ContainerStarted","Data":"44ae4e2064a97aa6581eb8c2093a9a5b74afe4eaeed3f2dd27afe517caebef4a"} Sep 07 10:28:48.740074 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:48.740038 2575 generic.go:356] "Generic (PLEG): container finished" podID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerID="44ae4e2064a97aa6581eb8c2093a9a5b74afe4eaeed3f2dd27afe517caebef4a" exitCode=0 Sep 07 10:28:48.740481 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:48.740118 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" event={"ID":"2b7251f6-99ae-4597-a8e7-5accb9db4efa","Type":"ContainerDied","Data":"44ae4e2064a97aa6581eb8c2093a9a5b74afe4eaeed3f2dd27afe517caebef4a"} Sep 07 10:28:49.750593 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:49.750552 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" event={"ID":"2b7251f6-99ae-4597-a8e7-5accb9db4efa","Type":"ContainerStarted","Data":"6bfa77e4b2d0d99be61aef77ffed9dcb6a74520c79b534e4f62e58f092d36584"} Sep 07 10:28:49.751100 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:49.750613 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" event={"ID":"2b7251f6-99ae-4597-a8e7-5accb9db4efa","Type":"ContainerStarted","Data":"7133b7daff4a4f88f7882831ac080dc608db97b03a01e69e7f63c1e5e1c9fd2d"} Sep 07 10:28:49.751100 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:49.750631 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" event={"ID":"2b7251f6-99ae-4597-a8e7-5accb9db4efa","Type":"ContainerStarted","Data":"fa92c668a6ab864d238e31da326574c78e5debbbc123401cfa713b70bdb02f1a"} Sep 07 10:28:49.751100 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:49.750648 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" event={"ID":"2b7251f6-99ae-4597-a8e7-5accb9db4efa","Type":"ContainerStarted","Data":"eda9597fa3a4e9d7058b632b090970f37829bfbe5ba0f7cb8d4befb13c3e5313"} Sep 07 10:28:49.773630 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:49.773565 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podStartSLOduration=15.059763588 podStartE2EDuration="27.773543032s" totalImagesPullingTime="12.713779444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:28:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:28:49.771320723 +0000 UTC m=+1247.483562716" watchObservedRunningTime="2026-09-07 10:28:49.773543032 +0000 UTC m=+1247.485784978" Sep 07 10:28:50.319150 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:50.318995 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:50.319150 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:50.319060 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:50.319150 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:50.319082 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:50.319491 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:50.319207 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:50.321149 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:50.320983 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused" Sep 07 10:28:50.321149 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:50.321006 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.25:6571: connect: connection refused" Sep 07 10:28:50.321149 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:50.321036 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.25:8816: connect: connection refused" Sep 07 10:28:50.321149 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:50.320984 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused" Sep 07 10:28:53.319561 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:53.319313 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:53.319561 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:53.319391 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:53.319561 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:53.319414 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:53.320154 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:53.319650 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.25:6571: connect: connection refused" Sep 07 10:28:53.320154 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:53.319665 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused" Sep 07 10:28:53.320261 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:53.320183 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused" Sep 07 10:28:53.320306 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:53.320276 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:53.320512 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:53.320491 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.25:8816: connect: connection refused" Sep 07 10:28:56.319392 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:56.319294 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.25:8816: connect: connection refused" Sep 07 10:28:56.319970 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:56.319306 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused" Sep 07 10:28:56.320090 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:56.319321 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.25:6571: connect: connection refused" Sep 07 10:28:56.320155 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:56.319775 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused" Sep 07 10:28:59.320540 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:59.320508 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:59.321026 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:59.320629 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:59.321026 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:59.320858 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:59.321391 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:59.321366 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:59.321507 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:59.321474 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:59.321680 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:59.321667 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:59.325960 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:59.325943 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:28:59.809929 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:28:59.809895 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:30:02.741931 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:02.741896 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j"] Sep 07 10:30:02.743874 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:02.742171 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="offline" containerID="cri-o://7133b7daff4a4f88f7882831ac080dc608db97b03a01e69e7f63c1e5e1c9fd2d" gracePeriod=30 Sep 07 10:30:02.743874 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:02.742201 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="online" containerID="cri-o://fa92c668a6ab864d238e31da326574c78e5debbbc123401cfa713b70bdb02f1a" gracePeriod=30 Sep 07 10:30:02.743874 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:02.742195 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="ui" containerID="cri-o://6bfa77e4b2d0d99be61aef77ffed9dcb6a74520c79b534e4f62e58f092d36584" gracePeriod=30 Sep 07 10:30:02.743874 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:02.742176 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="registry" containerID="cri-o://eda9597fa3a4e9d7058b632b090970f37829bfbe5ba0f7cb8d4befb13c3e5313" gracePeriod=30 Sep 07 10:30:03.169987 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.169937 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7"] Sep 07 10:30:03.174025 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.174003 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:03.176528 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.176498 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-7xsln\"" Sep 07 10:30:03.186443 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.186409 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7"] Sep 07 10:30:03.298190 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.298134 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7b2d\" (UniqueName: \"kubernetes.io/projected/720b352f-8260-41bf-85c4-6b62516a8ee9-kube-api-access-w7b2d\") pod \"feast-simple-feast-setup-57576b8fcf-l8hb7\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:03.298407 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.298215 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/720b352f-8260-41bf-85c4-6b62516a8ee9-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-l8hb7\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:03.298407 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.298248 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l8hb7\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:03.298407 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.298286 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l8hb7\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:03.298407 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.298306 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l8hb7\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:03.298614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.298406 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l8hb7\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:03.399605 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.399555 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l8hb7\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:03.399906 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.399632 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l8hb7\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:03.399906 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.399658 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l8hb7\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:03.399906 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.399737 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l8hb7\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:03.399906 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.399767 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7b2d\" (UniqueName: \"kubernetes.io/projected/720b352f-8260-41bf-85c4-6b62516a8ee9-kube-api-access-w7b2d\") pod \"feast-simple-feast-setup-57576b8fcf-l8hb7\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:03.399906 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:30:03.399834 2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 07 10:30:03.399906 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.399879 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/720b352f-8260-41bf-85c4-6b62516a8ee9-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-l8hb7\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:03.400227 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:30:03.399946 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-ui-tls podName:720b352f-8260-41bf-85c4-6b62516a8ee9 nodeName:}" failed. No retries permitted until 2026-09-07 10:30:03.899921144 +0000 UTC m=+1321.612163075 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-ui-tls") pod "feast-simple-feast-setup-57576b8fcf-l8hb7" (UID: "720b352f-8260-41bf-85c4-6b62516a8ee9") : secret "feast-simple-feast-setup-ui-tls" not found Sep 07 10:30:03.400324 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.400289 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/720b352f-8260-41bf-85c4-6b62516a8ee9-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-l8hb7\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:03.403001 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.402964 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l8hb7\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:03.403230 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.403213 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l8hb7\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:03.403437 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.403420 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l8hb7\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:03.408593 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.408557 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7b2d\" (UniqueName: \"kubernetes.io/projected/720b352f-8260-41bf-85c4-6b62516a8ee9-kube-api-access-w7b2d\") pod \"feast-simple-feast-setup-57576b8fcf-l8hb7\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:03.903697 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.903656 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l8hb7\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:03.906093 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:03.906072 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l8hb7\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:04.059514 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:04.059481 2575 generic.go:356] "Generic (PLEG): container finished" podID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerID="6bfa77e4b2d0d99be61aef77ffed9dcb6a74520c79b534e4f62e58f092d36584" exitCode=0 Sep 07 10:30:04.059514 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:04.059510 2575 generic.go:356] "Generic (PLEG): container finished" podID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerID="fa92c668a6ab864d238e31da326574c78e5debbbc123401cfa713b70bdb02f1a" exitCode=0 Sep 07 10:30:04.059710 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:04.059547 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" event={"ID":"2b7251f6-99ae-4597-a8e7-5accb9db4efa","Type":"ContainerDied","Data":"6bfa77e4b2d0d99be61aef77ffed9dcb6a74520c79b534e4f62e58f092d36584"} Sep 07 10:30:04.059710 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:04.059590 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" event={"ID":"2b7251f6-99ae-4597-a8e7-5accb9db4efa","Type":"ContainerDied","Data":"fa92c668a6ab864d238e31da326574c78e5debbbc123401cfa713b70bdb02f1a"} Sep 07 10:30:04.084463 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:04.084423 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:04.363744 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:04.363716 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7"] Sep 07 10:30:04.365497 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:30:04.365471 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod720b352f_8260_41bf_85c4_6b62516a8ee9.slice/crio-e53f8cbfade88cb84413a10feb80763c7dc0614c774805081639102f31b5b84b WatchSource:0}: Error finding container e53f8cbfade88cb84413a10feb80763c7dc0614c774805081639102f31b5b84b: Status 404 returned error can't find the container with id e53f8cbfade88cb84413a10feb80763c7dc0614c774805081639102f31b5b84b Sep 07 10:30:05.068193 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:05.068152 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" event={"ID":"720b352f-8260-41bf-85c4-6b62516a8ee9","Type":"ContainerStarted","Data":"c99065526ce4b7621c2f60212374cd85baa45b906321026030c45e597b506559"} Sep 07 10:30:05.068565 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:05.068216 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" event={"ID":"720b352f-8260-41bf-85c4-6b62516a8ee9","Type":"ContainerStarted","Data":"e53f8cbfade88cb84413a10feb80763c7dc0614c774805081639102f31b5b84b"} Sep 07 10:30:09.321918 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:09.321857 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused" Sep 07 10:30:09.807079 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:09.806964 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused" Sep 07 10:30:10.089671 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:10.089634 2575 generic.go:356] "Generic (PLEG): container finished" podID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerID="c99065526ce4b7621c2f60212374cd85baa45b906321026030c45e597b506559" exitCode=0 Sep 07 10:30:10.089864 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:10.089706 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" event={"ID":"720b352f-8260-41bf-85c4-6b62516a8ee9","Type":"ContainerDied","Data":"c99065526ce4b7621c2f60212374cd85baa45b906321026030c45e597b506559"} Sep 07 10:30:11.095388 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:11.095350 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" event={"ID":"720b352f-8260-41bf-85c4-6b62516a8ee9","Type":"ContainerStarted","Data":"7f519681d9d575cb2014ffaf6698cec0adebc57d7f18a55f8c065d08cd841f5c"} Sep 07 10:30:17.123011 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:17.122971 2575 generic.go:356] "Generic (PLEG): container finished" podID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerID="7f519681d9d575cb2014ffaf6698cec0adebc57d7f18a55f8c065d08cd841f5c" exitCode=0 Sep 07 10:30:17.123389 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:17.123035 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" event={"ID":"720b352f-8260-41bf-85c4-6b62516a8ee9","Type":"ContainerDied","Data":"7f519681d9d575cb2014ffaf6698cec0adebc57d7f18a55f8c065d08cd841f5c"} Sep 07 10:30:18.134585 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:18.134543 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" event={"ID":"720b352f-8260-41bf-85c4-6b62516a8ee9","Type":"ContainerStarted","Data":"2b630535ed228d584286afe579acf87bf4681727072852557f59e30ad90bd944"} Sep 07 10:30:18.135100 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:18.134610 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" event={"ID":"720b352f-8260-41bf-85c4-6b62516a8ee9","Type":"ContainerStarted","Data":"a68b4b4f162eb04690cb2935e4414545f4cd3868b09b1f53c358d79368e217e0"} Sep 07 10:30:18.135100 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:18.134631 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" event={"ID":"720b352f-8260-41bf-85c4-6b62516a8ee9","Type":"ContainerStarted","Data":"6d253e0ad54575b24b787d9fa1e79b158370b1f8a19a3ab04717941071b81c95"} Sep 07 10:30:18.135100 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:18.134649 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" event={"ID":"720b352f-8260-41bf-85c4-6b62516a8ee9","Type":"ContainerStarted","Data":"02c53ac6dcbd43d9f6f9cd625dc3fbed18207c13a4f2b28f333eb40e83e6f8d5"} Sep 07 10:30:18.179349 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:18.179276 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podStartSLOduration=15.179251625 podStartE2EDuration="15.179251625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:30:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:30:18.17612581 +0000 UTC m=+1335.888367755" watchObservedRunningTime="2026-09-07 10:30:18.179251625 +0000 UTC m=+1335.891493571" Sep 07 10:30:19.084996 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:19.084955 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:19.085198 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:19.085023 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:19.085198 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:19.085044 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:19.085198 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:19.085061 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:19.087120 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:19.087057 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 07 10:30:19.087120 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:19.087096 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.26:8816: connect: connection refused" Sep 07 10:30:19.087316 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:19.087082 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.26:6571: connect: connection refused" Sep 07 10:30:19.087316 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:19.087150 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 07 10:30:19.322076 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:19.321666 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused" Sep 07 10:30:19.807720 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:19.807668 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused" Sep 07 10:30:22.085702 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:22.085651 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 07 10:30:22.086356 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:22.085672 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.26:6571: connect: connection refused" Sep 07 10:30:22.086487 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:22.085672 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 07 10:30:22.086487 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:22.085671 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.26:8816: connect: connection refused" Sep 07 10:30:24.084946 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:24.084900 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:24.085514 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:24.085486 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:24.085514 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:24.085520 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:24.085739 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:24.085537 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:25.085371 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:25.085275 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.26:8816: connect: connection refused" Sep 07 10:30:25.085951 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:25.085283 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.26:6571: connect: connection refused" Sep 07 10:30:25.086152 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:25.085302 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 07 10:30:25.086250 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:25.085308 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 07 10:30:28.086354 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:28.086308 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:28.086842 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:28.086524 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:28.086842 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:28.086655 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:28.087440 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:28.087419 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:28.087564 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:28.087470 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:28.087564 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:28.087533 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:28.092016 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:28.091992 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:28.192697 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:28.192657 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:30:29.321633 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:29.321579 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused" Sep 07 10:30:29.322083 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:29.321731 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:30:29.807252 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:29.807150 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused" Sep 07 10:30:29.807431 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:29.807279 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:30:33.211284 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.211246 2575 generic.go:356] "Generic (PLEG): container finished" podID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerID="7133b7daff4a4f88f7882831ac080dc608db97b03a01e69e7f63c1e5e1c9fd2d" exitCode=137 Sep 07 10:30:33.211284 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.211277 2575 generic.go:356] "Generic (PLEG): container finished" podID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerID="eda9597fa3a4e9d7058b632b090970f37829bfbe5ba0f7cb8d4befb13c3e5313" exitCode=137 Sep 07 10:30:33.211721 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.211317 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" event={"ID":"2b7251f6-99ae-4597-a8e7-5accb9db4efa","Type":"ContainerDied","Data":"7133b7daff4a4f88f7882831ac080dc608db97b03a01e69e7f63c1e5e1c9fd2d"} Sep 07 10:30:33.211721 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.211361 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" event={"ID":"2b7251f6-99ae-4597-a8e7-5accb9db4efa","Type":"ContainerDied","Data":"eda9597fa3a4e9d7058b632b090970f37829bfbe5ba0f7cb8d4befb13c3e5313"} Sep 07 10:30:33.394806 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.394765 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:30:33.570966 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.570934 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-online-tls\") pod \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " Sep 07 10:30:33.571150 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.570987 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2b7251f6-99ae-4597-a8e7-5accb9db4efa-feast-data\") pod \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " Sep 07 10:30:33.571150 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.571009 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-offline-tls\") pod \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " Sep 07 10:30:33.571150 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.571044 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgzj5\" (UniqueName: \"kubernetes.io/projected/2b7251f6-99ae-4597-a8e7-5accb9db4efa-kube-api-access-tgzj5\") pod \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " Sep 07 10:30:33.571150 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.571088 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-registry-tls\") pod \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " Sep 07 10:30:33.571150 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.571112 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-ui-tls\") pod \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\" (UID: \"2b7251f6-99ae-4597-a8e7-5accb9db4efa\") " Sep 07 10:30:33.571610 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.571574 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b7251f6-99ae-4597-a8e7-5accb9db4efa-feast-data" (OuterVolumeSpecName: "feast-data") pod "2b7251f6-99ae-4597-a8e7-5accb9db4efa" (UID: "2b7251f6-99ae-4597-a8e7-5accb9db4efa"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 10:30:33.573412 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.573376 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "2b7251f6-99ae-4597-a8e7-5accb9db4efa" (UID: "2b7251f6-99ae-4597-a8e7-5accb9db4efa"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:30:33.573526 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.573417 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "2b7251f6-99ae-4597-a8e7-5accb9db4efa" (UID: "2b7251f6-99ae-4597-a8e7-5accb9db4efa"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:30:33.573526 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.573432 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2b7251f6-99ae-4597-a8e7-5accb9db4efa" (UID: "2b7251f6-99ae-4597-a8e7-5accb9db4efa"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:30:33.573639 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.573580 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-online-tls" (OuterVolumeSpecName: "online-tls") pod "2b7251f6-99ae-4597-a8e7-5accb9db4efa" (UID: "2b7251f6-99ae-4597-a8e7-5accb9db4efa"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:30:33.573639 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.573591 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b7251f6-99ae-4597-a8e7-5accb9db4efa-kube-api-access-tgzj5" (OuterVolumeSpecName: "kube-api-access-tgzj5") pod "2b7251f6-99ae-4597-a8e7-5accb9db4efa" (UID: "2b7251f6-99ae-4597-a8e7-5accb9db4efa"). InnerVolumeSpecName "kube-api-access-tgzj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:30:33.671980 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.671934 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-online-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:30:33.671980 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.671970 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2b7251f6-99ae-4597-a8e7-5accb9db4efa-feast-data\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:30:33.671980 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.671979 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-offline-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:30:33.671980 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.671987 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgzj5\" (UniqueName: \"kubernetes.io/projected/2b7251f6-99ae-4597-a8e7-5accb9db4efa-kube-api-access-tgzj5\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:30:33.672245 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.671997 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-registry-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:30:33.672245 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:33.672005 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2b7251f6-99ae-4597-a8e7-5accb9db4efa-ui-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:30:34.218650 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:34.218618 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" event={"ID":"2b7251f6-99ae-4597-a8e7-5accb9db4efa","Type":"ContainerDied","Data":"d4c59fde41a8d8e9c62990b9a8f4eda61fbbdff5407126b133bcbb51c00c97d5"} Sep 07 10:30:34.219073 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:34.218673 2575 scope.go:117] "RemoveContainer" containerID="6bfa77e4b2d0d99be61aef77ffed9dcb6a74520c79b534e4f62e58f092d36584" Sep 07 10:30:34.219073 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:34.218629 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j" Sep 07 10:30:34.227822 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:34.227780 2575 scope.go:117] "RemoveContainer" containerID="7133b7daff4a4f88f7882831ac080dc608db97b03a01e69e7f63c1e5e1c9fd2d" Sep 07 10:30:34.235679 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:34.235659 2575 scope.go:117] "RemoveContainer" containerID="fa92c668a6ab864d238e31da326574c78e5debbbc123401cfa713b70bdb02f1a" Sep 07 10:30:34.240952 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:34.240918 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j"] Sep 07 10:30:34.244327 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:34.244309 2575 scope.go:117] "RemoveContainer" containerID="eda9597fa3a4e9d7058b632b090970f37829bfbe5ba0f7cb8d4befb13c3e5313" Sep 07 10:30:34.246646 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:34.246625 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-j494j"] Sep 07 10:30:34.252840 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:34.252779 2575 scope.go:117] "RemoveContainer" containerID="44ae4e2064a97aa6581eb8c2093a9a5b74afe4eaeed3f2dd27afe517caebef4a" Sep 07 10:30:34.260491 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:34.260471 2575 scope.go:117] "RemoveContainer" containerID="5d87c96084f14aea8991e11b411b17303922fe290f1b2235a39afc2eea8cd3fa" Sep 07 10:30:34.835133 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:30:34.835094 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" path="/var/lib/kubelet/pods/2b7251f6-99ae-4597-a8e7-5accb9db4efa/volumes" Sep 07 10:31:30.188751 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.188699 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r"] Sep 07 10:31:30.189307 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.189171 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="offline" Sep 07 10:31:30.189307 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.189190 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="offline" Sep 07 10:31:30.189307 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.189205 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="registry" Sep 07 10:31:30.189307 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.189214 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="registry" Sep 07 10:31:30.189307 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.189227 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="ui" Sep 07 10:31:30.189307 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.189236 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="ui" Sep 07 10:31:30.189307 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.189253 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="online" Sep 07 10:31:30.189307 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.189262 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="online" Sep 07 10:31:30.189307 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.189278 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="feast-apply" Sep 07 10:31:30.189307 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.189287 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="feast-apply" Sep 07 10:31:30.189307 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.189309 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="feast-init" Sep 07 10:31:30.189882 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.189318 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="feast-init" Sep 07 10:31:30.189882 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.189394 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="ui" Sep 07 10:31:30.189882 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.189406 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="registry" Sep 07 10:31:30.189882 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.189416 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="offline" Sep 07 10:31:30.189882 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.189428 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b7251f6-99ae-4597-a8e7-5accb9db4efa" containerName="online" Sep 07 10:31:30.194326 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.194305 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:31:30.196902 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.196854 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 07 10:31:30.196902 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.196860 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 07 10:31:30.197096 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.196916 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 07 10:31:30.197096 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.196916 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 07 10:31:30.197096 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.196981 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 07 10:31:30.197096 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.196860 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 07 10:31:30.197248 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.197165 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-chxt4\"" Sep 07 10:31:30.203904 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.203880 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r"] Sep 07 10:31:30.344683 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.344647 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/091d8b74-ca8b-43f1-b768-9447d0927edd-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8wc4r\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:31:30.344683 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.344686 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/091d8b74-ca8b-43f1-b768-9447d0927edd-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8wc4r\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:31:30.344998 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.344707 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/091d8b74-ca8b-43f1-b768-9447d0927edd-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8wc4r\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:31:30.344998 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.344739 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/091d8b74-ca8b-43f1-b768-9447d0927edd-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8wc4r\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:31:30.344998 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.344870 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v56z\" (UniqueName: \"kubernetes.io/projected/091d8b74-ca8b-43f1-b768-9447d0927edd-kube-api-access-9v56z\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8wc4r\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:31:30.344998 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.344931 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/091d8b74-ca8b-43f1-b768-9447d0927edd-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8wc4r\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:31:30.445663 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.445571 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v56z\" (UniqueName: \"kubernetes.io/projected/091d8b74-ca8b-43f1-b768-9447d0927edd-kube-api-access-9v56z\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8wc4r\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:31:30.445663 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.445616 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/091d8b74-ca8b-43f1-b768-9447d0927edd-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8wc4r\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:31:30.445937 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.445671 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/091d8b74-ca8b-43f1-b768-9447d0927edd-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8wc4r\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:31:30.445937 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.445696 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/091d8b74-ca8b-43f1-b768-9447d0927edd-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8wc4r\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:31:30.445937 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.445727 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/091d8b74-ca8b-43f1-b768-9447d0927edd-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8wc4r\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:31:30.445937 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.445758 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/091d8b74-ca8b-43f1-b768-9447d0927edd-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8wc4r\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:31:30.446284 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.446254 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/091d8b74-ca8b-43f1-b768-9447d0927edd-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8wc4r\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:31:30.446462 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.446438 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/091d8b74-ca8b-43f1-b768-9447d0927edd-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8wc4r\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:31:30.448357 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.448332 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/091d8b74-ca8b-43f1-b768-9447d0927edd-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8wc4r\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:31:30.448497 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.448471 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/091d8b74-ca8b-43f1-b768-9447d0927edd-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8wc4r\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:31:30.448737 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.448714 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/091d8b74-ca8b-43f1-b768-9447d0927edd-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8wc4r\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:31:30.453961 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.453937 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v56z\" (UniqueName: \"kubernetes.io/projected/091d8b74-ca8b-43f1-b768-9447d0927edd-kube-api-access-9v56z\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8wc4r\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:31:30.504070 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.504034 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:31:30.779735 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:30.779651 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r"] Sep 07 10:31:30.783148 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:31:30.783120 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod091d8b74_ca8b_43f1_b768_9447d0927edd.slice/crio-30db761e4a626e240b9b923e6aad4757ba5bddc2b66056b75e85ec7675fbd776 WatchSource:0}: Error finding container 30db761e4a626e240b9b923e6aad4757ba5bddc2b66056b75e85ec7675fbd776: Status 404 returned error can't find the container with id 30db761e4a626e240b9b923e6aad4757ba5bddc2b66056b75e85ec7675fbd776 Sep 07 10:31:31.454012 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:31.453973 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" event={"ID":"091d8b74-ca8b-43f1-b768-9447d0927edd","Type":"ContainerStarted","Data":"e8a25f1e9a523f26393ce0d34bfb763d1446471ccf47144f3e06a392d08c7b27"} Sep 07 10:31:31.454418 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:31.454026 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" event={"ID":"091d8b74-ca8b-43f1-b768-9447d0927edd","Type":"ContainerStarted","Data":"30db761e4a626e240b9b923e6aad4757ba5bddc2b66056b75e85ec7675fbd776"} Sep 07 10:31:36.477375 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:36.477336 2575 generic.go:356] "Generic (PLEG): container finished" podID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerID="e8a25f1e9a523f26393ce0d34bfb763d1446471ccf47144f3e06a392d08c7b27" exitCode=0 Sep 07 10:31:36.477762 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:36.477394 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" event={"ID":"091d8b74-ca8b-43f1-b768-9447d0927edd","Type":"ContainerDied","Data":"e8a25f1e9a523f26393ce0d34bfb763d1446471ccf47144f3e06a392d08c7b27"} Sep 07 10:31:37.484347 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:37.484308 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" event={"ID":"091d8b74-ca8b-43f1-b768-9447d0927edd","Type":"ContainerStarted","Data":"9825433a7cf455aaff93d551d0b302bd00bec236b661e64404c7b4c85fae23e1"} Sep 07 10:31:58.582329 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:58.582294 2575 generic.go:356] "Generic (PLEG): container finished" podID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerID="9825433a7cf455aaff93d551d0b302bd00bec236b661e64404c7b4c85fae23e1" exitCode=0 Sep 07 10:31:58.582810 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:58.582373 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" event={"ID":"091d8b74-ca8b-43f1-b768-9447d0927edd","Type":"ContainerDied","Data":"9825433a7cf455aaff93d551d0b302bd00bec236b661e64404c7b4c85fae23e1"} Sep 07 10:31:59.591082 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:59.591045 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" event={"ID":"091d8b74-ca8b-43f1-b768-9447d0927edd","Type":"ContainerStarted","Data":"ca9bcfd0b8dc4120d8dc601851c52a0da3f52cd7a4f9335149640bb6b0f811fd"} Sep 07 10:31:59.591464 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:59.591105 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" event={"ID":"091d8b74-ca8b-43f1-b768-9447d0927edd","Type":"ContainerStarted","Data":"133c33b83d483591367ba353724c146089c79e64f9a36723b80e6d893469ebfc"} Sep 07 10:31:59.591464 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:59.591122 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" event={"ID":"091d8b74-ca8b-43f1-b768-9447d0927edd","Type":"ContainerStarted","Data":"48d0c86f81852767f8b1e00d4da77f843449179a98234a4af7e52afafa7601df"} Sep 07 10:31:59.613195 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:31:59.613137 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podStartSLOduration=29.613119719 podStartE2EDuration="29.613119719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:31:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:31:59.611402622 +0000 UTC m=+1437.323644579" watchObservedRunningTime="2026-09-07 10:31:59.613119719 +0000 UTC m=+1437.325361664" Sep 07 10:32:00.505174 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:00.505127 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:32:00.505373 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:00.505199 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:32:00.505373 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:00.505217 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:32:00.505373 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:00.505232 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:32:00.505373 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:00.505245 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:32:00.505373 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:00.505260 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:32:00.507312 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:00.507267 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.27:8816: connect: connection refused" Sep 07 10:32:00.507448 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:00.507317 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused" Sep 07 10:32:00.507448 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:00.507285 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused" Sep 07 10:32:03.504578 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:03.504524 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.27:8816: connect: connection refused" Sep 07 10:32:03.505096 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:03.504612 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused" Sep 07 10:32:03.505096 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:03.504548 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused" Sep 07 10:32:06.504706 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:06.504651 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.27:8816: connect: connection refused" Sep 07 10:32:06.505346 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:06.504708 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused" Sep 07 10:32:06.505346 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:06.504651 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused" Sep 07 10:32:09.506035 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:09.506003 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:32:09.506416 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:09.506124 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:32:09.506865 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:09.506843 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:32:09.506972 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:09.506897 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:32:10.244559 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:10.244525 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:32:11.269542 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:11.269502 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:32:11.507985 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:11.507927 2575 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 07 10:32:12.273946 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:32:12.273901 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 07 10:33:02.806584 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:02.806554 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-m6mmt_6aad73e9-87ee-49b0-8651-d3fd8390e4e4/console-operator/1.log" Sep 07 10:33:02.807439 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:02.807351 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-m6mmt_6aad73e9-87ee-49b0-8651-d3fd8390e4e4/console-operator/1.log" Sep 07 10:33:02.813600 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:02.813577 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/ovn-acl-logging/0.log" Sep 07 10:33:02.814607 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:02.814586 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/ovn-acl-logging/0.log" Sep 07 10:33:11.508962 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:11.508913 2575 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 07 10:33:12.273958 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:12.273913 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 07 10:33:12.669212 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:12.669177 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r"] Sep 07 10:33:12.669635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:12.669444 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="online" containerID="cri-o://48d0c86f81852767f8b1e00d4da77f843449179a98234a4af7e52afafa7601df" gracePeriod=30 Sep 07 10:33:12.669635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:12.669486 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="offline" containerID="cri-o://133c33b83d483591367ba353724c146089c79e64f9a36723b80e6d893469ebfc" gracePeriod=30 Sep 07 10:33:12.669635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:12.669480 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="ui" containerID="cri-o://ca9bcfd0b8dc4120d8dc601851c52a0da3f52cd7a4f9335149640bb6b0f811fd" gracePeriod=30 Sep 07 10:33:12.859281 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:12.859244 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7"] Sep 07 10:33:12.859604 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:12.859570 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="registry" containerID="cri-o://02c53ac6dcbd43d9f6f9cd625dc3fbed18207c13a4f2b28f333eb40e83e6f8d5" gracePeriod=30 Sep 07 10:33:12.859850 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:12.859618 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="ui" containerID="cri-o://2b630535ed228d584286afe579acf87bf4681727072852557f59e30ad90bd944" gracePeriod=30 Sep 07 10:33:12.859949 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:12.859637 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="online" containerID="cri-o://6d253e0ad54575b24b787d9fa1e79b158370b1f8a19a3ab04717941071b81c95" gracePeriod=30 Sep 07 10:33:12.860006 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:12.859631 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="offline" containerID="cri-o://a68b4b4f162eb04690cb2935e4414545f4cd3868b09b1f53c358d79368e217e0" gracePeriod=30 Sep 07 10:33:13.906935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:13.906844 2575 generic.go:356] "Generic (PLEG): container finished" podID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerID="2b630535ed228d584286afe579acf87bf4681727072852557f59e30ad90bd944" exitCode=0 Sep 07 10:33:13.906935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:13.906871 2575 generic.go:356] "Generic (PLEG): container finished" podID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerID="6d253e0ad54575b24b787d9fa1e79b158370b1f8a19a3ab04717941071b81c95" exitCode=0 Sep 07 10:33:13.906935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:13.906913 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" event={"ID":"720b352f-8260-41bf-85c4-6b62516a8ee9","Type":"ContainerDied","Data":"2b630535ed228d584286afe579acf87bf4681727072852557f59e30ad90bd944"} Sep 07 10:33:13.907368 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:13.906958 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" event={"ID":"720b352f-8260-41bf-85c4-6b62516a8ee9","Type":"ContainerDied","Data":"6d253e0ad54575b24b787d9fa1e79b158370b1f8a19a3ab04717941071b81c95"} Sep 07 10:33:13.909460 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:13.909436 2575 generic.go:356] "Generic (PLEG): container finished" podID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerID="ca9bcfd0b8dc4120d8dc601851c52a0da3f52cd7a4f9335149640bb6b0f811fd" exitCode=0 Sep 07 10:33:13.909572 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:13.909484 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" event={"ID":"091d8b74-ca8b-43f1-b768-9447d0927edd","Type":"ContainerDied","Data":"ca9bcfd0b8dc4120d8dc601851c52a0da3f52cd7a4f9335149640bb6b0f811fd"} Sep 07 10:33:18.088092 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:18.088048 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 07 10:33:18.194842 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:18.190909 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 07 10:33:19.506715 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:19.506663 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused" Sep 07 10:33:22.270366 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:22.270324 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Sep 07 10:33:28.087940 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:28.087835 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 07 10:33:28.190088 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:28.190040 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 07 10:33:29.506822 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:29.506756 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused" Sep 07 10:33:32.270227 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:32.270185 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 07 10:33:32.270668 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:32.270313 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:33:38.087531 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:38.087481 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 07 10:33:38.088106 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:38.087658 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:33:38.190352 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:38.190300 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 07 10:33:38.190543 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:38.190463 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:33:39.506790 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:39.506738 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused" Sep 07 10:33:39.507198 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:39.506898 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:33:42.270051 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:42.270003 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": context deadline exceeded" Sep 07 10:33:43.035462 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.035376 2575 generic.go:356] "Generic (PLEG): container finished" podID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerID="a68b4b4f162eb04690cb2935e4414545f4cd3868b09b1f53c358d79368e217e0" exitCode=137 Sep 07 10:33:43.035462 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.035403 2575 generic.go:356] "Generic (PLEG): container finished" podID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerID="02c53ac6dcbd43d9f6f9cd625dc3fbed18207c13a4f2b28f333eb40e83e6f8d5" exitCode=137 Sep 07 10:33:43.035462 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.035444 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" event={"ID":"720b352f-8260-41bf-85c4-6b62516a8ee9","Type":"ContainerDied","Data":"a68b4b4f162eb04690cb2935e4414545f4cd3868b09b1f53c358d79368e217e0"} Sep 07 10:33:43.035700 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.035491 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" event={"ID":"720b352f-8260-41bf-85c4-6b62516a8ee9","Type":"ContainerDied","Data":"02c53ac6dcbd43d9f6f9cd625dc3fbed18207c13a4f2b28f333eb40e83e6f8d5"} Sep 07 10:33:43.038150 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.038124 2575 generic.go:356] "Generic (PLEG): container finished" podID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerID="133c33b83d483591367ba353724c146089c79e64f9a36723b80e6d893469ebfc" exitCode=137 Sep 07 10:33:43.038150 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.038148 2575 generic.go:356] "Generic (PLEG): container finished" podID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerID="48d0c86f81852767f8b1e00d4da77f843449179a98234a4af7e52afafa7601df" exitCode=137 Sep 07 10:33:43.038316 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.038190 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" event={"ID":"091d8b74-ca8b-43f1-b768-9447d0927edd","Type":"ContainerDied","Data":"133c33b83d483591367ba353724c146089c79e64f9a36723b80e6d893469ebfc"} Sep 07 10:33:43.038316 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.038223 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" event={"ID":"091d8b74-ca8b-43f1-b768-9447d0927edd","Type":"ContainerDied","Data":"48d0c86f81852767f8b1e00d4da77f843449179a98234a4af7e52afafa7601df"} Sep 07 10:33:43.315991 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.315967 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:33:43.383658 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.383631 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/091d8b74-ca8b-43f1-b768-9447d0927edd-offline-tls\") pod \"091d8b74-ca8b-43f1-b768-9447d0927edd\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " Sep 07 10:33:43.383791 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.383674 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9v56z\" (UniqueName: \"kubernetes.io/projected/091d8b74-ca8b-43f1-b768-9447d0927edd-kube-api-access-9v56z\") pod \"091d8b74-ca8b-43f1-b768-9447d0927edd\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " Sep 07 10:33:43.383791 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.383694 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/091d8b74-ca8b-43f1-b768-9447d0927edd-registry-tls\") pod \"091d8b74-ca8b-43f1-b768-9447d0927edd\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " Sep 07 10:33:43.383791 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.383725 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/091d8b74-ca8b-43f1-b768-9447d0927edd-feast-data\") pod \"091d8b74-ca8b-43f1-b768-9447d0927edd\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " Sep 07 10:33:43.383791 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.383775 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/091d8b74-ca8b-43f1-b768-9447d0927edd-ui-tls\") pod \"091d8b74-ca8b-43f1-b768-9447d0927edd\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " Sep 07 10:33:43.384017 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.383831 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/091d8b74-ca8b-43f1-b768-9447d0927edd-online-tls\") pod \"091d8b74-ca8b-43f1-b768-9447d0927edd\" (UID: \"091d8b74-ca8b-43f1-b768-9447d0927edd\") " Sep 07 10:33:43.384125 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.384104 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/091d8b74-ca8b-43f1-b768-9447d0927edd-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "091d8b74-ca8b-43f1-b768-9447d0927edd" (UID: "091d8b74-ca8b-43f1-b768-9447d0927edd"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 10:33:43.384312 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.384282 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/091d8b74-ca8b-43f1-b768-9447d0927edd-feast-data" (OuterVolumeSpecName: "feast-data") pod "091d8b74-ca8b-43f1-b768-9447d0927edd" (UID: "091d8b74-ca8b-43f1-b768-9447d0927edd"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 10:33:43.385978 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.385951 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/091d8b74-ca8b-43f1-b768-9447d0927edd-kube-api-access-9v56z" (OuterVolumeSpecName: "kube-api-access-9v56z") pod "091d8b74-ca8b-43f1-b768-9447d0927edd" (UID: "091d8b74-ca8b-43f1-b768-9447d0927edd"). InnerVolumeSpecName "kube-api-access-9v56z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:33:43.386080 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.385987 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/091d8b74-ca8b-43f1-b768-9447d0927edd-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "091d8b74-ca8b-43f1-b768-9447d0927edd" (UID: "091d8b74-ca8b-43f1-b768-9447d0927edd"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:33:43.386211 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.386181 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/091d8b74-ca8b-43f1-b768-9447d0927edd-online-tls" (OuterVolumeSpecName: "online-tls") pod "091d8b74-ca8b-43f1-b768-9447d0927edd" (UID: "091d8b74-ca8b-43f1-b768-9447d0927edd"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:33:43.386297 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.386255 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/091d8b74-ca8b-43f1-b768-9447d0927edd-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "091d8b74-ca8b-43f1-b768-9447d0927edd" (UID: "091d8b74-ca8b-43f1-b768-9447d0927edd"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:33:43.485486 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.485450 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/091d8b74-ca8b-43f1-b768-9447d0927edd-ui-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:33:43.485486 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.485483 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/091d8b74-ca8b-43f1-b768-9447d0927edd-online-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:33:43.485678 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.485499 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/091d8b74-ca8b-43f1-b768-9447d0927edd-offline-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:33:43.485678 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.485509 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9v56z\" (UniqueName: \"kubernetes.io/projected/091d8b74-ca8b-43f1-b768-9447d0927edd-kube-api-access-9v56z\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:33:43.485678 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.485520 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/091d8b74-ca8b-43f1-b768-9447d0927edd-registry-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:33:43.485678 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.485529 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/091d8b74-ca8b-43f1-b768-9447d0927edd-feast-data\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:33:43.511444 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.511413 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:33:43.586442 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.586406 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-online-tls\") pod \"720b352f-8260-41bf-85c4-6b62516a8ee9\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " Sep 07 10:33:43.586614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.586450 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-registry-tls\") pod \"720b352f-8260-41bf-85c4-6b62516a8ee9\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " Sep 07 10:33:43.586614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.586498 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/720b352f-8260-41bf-85c4-6b62516a8ee9-feast-data\") pod \"720b352f-8260-41bf-85c4-6b62516a8ee9\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " Sep 07 10:33:43.586614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.586529 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-offline-tls\") pod \"720b352f-8260-41bf-85c4-6b62516a8ee9\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " Sep 07 10:33:43.586614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.586554 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-ui-tls\") pod \"720b352f-8260-41bf-85c4-6b62516a8ee9\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " Sep 07 10:33:43.586614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.586596 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7b2d\" (UniqueName: \"kubernetes.io/projected/720b352f-8260-41bf-85c4-6b62516a8ee9-kube-api-access-w7b2d\") pod \"720b352f-8260-41bf-85c4-6b62516a8ee9\" (UID: \"720b352f-8260-41bf-85c4-6b62516a8ee9\") " Sep 07 10:33:43.587205 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.587161 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/720b352f-8260-41bf-85c4-6b62516a8ee9-feast-data" (OuterVolumeSpecName: "feast-data") pod "720b352f-8260-41bf-85c4-6b62516a8ee9" (UID: "720b352f-8260-41bf-85c4-6b62516a8ee9"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 10:33:43.588779 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.588742 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/720b352f-8260-41bf-85c4-6b62516a8ee9-kube-api-access-w7b2d" (OuterVolumeSpecName: "kube-api-access-w7b2d") pod "720b352f-8260-41bf-85c4-6b62516a8ee9" (UID: "720b352f-8260-41bf-85c4-6b62516a8ee9"). InnerVolumeSpecName "kube-api-access-w7b2d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:33:43.588779 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.588769 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-online-tls" (OuterVolumeSpecName: "online-tls") pod "720b352f-8260-41bf-85c4-6b62516a8ee9" (UID: "720b352f-8260-41bf-85c4-6b62516a8ee9"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:33:43.589005 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.588806 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "720b352f-8260-41bf-85c4-6b62516a8ee9" (UID: "720b352f-8260-41bf-85c4-6b62516a8ee9"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:33:43.589083 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.589066 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "720b352f-8260-41bf-85c4-6b62516a8ee9" (UID: "720b352f-8260-41bf-85c4-6b62516a8ee9"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:33:43.589120 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.589086 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "720b352f-8260-41bf-85c4-6b62516a8ee9" (UID: "720b352f-8260-41bf-85c4-6b62516a8ee9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:33:43.687594 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.687564 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-online-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:33:43.687594 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.687592 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-registry-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:33:43.687753 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.687600 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/720b352f-8260-41bf-85c4-6b62516a8ee9-feast-data\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:33:43.687753 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.687608 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-offline-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:33:43.687753 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.687616 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/720b352f-8260-41bf-85c4-6b62516a8ee9-ui-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:33:43.687753 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:43.687624 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7b2d\" (UniqueName: \"kubernetes.io/projected/720b352f-8260-41bf-85c4-6b62516a8ee9-kube-api-access-w7b2d\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:33:44.045148 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:44.045067 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" event={"ID":"720b352f-8260-41bf-85c4-6b62516a8ee9","Type":"ContainerDied","Data":"e53f8cbfade88cb84413a10feb80763c7dc0614c774805081639102f31b5b84b"} Sep 07 10:33:44.045148 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:44.045075 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7" Sep 07 10:33:44.045148 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:44.045121 2575 scope.go:117] "RemoveContainer" containerID="2b630535ed228d584286afe579acf87bf4681727072852557f59e30ad90bd944" Sep 07 10:33:44.047923 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:44.047892 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" event={"ID":"091d8b74-ca8b-43f1-b768-9447d0927edd","Type":"ContainerDied","Data":"30db761e4a626e240b9b923e6aad4757ba5bddc2b66056b75e85ec7675fbd776"} Sep 07 10:33:44.048050 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:44.047986 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r" Sep 07 10:33:44.053929 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:44.053909 2575 scope.go:117] "RemoveContainer" containerID="a68b4b4f162eb04690cb2935e4414545f4cd3868b09b1f53c358d79368e217e0" Sep 07 10:33:44.062303 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:44.062282 2575 scope.go:117] "RemoveContainer" containerID="6d253e0ad54575b24b787d9fa1e79b158370b1f8a19a3ab04717941071b81c95" Sep 07 10:33:44.069340 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:44.069318 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7"] Sep 07 10:33:44.070107 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:44.070086 2575 scope.go:117] "RemoveContainer" containerID="02c53ac6dcbd43d9f6f9cd625dc3fbed18207c13a4f2b28f333eb40e83e6f8d5" Sep 07 10:33:44.074345 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:44.074323 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-l8hb7"] Sep 07 10:33:44.078191 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:44.078174 2575 scope.go:117] "RemoveContainer" containerID="7f519681d9d575cb2014ffaf6698cec0adebc57d7f18a55f8c065d08cd841f5c" Sep 07 10:33:44.085700 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:44.085681 2575 scope.go:117] "RemoveContainer" containerID="c99065526ce4b7621c2f60212374cd85baa45b906321026030c45e597b506559" Sep 07 10:33:44.086571 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:44.086547 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r"] Sep 07 10:33:44.094565 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:44.091782 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8wc4r"] Sep 07 10:33:44.099237 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:44.099200 2575 scope.go:117] "RemoveContainer" containerID="ca9bcfd0b8dc4120d8dc601851c52a0da3f52cd7a4f9335149640bb6b0f811fd" Sep 07 10:33:44.106564 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:44.106547 2575 scope.go:117] "RemoveContainer" containerID="133c33b83d483591367ba353724c146089c79e64f9a36723b80e6d893469ebfc" Sep 07 10:33:44.113862 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:44.113836 2575 scope.go:117] "RemoveContainer" containerID="48d0c86f81852767f8b1e00d4da77f843449179a98234a4af7e52afafa7601df" Sep 07 10:33:44.122672 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:44.122650 2575 scope.go:117] "RemoveContainer" containerID="9825433a7cf455aaff93d551d0b302bd00bec236b661e64404c7b4c85fae23e1" Sep 07 10:33:44.130024 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:44.129993 2575 scope.go:117] "RemoveContainer" containerID="e8a25f1e9a523f26393ce0d34bfb763d1446471ccf47144f3e06a392d08c7b27" Sep 07 10:33:44.830690 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:44.830653 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" path="/var/lib/kubelet/pods/091d8b74-ca8b-43f1-b768-9447d0927edd/volumes" Sep 07 10:33:44.831247 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:33:44.831232 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" path="/var/lib/kubelet/pods/720b352f-8260-41bf-85c4-6b62516a8ee9/volumes" Sep 07 10:34:01.212714 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:01.212668 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-n8hzm"] Sep 07 10:34:01.213208 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:01.212932 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-n8hzm" podUID="c57c18bd-172d-420c-84d5-004cb82b2db1" containerName="manager" containerID="cri-o://c864bc966dbd172cea3e241b34263abe09162951a2c0a7b83b4d442d565d9fad" gracePeriod=10 Sep 07 10:34:01.458308 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:01.458277 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-n8hzm" Sep 07 10:34:01.536500 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:01.536406 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7k567\" (UniqueName: \"kubernetes.io/projected/c57c18bd-172d-420c-84d5-004cb82b2db1-kube-api-access-7k567\") pod \"c57c18bd-172d-420c-84d5-004cb82b2db1\" (UID: \"c57c18bd-172d-420c-84d5-004cb82b2db1\") " Sep 07 10:34:01.538626 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:01.538593 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c57c18bd-172d-420c-84d5-004cb82b2db1-kube-api-access-7k567" (OuterVolumeSpecName: "kube-api-access-7k567") pod "c57c18bd-172d-420c-84d5-004cb82b2db1" (UID: "c57c18bd-172d-420c-84d5-004cb82b2db1"). InnerVolumeSpecName "kube-api-access-7k567". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:34:01.637787 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:01.637745 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7k567\" (UniqueName: \"kubernetes.io/projected/c57c18bd-172d-420c-84d5-004cb82b2db1-kube-api-access-7k567\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:34:02.118858 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:02.118817 2575 generic.go:356] "Generic (PLEG): container finished" podID="c57c18bd-172d-420c-84d5-004cb82b2db1" containerID="c864bc966dbd172cea3e241b34263abe09162951a2c0a7b83b4d442d565d9fad" exitCode=0 Sep 07 10:34:02.118858 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:02.118859 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-n8hzm" Sep 07 10:34:02.119098 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:02.118883 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-n8hzm" event={"ID":"c57c18bd-172d-420c-84d5-004cb82b2db1","Type":"ContainerDied","Data":"c864bc966dbd172cea3e241b34263abe09162951a2c0a7b83b4d442d565d9fad"} Sep 07 10:34:02.119098 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:02.118947 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-n8hzm" event={"ID":"c57c18bd-172d-420c-84d5-004cb82b2db1","Type":"ContainerDied","Data":"401f980dda958f79ef8e98a572217e0170229974c8979f3ffb84e4ff84f7bd3d"} Sep 07 10:34:02.119098 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:02.118981 2575 scope.go:117] "RemoveContainer" containerID="c864bc966dbd172cea3e241b34263abe09162951a2c0a7b83b4d442d565d9fad" Sep 07 10:34:02.127976 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:02.127958 2575 scope.go:117] "RemoveContainer" containerID="c864bc966dbd172cea3e241b34263abe09162951a2c0a7b83b4d442d565d9fad" Sep 07 10:34:02.128305 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:34:02.128283 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c864bc966dbd172cea3e241b34263abe09162951a2c0a7b83b4d442d565d9fad\": container with ID starting with c864bc966dbd172cea3e241b34263abe09162951a2c0a7b83b4d442d565d9fad not found: ID does not exist" containerID="c864bc966dbd172cea3e241b34263abe09162951a2c0a7b83b4d442d565d9fad" Sep 07 10:34:02.128358 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:02.128325 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c864bc966dbd172cea3e241b34263abe09162951a2c0a7b83b4d442d565d9fad"} err="failed to get container status \"c864bc966dbd172cea3e241b34263abe09162951a2c0a7b83b4d442d565d9fad\": rpc error: code = NotFound desc = could not find container \"c864bc966dbd172cea3e241b34263abe09162951a2c0a7b83b4d442d565d9fad\": container with ID starting with c864bc966dbd172cea3e241b34263abe09162951a2c0a7b83b4d442d565d9fad not found: ID does not exist" Sep 07 10:34:02.139212 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:02.139180 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-n8hzm"] Sep 07 10:34:02.142924 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:02.142896 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-n8hzm"] Sep 07 10:34:02.831264 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:02.831233 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c57c18bd-172d-420c-84d5-004cb82b2db1" path="/var/lib/kubelet/pods/c57c18bd-172d-420c-84d5-004cb82b2db1/volumes" Sep 07 10:34:05.835782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.835743 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-pgfkx"] Sep 07 10:34:05.836163 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836037 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c57c18bd-172d-420c-84d5-004cb82b2db1" containerName="manager" Sep 07 10:34:05.836163 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836049 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c57c18bd-172d-420c-84d5-004cb82b2db1" containerName="manager" Sep 07 10:34:05.836163 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836058 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="registry" Sep 07 10:34:05.836163 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836063 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="registry" Sep 07 10:34:05.836163 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836072 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="offline" Sep 07 10:34:05.836163 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836078 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="offline" Sep 07 10:34:05.836163 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836088 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="online" Sep 07 10:34:05.836163 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836093 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="online" Sep 07 10:34:05.836163 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836102 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="online" Sep 07 10:34:05.836163 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836107 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="online" Sep 07 10:34:05.836163 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836113 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="ui" Sep 07 10:34:05.836163 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836118 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="ui" Sep 07 10:34:05.836163 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836124 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="feast-init" Sep 07 10:34:05.836163 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836130 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="feast-init" Sep 07 10:34:05.836163 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836136 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="feast-apply" Sep 07 10:34:05.836163 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836141 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="feast-apply" Sep 07 10:34:05.836163 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836149 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="ui" Sep 07 10:34:05.836163 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836154 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="ui" Sep 07 10:34:05.836669 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836177 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="feast-apply" Sep 07 10:34:05.836669 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836182 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="feast-apply" Sep 07 10:34:05.836669 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836188 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="offline" Sep 07 10:34:05.836669 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836192 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="offline" Sep 07 10:34:05.836669 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836202 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="feast-init" Sep 07 10:34:05.836669 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836207 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="feast-init" Sep 07 10:34:05.836669 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836252 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="online" Sep 07 10:34:05.836669 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836258 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="offline" Sep 07 10:34:05.836669 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836267 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="online" Sep 07 10:34:05.836669 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836274 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="ui" Sep 07 10:34:05.836669 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836281 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="offline" Sep 07 10:34:05.836669 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836285 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="720b352f-8260-41bf-85c4-6b62516a8ee9" containerName="registry" Sep 07 10:34:05.836669 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836291 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="091d8b74-ca8b-43f1-b768-9447d0927edd" containerName="ui" Sep 07 10:34:05.836669 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.836297 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c57c18bd-172d-420c-84d5-004cb82b2db1" containerName="manager" Sep 07 10:34:05.839173 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.839154 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pgfkx" Sep 07 10:34:05.841784 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.841758 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-wdvhx\"" Sep 07 10:34:05.841923 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.841822 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 07 10:34:05.841923 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.841827 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 07 10:34:05.846517 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.846494 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-pgfkx"] Sep 07 10:34:05.972926 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:05.972868 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz6nb\" (UniqueName: \"kubernetes.io/projected/b428f84c-d6f7-436c-a8e7-10943517a782-kube-api-access-cz6nb\") pod \"feast-operator-controller-manager-689b877b4b-pgfkx\" (UID: \"b428f84c-d6f7-436c-a8e7-10943517a782\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pgfkx" Sep 07 10:34:06.073506 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:06.073461 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz6nb\" (UniqueName: \"kubernetes.io/projected/b428f84c-d6f7-436c-a8e7-10943517a782-kube-api-access-cz6nb\") pod \"feast-operator-controller-manager-689b877b4b-pgfkx\" (UID: \"b428f84c-d6f7-436c-a8e7-10943517a782\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pgfkx" Sep 07 10:34:06.082193 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:06.082162 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz6nb\" (UniqueName: \"kubernetes.io/projected/b428f84c-d6f7-436c-a8e7-10943517a782-kube-api-access-cz6nb\") pod \"feast-operator-controller-manager-689b877b4b-pgfkx\" (UID: \"b428f84c-d6f7-436c-a8e7-10943517a782\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pgfkx" Sep 07 10:34:06.150280 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:06.150192 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pgfkx" Sep 07 10:34:06.418923 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:06.418843 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-pgfkx"] Sep 07 10:34:06.422119 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:34:06.422090 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb428f84c_d6f7_436c_a8e7_10943517a782.slice/crio-e57748ad47d2b5796444251f38d76f86c09e8677ea278125f5bfd92edfe835b5 WatchSource:0}: Error finding container e57748ad47d2b5796444251f38d76f86c09e8677ea278125f5bfd92edfe835b5: Status 404 returned error can't find the container with id e57748ad47d2b5796444251f38d76f86c09e8677ea278125f5bfd92edfe835b5 Sep 07 10:34:07.139385 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:07.139347 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pgfkx" event={"ID":"b428f84c-d6f7-436c-a8e7-10943517a782","Type":"ContainerStarted","Data":"bd841cd66abe89e7497c784c63e0e9b42fae017fea7f38608bf21bda730c12a5"} Sep 07 10:34:07.139787 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:07.139401 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pgfkx" event={"ID":"b428f84c-d6f7-436c-a8e7-10943517a782","Type":"ContainerStarted","Data":"e57748ad47d2b5796444251f38d76f86c09e8677ea278125f5bfd92edfe835b5"} Sep 07 10:34:07.139787 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:07.139461 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pgfkx" Sep 07 10:34:07.154550 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:07.154503 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pgfkx" podStartSLOduration=2.154489286 podStartE2EDuration="2.154489286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:34:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:34:07.153333128 +0000 UTC m=+1564.865575090" watchObservedRunningTime="2026-09-07 10:34:07.154489286 +0000 UTC m=+1564.866731230" Sep 07 10:34:18.148229 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:18.148195 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pgfkx" Sep 07 10:34:22.975989 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:22.975956 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt"] Sep 07 10:34:22.979789 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:22.979766 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:22.982485 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:22.982457 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 07 10:34:22.982485 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:22.982473 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 07 10:34:22.982737 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:22.982595 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 07 10:34:22.982737 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:22.982608 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 07 10:34:22.982737 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:22.982635 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 07 10:34:22.982737 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:22.982680 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 07 10:34:22.982933 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:22.982875 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-8nft4\"" Sep 07 10:34:22.991010 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:22.990986 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt"] Sep 07 10:34:23.120143 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:23.120102 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frhjk\" (UniqueName: \"kubernetes.io/projected/b2c0db3f-df10-4233-9e3d-44c6a78801b7-kube-api-access-frhjk\") pod \"feast-simple-feast-setup-57576b8fcf-hbrrt\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:23.120143 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:23.120144 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b2c0db3f-df10-4233-9e3d-44c6a78801b7-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-hbrrt\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:23.120407 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:23.120170 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hbrrt\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:23.120407 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:23.120304 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hbrrt\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:23.120407 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:23.120343 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hbrrt\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:23.120526 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:23.120426 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hbrrt\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:23.221535 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:23.221488 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hbrrt\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:23.221740 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:23.221547 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hbrrt\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:23.221740 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:23.221597 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hbrrt\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:23.221740 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:23.221650 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frhjk\" (UniqueName: \"kubernetes.io/projected/b2c0db3f-df10-4233-9e3d-44c6a78801b7-kube-api-access-frhjk\") pod \"feast-simple-feast-setup-57576b8fcf-hbrrt\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:23.221740 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:23.221688 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b2c0db3f-df10-4233-9e3d-44c6a78801b7-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-hbrrt\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:23.221740 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:23.221730 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hbrrt\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:23.225190 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:23.225152 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hbrrt\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:23.227449 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:23.227366 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b2c0db3f-df10-4233-9e3d-44c6a78801b7-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-hbrrt\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:23.228055 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:23.228029 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hbrrt\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:23.229287 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:23.229264 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hbrrt\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:23.229724 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:23.229706 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hbrrt\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:23.235112 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:23.235085 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frhjk\" (UniqueName: \"kubernetes.io/projected/b2c0db3f-df10-4233-9e3d-44c6a78801b7-kube-api-access-frhjk\") pod \"feast-simple-feast-setup-57576b8fcf-hbrrt\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:23.289313 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:23.289271 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:23.565023 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:23.564816 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt"] Sep 07 10:34:23.567698 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:34:23.567670 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2c0db3f_df10_4233_9e3d_44c6a78801b7.slice/crio-dd7df6bfb69c7b6d22e5071377e8bd9e8ca7cb3fd833e1698720f4f2ebd6d240 WatchSource:0}: Error finding container dd7df6bfb69c7b6d22e5071377e8bd9e8ca7cb3fd833e1698720f4f2ebd6d240: Status 404 returned error can't find the container with id dd7df6bfb69c7b6d22e5071377e8bd9e8ca7cb3fd833e1698720f4f2ebd6d240 Sep 07 10:34:24.209530 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:24.209492 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" event={"ID":"b2c0db3f-df10-4233-9e3d-44c6a78801b7","Type":"ContainerStarted","Data":"5005695e839b247cf537cec183680d2ae03d85e2cdc65c1fe2608cd1bb3b8c1e"} Sep 07 10:34:24.210040 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:24.209544 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" event={"ID":"b2c0db3f-df10-4233-9e3d-44c6a78801b7","Type":"ContainerStarted","Data":"dd7df6bfb69c7b6d22e5071377e8bd9e8ca7cb3fd833e1698720f4f2ebd6d240"} Sep 07 10:34:29.231650 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:29.231613 2575 generic.go:356] "Generic (PLEG): container finished" podID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerID="5005695e839b247cf537cec183680d2ae03d85e2cdc65c1fe2608cd1bb3b8c1e" exitCode=0 Sep 07 10:34:29.232122 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:29.231668 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" event={"ID":"b2c0db3f-df10-4233-9e3d-44c6a78801b7","Type":"ContainerDied","Data":"5005695e839b247cf537cec183680d2ae03d85e2cdc65c1fe2608cd1bb3b8c1e"} Sep 07 10:34:30.237989 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:30.237947 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" event={"ID":"b2c0db3f-df10-4233-9e3d-44c6a78801b7","Type":"ContainerStarted","Data":"8d93b24ff080f03d3058d4a93c468dbb267e870957c6135cfa8e837d51ae9c21"} Sep 07 10:34:36.263110 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:36.263068 2575 generic.go:356] "Generic (PLEG): container finished" podID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerID="8d93b24ff080f03d3058d4a93c468dbb267e870957c6135cfa8e837d51ae9c21" exitCode=0 Sep 07 10:34:36.263519 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:36.263152 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" event={"ID":"b2c0db3f-df10-4233-9e3d-44c6a78801b7","Type":"ContainerDied","Data":"8d93b24ff080f03d3058d4a93c468dbb267e870957c6135cfa8e837d51ae9c21"} Sep 07 10:34:37.273567 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:37.273530 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" event={"ID":"b2c0db3f-df10-4233-9e3d-44c6a78801b7","Type":"ContainerStarted","Data":"cdb677c4a5eadaa873fafff5b6517ab69847392f1b2f79d768b8873296d6209a"} Sep 07 10:34:37.274040 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:37.273587 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" event={"ID":"b2c0db3f-df10-4233-9e3d-44c6a78801b7","Type":"ContainerStarted","Data":"cfafb3e3913ce6ac6ed620012943ff5c997b01763672225210b9ac25d1187f1a"} Sep 07 10:34:37.274040 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:37.273610 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" event={"ID":"b2c0db3f-df10-4233-9e3d-44c6a78801b7","Type":"ContainerStarted","Data":"69dd139037d7eac44aaeb881c9d25ac00e7e391c06b7f27e101671cb96a392a8"} Sep 07 10:34:37.274040 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:37.273628 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" event={"ID":"b2c0db3f-df10-4233-9e3d-44c6a78801b7","Type":"ContainerStarted","Data":"3832642466b72e703316af28e7a75f135a30822e175ce10245b065a740cf5229"} Sep 07 10:34:37.303851 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:37.303764 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" podStartSLOduration=15.303742863 podStartE2EDuration="15.303742863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:34:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:34:37.30162809 +0000 UTC m=+1595.013870035" watchObservedRunningTime="2026-09-07 10:34:37.303742863 +0000 UTC m=+1595.015984808" Sep 07 10:34:38.290188 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:38.290149 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:38.290763 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:38.290732 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:38.290931 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:38.290851 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:38.290931 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:38.290872 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:38.291918 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:38.291885 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Sep 07 10:34:38.292046 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:38.291950 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused" Sep 07 10:34:38.292046 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:38.292037 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.29:8816: connect: connection refused" Sep 07 10:34:38.292163 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:38.292067 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.29:6571: connect: connection refused" Sep 07 10:34:41.290250 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:41.290197 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused" Sep 07 10:34:41.290844 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:41.290220 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.29:8816: connect: connection refused" Sep 07 10:34:41.290982 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:41.290228 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Sep 07 10:34:41.290982 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:41.290274 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.29:6571: connect: connection refused" Sep 07 10:34:43.289821 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:43.289751 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:43.290317 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:43.289851 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:43.290317 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:43.289874 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:43.290440 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:43.290371 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:44.289993 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:44.289835 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Sep 07 10:34:44.289993 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:44.289944 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused" Sep 07 10:34:44.291066 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:44.291042 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:44.291200 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:44.291189 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:44.314951 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:44.314908 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:44.315283 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:44.315260 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:47.290463 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:47.290431 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:47.295352 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:47.295330 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:47.327987 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:47.327947 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:34:47.330593 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:34:47.330573 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:35:48.973579 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:48.973539 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt"] Sep 07 10:35:48.974201 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:48.973785 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="registry" containerID="cri-o://3832642466b72e703316af28e7a75f135a30822e175ce10245b065a740cf5229" gracePeriod=30 Sep 07 10:35:48.974201 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:48.973872 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="ui" containerID="cri-o://cdb677c4a5eadaa873fafff5b6517ab69847392f1b2f79d768b8873296d6209a" gracePeriod=30 Sep 07 10:35:48.974201 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:48.973881 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="offline" containerID="cri-o://cfafb3e3913ce6ac6ed620012943ff5c997b01763672225210b9ac25d1187f1a" gracePeriod=30 Sep 07 10:35:48.974201 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:48.973969 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="online" containerID="cri-o://69dd139037d7eac44aaeb881c9d25ac00e7e391c06b7f27e101671cb96a392a8" gracePeriod=30 Sep 07 10:35:49.378063 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.378026 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64"] Sep 07 10:35:49.382134 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.382109 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:35:49.384667 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.384641 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-bbc65\"" Sep 07 10:35:49.393743 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.393717 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64"] Sep 07 10:35:49.468871 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.468823 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j2h64\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:35:49.469082 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.468895 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j2h64\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:35:49.469082 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.468923 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r64m9\" (UniqueName: \"kubernetes.io/projected/d25f39bb-9452-4682-ad76-3fafb7cc75f2-kube-api-access-r64m9\") pod \"feast-simple-feast-setup-57576b8fcf-j2h64\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:35:49.469082 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.468985 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d25f39bb-9452-4682-ad76-3fafb7cc75f2-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-j2h64\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:35:49.469082 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.469048 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j2h64\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:35:49.469285 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.469132 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j2h64\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:35:49.570274 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.570220 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j2h64\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:35:49.570274 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.570277 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j2h64\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:35:49.570534 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.570321 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j2h64\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:35:49.570534 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.570344 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j2h64\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:35:49.570534 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.570361 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r64m9\" (UniqueName: \"kubernetes.io/projected/d25f39bb-9452-4682-ad76-3fafb7cc75f2-kube-api-access-r64m9\") pod \"feast-simple-feast-setup-57576b8fcf-j2h64\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:35:49.570678 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.570594 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d25f39bb-9452-4682-ad76-3fafb7cc75f2-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-j2h64\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:35:49.571094 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.571066 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d25f39bb-9452-4682-ad76-3fafb7cc75f2-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-j2h64\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:35:49.573500 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.573472 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j2h64\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:35:49.573622 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.573472 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j2h64\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:35:49.574132 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.574107 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j2h64\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:35:49.574691 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.574669 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-j2h64\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:35:49.580945 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.580908 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r64m9\" (UniqueName: \"kubernetes.io/projected/d25f39bb-9452-4682-ad76-3fafb7cc75f2-kube-api-access-r64m9\") pod \"feast-simple-feast-setup-57576b8fcf-j2h64\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:35:49.691540 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.691505 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:35:49.969159 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:49.969129 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64"] Sep 07 10:35:49.971503 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:35:49.971475 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd25f39bb_9452_4682_ad76_3fafb7cc75f2.slice/crio-6c1e3e2812bcd2396fec707be0702a46d698e52a438d68d089429948e35a0342 WatchSource:0}: Error finding container 6c1e3e2812bcd2396fec707be0702a46d698e52a438d68d089429948e35a0342: Status 404 returned error can't find the container with id 6c1e3e2812bcd2396fec707be0702a46d698e52a438d68d089429948e35a0342 Sep 07 10:35:50.584836 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:50.584790 2575 generic.go:356] "Generic (PLEG): container finished" podID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerID="cdb677c4a5eadaa873fafff5b6517ab69847392f1b2f79d768b8873296d6209a" exitCode=0 Sep 07 10:35:50.584836 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:50.584830 2575 generic.go:356] "Generic (PLEG): container finished" podID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerID="69dd139037d7eac44aaeb881c9d25ac00e7e391c06b7f27e101671cb96a392a8" exitCode=0 Sep 07 10:35:50.585262 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:50.584835 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" event={"ID":"b2c0db3f-df10-4233-9e3d-44c6a78801b7","Type":"ContainerDied","Data":"cdb677c4a5eadaa873fafff5b6517ab69847392f1b2f79d768b8873296d6209a"} Sep 07 10:35:50.585262 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:50.584877 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" event={"ID":"b2c0db3f-df10-4233-9e3d-44c6a78801b7","Type":"ContainerDied","Data":"69dd139037d7eac44aaeb881c9d25ac00e7e391c06b7f27e101671cb96a392a8"} Sep 07 10:35:50.586370 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:50.586348 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" event={"ID":"d25f39bb-9452-4682-ad76-3fafb7cc75f2","Type":"ContainerStarted","Data":"ed26a6ffa278dc9cb2b843074ba07e08e39a3706dc98b1c45301b74269e6ab7f"} Sep 07 10:35:50.586466 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:50.586382 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" event={"ID":"d25f39bb-9452-4682-ad76-3fafb7cc75f2","Type":"ContainerStarted","Data":"6c1e3e2812bcd2396fec707be0702a46d698e52a438d68d089429948e35a0342"} Sep 07 10:35:55.611475 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:55.611441 2575 generic.go:356] "Generic (PLEG): container finished" podID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerID="ed26a6ffa278dc9cb2b843074ba07e08e39a3706dc98b1c45301b74269e6ab7f" exitCode=0 Sep 07 10:35:55.611888 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:55.611514 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" event={"ID":"d25f39bb-9452-4682-ad76-3fafb7cc75f2","Type":"ContainerDied","Data":"ed26a6ffa278dc9cb2b843074ba07e08e39a3706dc98b1c45301b74269e6ab7f"} Sep 07 10:35:56.617983 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:56.617946 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" event={"ID":"d25f39bb-9452-4682-ad76-3fafb7cc75f2","Type":"ContainerStarted","Data":"de1cefac3a957ce4838b91c439e8e3de4dcd9f1f9f625b1df9a4c68f4303bf6b"} Sep 07 10:35:57.327679 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:57.327632 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused" Sep 07 10:35:57.327868 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:35:57.327722 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Sep 07 10:36:02.644182 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:02.644144 2575 generic.go:356] "Generic (PLEG): container finished" podID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerID="de1cefac3a957ce4838b91c439e8e3de4dcd9f1f9f625b1df9a4c68f4303bf6b" exitCode=0 Sep 07 10:36:02.644557 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:02.644215 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" event={"ID":"d25f39bb-9452-4682-ad76-3fafb7cc75f2","Type":"ContainerDied","Data":"de1cefac3a957ce4838b91c439e8e3de4dcd9f1f9f625b1df9a4c68f4303bf6b"} Sep 07 10:36:03.653973 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:03.653930 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" event={"ID":"d25f39bb-9452-4682-ad76-3fafb7cc75f2","Type":"ContainerStarted","Data":"ed8ed32cbd2ef45888d1fe4c5fe8daf4fd2916e03c91548012c233f1a6ee64be"} Sep 07 10:36:03.654462 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:03.653995 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" event={"ID":"d25f39bb-9452-4682-ad76-3fafb7cc75f2","Type":"ContainerStarted","Data":"6eb1848f2d66dd65175120dba66e1605e25adac474a4834812857e716a4c20c3"} Sep 07 10:36:03.654462 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:03.654017 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" event={"ID":"d25f39bb-9452-4682-ad76-3fafb7cc75f2","Type":"ContainerStarted","Data":"a2d70d7520be57d2da47dde4e7f2f7e1bb5c2b26998ce867797feb2530bf3fd0"} Sep 07 10:36:03.654462 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:03.654036 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" event={"ID":"d25f39bb-9452-4682-ad76-3fafb7cc75f2","Type":"ContainerStarted","Data":"53fc8af5d527cc1e4069c3ee81879000a33d663efec14d95276c130ceda01825"} Sep 07 10:36:03.680420 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:03.680328 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" podStartSLOduration=14.680304501 podStartE2EDuration="14.680304501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:35:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:36:03.677458477 +0000 UTC m=+1681.389700437" watchObservedRunningTime="2026-09-07 10:36:03.680304501 +0000 UTC m=+1681.392546448" Sep 07 10:36:04.692046 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:04.692002 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:36:04.692509 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:04.692074 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:36:04.692509 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:04.692092 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:36:04.692509 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:04.692107 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:36:04.694334 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:04.694298 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.30:6571: connect: connection refused" Sep 07 10:36:04.694450 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:04.694342 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused" Sep 07 10:36:04.694450 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:04.694421 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.30:8443: connect: connection refused" Sep 07 10:36:04.694602 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:04.694320 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.30:8816: connect: connection refused" Sep 07 10:36:07.328013 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:07.327963 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused" Sep 07 10:36:07.328582 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:07.328526 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Sep 07 10:36:07.692609 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:07.692486 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused" Sep 07 10:36:07.692609 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:07.692499 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.30:8816: connect: connection refused" Sep 07 10:36:07.692956 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:07.692918 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.30:6571: connect: connection refused" Sep 07 10:36:07.693110 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:07.692936 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.30:8443: connect: connection refused" Sep 07 10:36:09.691693 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:09.691650 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:36:09.692238 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:09.691750 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:36:09.692238 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:09.691776 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:36:09.692238 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:09.691816 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:36:10.692611 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:10.692566 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.30:8443: connect: connection refused" Sep 07 10:36:10.693732 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:10.693705 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:36:10.694289 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:10.694269 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:36:10.696950 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:10.696926 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:36:10.697184 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:10.697164 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:36:10.754449 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:10.754412 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:36:11.704197 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:11.704170 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:36:13.692544 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:13.692517 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:36:13.711086 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:13.711059 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:36:17.327570 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:17.327512 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused" Sep 07 10:36:17.328055 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:17.327537 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Sep 07 10:36:17.328055 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:17.327670 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:36:17.328055 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:17.327728 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:36:19.619075 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.619048 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:36:19.642374 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.641920 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-online-tls\") pod \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " Sep 07 10:36:19.642374 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.641999 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frhjk\" (UniqueName: \"kubernetes.io/projected/b2c0db3f-df10-4233-9e3d-44c6a78801b7-kube-api-access-frhjk\") pod \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " Sep 07 10:36:19.642374 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.642121 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b2c0db3f-df10-4233-9e3d-44c6a78801b7-feast-data\") pod \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " Sep 07 10:36:19.642374 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.642151 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-ui-tls\") pod \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " Sep 07 10:36:19.642374 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.642173 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-registry-tls\") pod \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " Sep 07 10:36:19.642374 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.642261 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-offline-tls\") pod \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\" (UID: \"b2c0db3f-df10-4233-9e3d-44c6a78801b7\") " Sep 07 10:36:19.643240 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.642958 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2c0db3f-df10-4233-9e3d-44c6a78801b7-feast-data" (OuterVolumeSpecName: "feast-data") pod "b2c0db3f-df10-4233-9e3d-44c6a78801b7" (UID: "b2c0db3f-df10-4233-9e3d-44c6a78801b7"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 10:36:19.645065 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.645025 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "b2c0db3f-df10-4233-9e3d-44c6a78801b7" (UID: "b2c0db3f-df10-4233-9e3d-44c6a78801b7"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:36:19.645550 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.645520 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b2c0db3f-df10-4233-9e3d-44c6a78801b7" (UID: "b2c0db3f-df10-4233-9e3d-44c6a78801b7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:36:19.645646 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.645572 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-online-tls" (OuterVolumeSpecName: "online-tls") pod "b2c0db3f-df10-4233-9e3d-44c6a78801b7" (UID: "b2c0db3f-df10-4233-9e3d-44c6a78801b7"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:36:19.645696 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.645642 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "b2c0db3f-df10-4233-9e3d-44c6a78801b7" (UID: "b2c0db3f-df10-4233-9e3d-44c6a78801b7"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:36:19.645978 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.645960 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2c0db3f-df10-4233-9e3d-44c6a78801b7-kube-api-access-frhjk" (OuterVolumeSpecName: "kube-api-access-frhjk") pod "b2c0db3f-df10-4233-9e3d-44c6a78801b7" (UID: "b2c0db3f-df10-4233-9e3d-44c6a78801b7"). InnerVolumeSpecName "kube-api-access-frhjk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:36:19.739837 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.739734 2575 generic.go:356] "Generic (PLEG): container finished" podID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerID="cfafb3e3913ce6ac6ed620012943ff5c997b01763672225210b9ac25d1187f1a" exitCode=137 Sep 07 10:36:19.739837 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.739769 2575 generic.go:356] "Generic (PLEG): container finished" podID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerID="3832642466b72e703316af28e7a75f135a30822e175ce10245b065a740cf5229" exitCode=137 Sep 07 10:36:19.739837 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.739809 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" event={"ID":"b2c0db3f-df10-4233-9e3d-44c6a78801b7","Type":"ContainerDied","Data":"cfafb3e3913ce6ac6ed620012943ff5c997b01763672225210b9ac25d1187f1a"} Sep 07 10:36:19.739837 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.739827 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" Sep 07 10:36:19.740115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.739872 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" event={"ID":"b2c0db3f-df10-4233-9e3d-44c6a78801b7","Type":"ContainerDied","Data":"3832642466b72e703316af28e7a75f135a30822e175ce10245b065a740cf5229"} Sep 07 10:36:19.740115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.739894 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt" event={"ID":"b2c0db3f-df10-4233-9e3d-44c6a78801b7","Type":"ContainerDied","Data":"dd7df6bfb69c7b6d22e5071377e8bd9e8ca7cb3fd833e1698720f4f2ebd6d240"} Sep 07 10:36:19.740115 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.739917 2575 scope.go:117] "RemoveContainer" containerID="cdb677c4a5eadaa873fafff5b6517ab69847392f1b2f79d768b8873296d6209a" Sep 07 10:36:19.743166 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.743135 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-online-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:36:19.743265 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.743178 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frhjk\" (UniqueName: \"kubernetes.io/projected/b2c0db3f-df10-4233-9e3d-44c6a78801b7-kube-api-access-frhjk\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:36:19.743265 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.743193 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b2c0db3f-df10-4233-9e3d-44c6a78801b7-feast-data\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:36:19.743265 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.743208 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-ui-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:36:19.743265 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.743221 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-registry-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:36:19.743265 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.743233 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b2c0db3f-df10-4233-9e3d-44c6a78801b7-offline-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:36:19.748829 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.748787 2575 scope.go:117] "RemoveContainer" containerID="cfafb3e3913ce6ac6ed620012943ff5c997b01763672225210b9ac25d1187f1a" Sep 07 10:36:19.756700 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.756681 2575 scope.go:117] "RemoveContainer" containerID="69dd139037d7eac44aaeb881c9d25ac00e7e391c06b7f27e101671cb96a392a8" Sep 07 10:36:19.762623 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.762594 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt"] Sep 07 10:36:19.765929 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.765897 2575 scope.go:117] "RemoveContainer" containerID="3832642466b72e703316af28e7a75f135a30822e175ce10245b065a740cf5229" Sep 07 10:36:19.766097 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.766076 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-hbrrt"] Sep 07 10:36:19.773713 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.773690 2575 scope.go:117] "RemoveContainer" containerID="8d93b24ff080f03d3058d4a93c468dbb267e870957c6135cfa8e837d51ae9c21" Sep 07 10:36:19.781073 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.781049 2575 scope.go:117] "RemoveContainer" containerID="5005695e839b247cf537cec183680d2ae03d85e2cdc65c1fe2608cd1bb3b8c1e" Sep 07 10:36:19.788210 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.788192 2575 scope.go:117] "RemoveContainer" containerID="cdb677c4a5eadaa873fafff5b6517ab69847392f1b2f79d768b8873296d6209a" Sep 07 10:36:19.788482 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:36:19.788463 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdb677c4a5eadaa873fafff5b6517ab69847392f1b2f79d768b8873296d6209a\": container with ID starting with cdb677c4a5eadaa873fafff5b6517ab69847392f1b2f79d768b8873296d6209a not found: ID does not exist" containerID="cdb677c4a5eadaa873fafff5b6517ab69847392f1b2f79d768b8873296d6209a" Sep 07 10:36:19.788553 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.788491 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdb677c4a5eadaa873fafff5b6517ab69847392f1b2f79d768b8873296d6209a"} err="failed to get container status \"cdb677c4a5eadaa873fafff5b6517ab69847392f1b2f79d768b8873296d6209a\": rpc error: code = NotFound desc = could not find container \"cdb677c4a5eadaa873fafff5b6517ab69847392f1b2f79d768b8873296d6209a\": container with ID starting with cdb677c4a5eadaa873fafff5b6517ab69847392f1b2f79d768b8873296d6209a not found: ID does not exist" Sep 07 10:36:19.788553 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.788512 2575 scope.go:117] "RemoveContainer" containerID="cfafb3e3913ce6ac6ed620012943ff5c997b01763672225210b9ac25d1187f1a" Sep 07 10:36:19.788835 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:36:19.788806 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfafb3e3913ce6ac6ed620012943ff5c997b01763672225210b9ac25d1187f1a\": container with ID starting with cfafb3e3913ce6ac6ed620012943ff5c997b01763672225210b9ac25d1187f1a not found: ID does not exist" containerID="cfafb3e3913ce6ac6ed620012943ff5c997b01763672225210b9ac25d1187f1a" Sep 07 10:36:19.788910 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.788835 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfafb3e3913ce6ac6ed620012943ff5c997b01763672225210b9ac25d1187f1a"} err="failed to get container status \"cfafb3e3913ce6ac6ed620012943ff5c997b01763672225210b9ac25d1187f1a\": rpc error: code = NotFound desc = could not find container \"cfafb3e3913ce6ac6ed620012943ff5c997b01763672225210b9ac25d1187f1a\": container with ID starting with cfafb3e3913ce6ac6ed620012943ff5c997b01763672225210b9ac25d1187f1a not found: ID does not exist" Sep 07 10:36:19.788910 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.788851 2575 scope.go:117] "RemoveContainer" containerID="69dd139037d7eac44aaeb881c9d25ac00e7e391c06b7f27e101671cb96a392a8" Sep 07 10:36:19.789105 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:36:19.789089 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69dd139037d7eac44aaeb881c9d25ac00e7e391c06b7f27e101671cb96a392a8\": container with ID starting with 69dd139037d7eac44aaeb881c9d25ac00e7e391c06b7f27e101671cb96a392a8 not found: ID does not exist" containerID="69dd139037d7eac44aaeb881c9d25ac00e7e391c06b7f27e101671cb96a392a8" Sep 07 10:36:19.789159 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.789110 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69dd139037d7eac44aaeb881c9d25ac00e7e391c06b7f27e101671cb96a392a8"} err="failed to get container status \"69dd139037d7eac44aaeb881c9d25ac00e7e391c06b7f27e101671cb96a392a8\": rpc error: code = NotFound desc = could not find container \"69dd139037d7eac44aaeb881c9d25ac00e7e391c06b7f27e101671cb96a392a8\": container with ID starting with 69dd139037d7eac44aaeb881c9d25ac00e7e391c06b7f27e101671cb96a392a8 not found: ID does not exist" Sep 07 10:36:19.789159 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.789124 2575 scope.go:117] "RemoveContainer" containerID="3832642466b72e703316af28e7a75f135a30822e175ce10245b065a740cf5229" Sep 07 10:36:19.789377 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:36:19.789359 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3832642466b72e703316af28e7a75f135a30822e175ce10245b065a740cf5229\": container with ID starting with 3832642466b72e703316af28e7a75f135a30822e175ce10245b065a740cf5229 not found: ID does not exist" containerID="3832642466b72e703316af28e7a75f135a30822e175ce10245b065a740cf5229" Sep 07 10:36:19.789451 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.789380 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3832642466b72e703316af28e7a75f135a30822e175ce10245b065a740cf5229"} err="failed to get container status \"3832642466b72e703316af28e7a75f135a30822e175ce10245b065a740cf5229\": rpc error: code = NotFound desc = could not find container \"3832642466b72e703316af28e7a75f135a30822e175ce10245b065a740cf5229\": container with ID starting with 3832642466b72e703316af28e7a75f135a30822e175ce10245b065a740cf5229 not found: ID does not exist" Sep 07 10:36:19.789451 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.789395 2575 scope.go:117] "RemoveContainer" containerID="8d93b24ff080f03d3058d4a93c468dbb267e870957c6135cfa8e837d51ae9c21" Sep 07 10:36:19.789621 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:36:19.789607 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d93b24ff080f03d3058d4a93c468dbb267e870957c6135cfa8e837d51ae9c21\": container with ID starting with 8d93b24ff080f03d3058d4a93c468dbb267e870957c6135cfa8e837d51ae9c21 not found: ID does not exist" containerID="8d93b24ff080f03d3058d4a93c468dbb267e870957c6135cfa8e837d51ae9c21" Sep 07 10:36:19.789660 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.789624 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d93b24ff080f03d3058d4a93c468dbb267e870957c6135cfa8e837d51ae9c21"} err="failed to get container status \"8d93b24ff080f03d3058d4a93c468dbb267e870957c6135cfa8e837d51ae9c21\": rpc error: code = NotFound desc = could not find container \"8d93b24ff080f03d3058d4a93c468dbb267e870957c6135cfa8e837d51ae9c21\": container with ID starting with 8d93b24ff080f03d3058d4a93c468dbb267e870957c6135cfa8e837d51ae9c21 not found: ID does not exist" Sep 07 10:36:19.789660 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.789637 2575 scope.go:117] "RemoveContainer" containerID="5005695e839b247cf537cec183680d2ae03d85e2cdc65c1fe2608cd1bb3b8c1e" Sep 07 10:36:19.789893 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:36:19.789873 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5005695e839b247cf537cec183680d2ae03d85e2cdc65c1fe2608cd1bb3b8c1e\": container with ID starting with 5005695e839b247cf537cec183680d2ae03d85e2cdc65c1fe2608cd1bb3b8c1e not found: ID does not exist" containerID="5005695e839b247cf537cec183680d2ae03d85e2cdc65c1fe2608cd1bb3b8c1e" Sep 07 10:36:19.789938 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.789902 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5005695e839b247cf537cec183680d2ae03d85e2cdc65c1fe2608cd1bb3b8c1e"} err="failed to get container status \"5005695e839b247cf537cec183680d2ae03d85e2cdc65c1fe2608cd1bb3b8c1e\": rpc error: code = NotFound desc = could not find container \"5005695e839b247cf537cec183680d2ae03d85e2cdc65c1fe2608cd1bb3b8c1e\": container with ID starting with 5005695e839b247cf537cec183680d2ae03d85e2cdc65c1fe2608cd1bb3b8c1e not found: ID does not exist" Sep 07 10:36:19.789938 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.789921 2575 scope.go:117] "RemoveContainer" containerID="cdb677c4a5eadaa873fafff5b6517ab69847392f1b2f79d768b8873296d6209a" Sep 07 10:36:19.790190 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.790169 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdb677c4a5eadaa873fafff5b6517ab69847392f1b2f79d768b8873296d6209a"} err="failed to get container status \"cdb677c4a5eadaa873fafff5b6517ab69847392f1b2f79d768b8873296d6209a\": rpc error: code = NotFound desc = could not find container \"cdb677c4a5eadaa873fafff5b6517ab69847392f1b2f79d768b8873296d6209a\": container with ID starting with cdb677c4a5eadaa873fafff5b6517ab69847392f1b2f79d768b8873296d6209a not found: ID does not exist" Sep 07 10:36:19.790190 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.790189 2575 scope.go:117] "RemoveContainer" containerID="cfafb3e3913ce6ac6ed620012943ff5c997b01763672225210b9ac25d1187f1a" Sep 07 10:36:19.790392 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.790376 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfafb3e3913ce6ac6ed620012943ff5c997b01763672225210b9ac25d1187f1a"} err="failed to get container status \"cfafb3e3913ce6ac6ed620012943ff5c997b01763672225210b9ac25d1187f1a\": rpc error: code = NotFound desc = could not find container \"cfafb3e3913ce6ac6ed620012943ff5c997b01763672225210b9ac25d1187f1a\": container with ID starting with cfafb3e3913ce6ac6ed620012943ff5c997b01763672225210b9ac25d1187f1a not found: ID does not exist" Sep 07 10:36:19.790445 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.790392 2575 scope.go:117] "RemoveContainer" containerID="69dd139037d7eac44aaeb881c9d25ac00e7e391c06b7f27e101671cb96a392a8" Sep 07 10:36:19.790610 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.790591 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69dd139037d7eac44aaeb881c9d25ac00e7e391c06b7f27e101671cb96a392a8"} err="failed to get container status \"69dd139037d7eac44aaeb881c9d25ac00e7e391c06b7f27e101671cb96a392a8\": rpc error: code = NotFound desc = could not find container \"69dd139037d7eac44aaeb881c9d25ac00e7e391c06b7f27e101671cb96a392a8\": container with ID starting with 69dd139037d7eac44aaeb881c9d25ac00e7e391c06b7f27e101671cb96a392a8 not found: ID does not exist" Sep 07 10:36:19.790658 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.790610 2575 scope.go:117] "RemoveContainer" containerID="3832642466b72e703316af28e7a75f135a30822e175ce10245b065a740cf5229" Sep 07 10:36:19.790828 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.790779 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3832642466b72e703316af28e7a75f135a30822e175ce10245b065a740cf5229"} err="failed to get container status \"3832642466b72e703316af28e7a75f135a30822e175ce10245b065a740cf5229\": rpc error: code = NotFound desc = could not find container \"3832642466b72e703316af28e7a75f135a30822e175ce10245b065a740cf5229\": container with ID starting with 3832642466b72e703316af28e7a75f135a30822e175ce10245b065a740cf5229 not found: ID does not exist" Sep 07 10:36:19.790828 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.790814 2575 scope.go:117] "RemoveContainer" containerID="8d93b24ff080f03d3058d4a93c468dbb267e870957c6135cfa8e837d51ae9c21" Sep 07 10:36:19.791029 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.791007 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d93b24ff080f03d3058d4a93c468dbb267e870957c6135cfa8e837d51ae9c21"} err="failed to get container status \"8d93b24ff080f03d3058d4a93c468dbb267e870957c6135cfa8e837d51ae9c21\": rpc error: code = NotFound desc = could not find container \"8d93b24ff080f03d3058d4a93c468dbb267e870957c6135cfa8e837d51ae9c21\": container with ID starting with 8d93b24ff080f03d3058d4a93c468dbb267e870957c6135cfa8e837d51ae9c21 not found: ID does not exist" Sep 07 10:36:19.791029 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.791028 2575 scope.go:117] "RemoveContainer" containerID="5005695e839b247cf537cec183680d2ae03d85e2cdc65c1fe2608cd1bb3b8c1e" Sep 07 10:36:19.791224 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:19.791207 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5005695e839b247cf537cec183680d2ae03d85e2cdc65c1fe2608cd1bb3b8c1e"} err="failed to get container status \"5005695e839b247cf537cec183680d2ae03d85e2cdc65c1fe2608cd1bb3b8c1e\": rpc error: code = NotFound desc = could not find container \"5005695e839b247cf537cec183680d2ae03d85e2cdc65c1fe2608cd1bb3b8c1e\": container with ID starting with 5005695e839b247cf537cec183680d2ae03d85e2cdc65c1fe2608cd1bb3b8c1e not found: ID does not exist" Sep 07 10:36:20.830783 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:36:20.830749 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" path="/var/lib/kubelet/pods/b2c0db3f-df10-4233-9e3d-44c6a78801b7/volumes" Sep 07 10:37:16.239060 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.239023 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7"] Sep 07 10:37:16.239476 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.239350 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="offline" Sep 07 10:37:16.239476 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.239360 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="offline" Sep 07 10:37:16.239476 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.239369 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="feast-init" Sep 07 10:37:16.239476 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.239375 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="feast-init" Sep 07 10:37:16.239476 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.239382 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="ui" Sep 07 10:37:16.239476 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.239388 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="ui" Sep 07 10:37:16.239476 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.239404 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="feast-apply" Sep 07 10:37:16.239476 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.239409 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="feast-apply" Sep 07 10:37:16.239476 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.239416 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="online" Sep 07 10:37:16.239476 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.239421 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="online" Sep 07 10:37:16.239476 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.239428 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="registry" Sep 07 10:37:16.239476 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.239433 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="registry" Sep 07 10:37:16.239910 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.239486 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="offline" Sep 07 10:37:16.239910 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.239498 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="online" Sep 07 10:37:16.239910 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.239505 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="ui" Sep 07 10:37:16.239910 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.239514 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2c0db3f-df10-4233-9e3d-44c6a78801b7" containerName="registry" Sep 07 10:37:16.242833 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.242815 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:16.245247 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.245223 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 07 10:37:16.245373 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.245244 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 07 10:37:16.245373 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.245231 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-csn55\"" Sep 07 10:37:16.245373 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.245301 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 07 10:37:16.245537 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.245420 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 07 10:37:16.245715 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.245699 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 07 10:37:16.246008 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.245988 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 07 10:37:16.254267 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.254247 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7"] Sep 07 10:37:16.316228 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.316192 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8trr\" (UniqueName: \"kubernetes.io/projected/d9ef2f40-8495-437a-9e6b-7735ada58880-kube-api-access-v8trr\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jfgr7\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:16.316386 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.316235 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d9ef2f40-8495-437a-9e6b-7735ada58880-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jfgr7\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:16.316386 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.316253 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jfgr7\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:16.316386 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.316324 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d9ef2f40-8495-437a-9e6b-7735ada58880-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jfgr7\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:16.316386 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.316353 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jfgr7\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:16.316386 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.316380 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jfgr7\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:16.417202 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.417148 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8trr\" (UniqueName: \"kubernetes.io/projected/d9ef2f40-8495-437a-9e6b-7735ada58880-kube-api-access-v8trr\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jfgr7\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:16.417404 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.417215 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d9ef2f40-8495-437a-9e6b-7735ada58880-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jfgr7\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:16.417404 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.417242 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jfgr7\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:16.417404 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.417269 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d9ef2f40-8495-437a-9e6b-7735ada58880-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jfgr7\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:16.417404 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.417293 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jfgr7\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:16.417404 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.417327 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jfgr7\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:16.417630 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:37:16.417403 2575 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 07 10:37:16.417630 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:37:16.417470 2575 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Sep 07 10:37:16.417630 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:37:16.417492 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-ui-tls podName:d9ef2f40-8495-437a-9e6b-7735ada58880 nodeName:}" failed. No retries permitted until 2026-09-07 10:37:16.917471236 +0000 UTC m=+1754.629713159 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-ui-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" (UID: "d9ef2f40-8495-437a-9e6b-7735ada58880") : secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 07 10:37:16.417630 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:37:16.417519 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-online-tls podName:d9ef2f40-8495-437a-9e6b-7735ada58880 nodeName:}" failed. No retries permitted until 2026-09-07 10:37:16.917503091 +0000 UTC m=+1754.629745016 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-online-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" (UID: "d9ef2f40-8495-437a-9e6b-7735ada58880") : secret "feast-simple-feast-remote-setup-online-tls" not found Sep 07 10:37:16.417770 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.417716 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d9ef2f40-8495-437a-9e6b-7735ada58880-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jfgr7\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:16.418042 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.418022 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d9ef2f40-8495-437a-9e6b-7735ada58880-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jfgr7\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:16.419964 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.419943 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jfgr7\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:16.426366 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.426346 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8trr\" (UniqueName: \"kubernetes.io/projected/d9ef2f40-8495-437a-9e6b-7735ada58880-kube-api-access-v8trr\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jfgr7\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:16.921097 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.921054 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jfgr7\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:16.921097 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.921107 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jfgr7\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:16.923451 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.923426 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jfgr7\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:16.923612 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:16.923589 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jfgr7\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:17.152305 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:17.152267 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:17.427163 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:17.427134 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7"] Sep 07 10:37:17.429123 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:37:17.429096 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9ef2f40_8495_437a_9e6b_7735ada58880.slice/crio-0d43d91519c7006d3332006928ab19648e5199b1803f09eeb8a737b7c81277be WatchSource:0}: Error finding container 0d43d91519c7006d3332006928ab19648e5199b1803f09eeb8a737b7c81277be: Status 404 returned error can't find the container with id 0d43d91519c7006d3332006928ab19648e5199b1803f09eeb8a737b7c81277be Sep 07 10:37:17.970255 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:17.970225 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" event={"ID":"d9ef2f40-8495-437a-9e6b-7735ada58880","Type":"ContainerStarted","Data":"9c3c028924e181ddc38fd6e655f59836cfda4b7c1fe9523b2865d89ce8e2bed4"} Sep 07 10:37:17.970439 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:17.970267 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" event={"ID":"d9ef2f40-8495-437a-9e6b-7735ada58880","Type":"ContainerStarted","Data":"0d43d91519c7006d3332006928ab19648e5199b1803f09eeb8a737b7c81277be"} Sep 07 10:37:22.993350 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:22.993316 2575 generic.go:356] "Generic (PLEG): container finished" podID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerID="9c3c028924e181ddc38fd6e655f59836cfda4b7c1fe9523b2865d89ce8e2bed4" exitCode=0 Sep 07 10:37:22.993739 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:22.993355 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" event={"ID":"d9ef2f40-8495-437a-9e6b-7735ada58880","Type":"ContainerDied","Data":"9c3c028924e181ddc38fd6e655f59836cfda4b7c1fe9523b2865d89ce8e2bed4"} Sep 07 10:37:23.999515 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:23.999477 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" event={"ID":"d9ef2f40-8495-437a-9e6b-7735ada58880","Type":"ContainerStarted","Data":"f2c67ec6ab87972b97b565aad0fd93d2642ff358cbfd02b225762e09fddb1318"} Sep 07 10:37:46.095767 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:46.095729 2575 generic.go:356] "Generic (PLEG): container finished" podID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerID="f2c67ec6ab87972b97b565aad0fd93d2642ff358cbfd02b225762e09fddb1318" exitCode=0 Sep 07 10:37:46.095767 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:46.095771 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" event={"ID":"d9ef2f40-8495-437a-9e6b-7735ada58880","Type":"ContainerDied","Data":"f2c67ec6ab87972b97b565aad0fd93d2642ff358cbfd02b225762e09fddb1318"} Sep 07 10:37:47.104768 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:47.104725 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" event={"ID":"d9ef2f40-8495-437a-9e6b-7735ada58880","Type":"ContainerStarted","Data":"0e0cdfa4944e518e8acc49a70638e92bcf411b6c05116aed961b6bf5b56d1753"} Sep 07 10:37:47.105195 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:47.104814 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" event={"ID":"d9ef2f40-8495-437a-9e6b-7735ada58880","Type":"ContainerStarted","Data":"4b912382bdf63458f407a3a5ca45406541f61afc8352689a970af493ce365371"} Sep 07 10:37:47.105195 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:47.104842 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" event={"ID":"d9ef2f40-8495-437a-9e6b-7735ada58880","Type":"ContainerStarted","Data":"fb9f1512afbfd4252318633d34515cc2b55a00f4cfadd68a2ab720f3f418ee96"} Sep 07 10:37:47.129179 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:47.129125 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" podStartSLOduration=31.129110138 podStartE2EDuration="31.129110138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:37:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:37:47.127648833 +0000 UTC m=+1784.839890778" watchObservedRunningTime="2026-09-07 10:37:47.129110138 +0000 UTC m=+1784.841352083" Sep 07 10:37:47.152838 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:47.152780 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:47.153018 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:47.152864 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:47.153018 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:47.152884 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:47.153018 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:47.152899 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:47.153018 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:47.152914 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:47.153018 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:47.152929 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:47.155188 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:47.155154 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.31:8816: connect: connection refused" Sep 07 10:37:47.155296 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:47.155193 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.31:8443: connect: connection refused" Sep 07 10:37:47.155296 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:47.155222 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused" Sep 07 10:37:50.153562 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:50.153517 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.31:8816: connect: connection refused" Sep 07 10:37:50.154056 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:50.153513 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.31:8443: connect: connection refused" Sep 07 10:37:50.154056 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:50.153533 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused" Sep 07 10:37:53.153342 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:53.153303 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.31:8816: connect: connection refused" Sep 07 10:37:53.153342 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:53.153323 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused" Sep 07 10:37:53.153812 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:53.153303 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.31:8443: connect: connection refused" Sep 07 10:37:56.154088 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:56.153869 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:56.154088 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:56.154037 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:56.154654 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:56.154632 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:56.154779 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:56.154759 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:37:57.156948 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:57.156905 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 07 10:37:59.852670 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:37:59.852638 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:38:00.600207 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:38:00.600175 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:38:02.834388 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:38:02.834359 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-m6mmt_6aad73e9-87ee-49b0-8651-d3fd8390e4e4/console-operator/1.log" Sep 07 10:38:02.836530 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:38:02.836510 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-m6mmt_6aad73e9-87ee-49b0-8651-d3fd8390e4e4/console-operator/1.log" Sep 07 10:38:02.841935 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:38:02.841909 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/ovn-acl-logging/0.log" Sep 07 10:38:02.843977 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:38:02.843959 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/ovn-acl-logging/0.log" Sep 07 10:38:58.156954 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:38:58.156900 2575 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 07 10:39:02.416975 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:02.416933 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7"] Sep 07 10:39:02.417470 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:02.417212 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="offline" containerID="cri-o://4b912382bdf63458f407a3a5ca45406541f61afc8352689a970af493ce365371" gracePeriod=30 Sep 07 10:39:02.417470 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:02.417211 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="ui" containerID="cri-o://0e0cdfa4944e518e8acc49a70638e92bcf411b6c05116aed961b6bf5b56d1753" gracePeriod=30 Sep 07 10:39:02.417470 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:02.417296 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="online" containerID="cri-o://fb9f1512afbfd4252318633d34515cc2b55a00f4cfadd68a2ab720f3f418ee96" gracePeriod=30 Sep 07 10:39:02.562060 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:02.562024 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64"] Sep 07 10:39:02.562332 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:02.562285 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="registry" containerID="cri-o://53fc8af5d527cc1e4069c3ee81879000a33d663efec14d95276c130ceda01825" gracePeriod=30 Sep 07 10:39:02.562417 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:02.562359 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="online" containerID="cri-o://a2d70d7520be57d2da47dde4e7f2f7e1bb5c2b26998ce867797feb2530bf3fd0" gracePeriod=30 Sep 07 10:39:02.562417 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:02.562361 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="ui" containerID="cri-o://ed8ed32cbd2ef45888d1fe4c5fe8daf4fd2916e03c91548012c233f1a6ee64be" gracePeriod=30 Sep 07 10:39:02.562417 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:02.562382 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="offline" containerID="cri-o://6eb1848f2d66dd65175120dba66e1605e25adac474a4834812857e716a4c20c3" gracePeriod=30 Sep 07 10:39:03.431136 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:03.431106 2575 generic.go:356] "Generic (PLEG): container finished" podID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerID="ed8ed32cbd2ef45888d1fe4c5fe8daf4fd2916e03c91548012c233f1a6ee64be" exitCode=0 Sep 07 10:39:03.431524 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:03.431174 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" event={"ID":"d25f39bb-9452-4682-ad76-3fafb7cc75f2","Type":"ContainerDied","Data":"ed8ed32cbd2ef45888d1fe4c5fe8daf4fd2916e03c91548012c233f1a6ee64be"} Sep 07 10:39:03.433988 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:03.433966 2575 generic.go:356] "Generic (PLEG): container finished" podID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerID="0e0cdfa4944e518e8acc49a70638e92bcf411b6c05116aed961b6bf5b56d1753" exitCode=0 Sep 07 10:39:03.433988 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:03.433985 2575 generic.go:356] "Generic (PLEG): container finished" podID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerID="fb9f1512afbfd4252318633d34515cc2b55a00f4cfadd68a2ab720f3f418ee96" exitCode=0 Sep 07 10:39:03.434173 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:03.434015 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" event={"ID":"d9ef2f40-8495-437a-9e6b-7735ada58880","Type":"ContainerDied","Data":"0e0cdfa4944e518e8acc49a70638e92bcf411b6c05116aed961b6bf5b56d1753"} Sep 07 10:39:03.434173 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:03.434042 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" event={"ID":"d9ef2f40-8495-437a-9e6b-7735ada58880","Type":"ContainerDied","Data":"fb9f1512afbfd4252318633d34515cc2b55a00f4cfadd68a2ab720f3f418ee96"} Sep 07 10:39:03.711225 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:03.711125 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.30:8443: connect: connection refused" Sep 07 10:39:04.440436 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:04.440398 2575 generic.go:356] "Generic (PLEG): container finished" podID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerID="a2d70d7520be57d2da47dde4e7f2f7e1bb5c2b26998ce867797feb2530bf3fd0" exitCode=0 Sep 07 10:39:04.440851 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:04.440469 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" event={"ID":"d25f39bb-9452-4682-ad76-3fafb7cc75f2","Type":"ContainerDied","Data":"a2d70d7520be57d2da47dde4e7f2f7e1bb5c2b26998ce867797feb2530bf3fd0"} Sep 07 10:39:06.154939 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:06.154894 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.31:8443: connect: connection refused" Sep 07 10:39:09.853785 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:09.853733 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused" Sep 07 10:39:11.701790 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:11.701747 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused" Sep 07 10:39:13.716675 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:13.716616 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.30:8443: connect: connection refused" Sep 07 10:39:16.154717 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:16.154669 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.31:8443: connect: connection refused" Sep 07 10:39:19.854050 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:19.854011 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused" Sep 07 10:39:21.701477 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:21.701431 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused" Sep 07 10:39:23.710886 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:23.710838 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.30:8443: connect: connection refused" Sep 07 10:39:23.711294 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:23.710968 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:39:26.155205 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:26.155163 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.31:8443: connect: connection refused" Sep 07 10:39:26.155594 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:26.155280 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:39:29.853672 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:29.853624 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused" Sep 07 10:39:29.854245 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:29.853759 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:39:31.701224 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:31.701170 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused" Sep 07 10:39:31.701619 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:31.701295 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:39:32.557118 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:32.557085 2575 generic.go:356] "Generic (PLEG): container finished" podID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerID="4b912382bdf63458f407a3a5ca45406541f61afc8352689a970af493ce365371" exitCode=137 Sep 07 10:39:32.557298 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:32.557150 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" event={"ID":"d9ef2f40-8495-437a-9e6b-7735ada58880","Type":"ContainerDied","Data":"4b912382bdf63458f407a3a5ca45406541f61afc8352689a970af493ce365371"} Sep 07 10:39:33.067635 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.067603 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:39:33.178962 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.178917 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d9ef2f40-8495-437a-9e6b-7735ada58880-registry-tls\") pod \"d9ef2f40-8495-437a-9e6b-7735ada58880\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " Sep 07 10:39:33.179135 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.178985 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8trr\" (UniqueName: \"kubernetes.io/projected/d9ef2f40-8495-437a-9e6b-7735ada58880-kube-api-access-v8trr\") pod \"d9ef2f40-8495-437a-9e6b-7735ada58880\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " Sep 07 10:39:33.179135 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.179017 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d9ef2f40-8495-437a-9e6b-7735ada58880-feast-data\") pod \"d9ef2f40-8495-437a-9e6b-7735ada58880\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " Sep 07 10:39:33.179135 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.179096 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-ui-tls\") pod \"d9ef2f40-8495-437a-9e6b-7735ada58880\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " Sep 07 10:39:33.179135 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.179124 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-online-tls\") pod \"d9ef2f40-8495-437a-9e6b-7735ada58880\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " Sep 07 10:39:33.179357 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.179175 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-offline-tls\") pod \"d9ef2f40-8495-437a-9e6b-7735ada58880\" (UID: \"d9ef2f40-8495-437a-9e6b-7735ada58880\") " Sep 07 10:39:33.179415 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.179361 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9ef2f40-8495-437a-9e6b-7735ada58880-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d9ef2f40-8495-437a-9e6b-7735ada58880" (UID: "d9ef2f40-8495-437a-9e6b-7735ada58880"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 10:39:33.179644 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.179584 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9ef2f40-8495-437a-9e6b-7735ada58880-feast-data" (OuterVolumeSpecName: "feast-data") pod "d9ef2f40-8495-437a-9e6b-7735ada58880" (UID: "d9ef2f40-8495-437a-9e6b-7735ada58880"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 10:39:33.181877 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.181829 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "d9ef2f40-8495-437a-9e6b-7735ada58880" (UID: "d9ef2f40-8495-437a-9e6b-7735ada58880"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:39:33.181877 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.181858 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "d9ef2f40-8495-437a-9e6b-7735ada58880" (UID: "d9ef2f40-8495-437a-9e6b-7735ada58880"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:39:33.182011 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.181908 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-online-tls" (OuterVolumeSpecName: "online-tls") pod "d9ef2f40-8495-437a-9e6b-7735ada58880" (UID: "d9ef2f40-8495-437a-9e6b-7735ada58880"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:39:33.182011 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.181929 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9ef2f40-8495-437a-9e6b-7735ada58880-kube-api-access-v8trr" (OuterVolumeSpecName: "kube-api-access-v8trr") pod "d9ef2f40-8495-437a-9e6b-7735ada58880" (UID: "d9ef2f40-8495-437a-9e6b-7735ada58880"). InnerVolumeSpecName "kube-api-access-v8trr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:39:33.203368 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.203343 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:39:33.280278 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.280246 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-online-tls\") pod \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " Sep 07 10:39:33.280450 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.280302 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-offline-tls\") pod \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " Sep 07 10:39:33.280450 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.280327 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d25f39bb-9452-4682-ad76-3fafb7cc75f2-feast-data\") pod \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " Sep 07 10:39:33.280450 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.280381 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-ui-tls\") pod \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " Sep 07 10:39:33.280593 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.280512 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-registry-tls\") pod \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " Sep 07 10:39:33.280648 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.280618 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r64m9\" (UniqueName: \"kubernetes.io/projected/d25f39bb-9452-4682-ad76-3fafb7cc75f2-kube-api-access-r64m9\") pod \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\" (UID: \"d25f39bb-9452-4682-ad76-3fafb7cc75f2\") " Sep 07 10:39:33.280956 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.280911 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d9ef2f40-8495-437a-9e6b-7735ada58880-registry-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:39:33.280956 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.280937 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8trr\" (UniqueName: \"kubernetes.io/projected/d9ef2f40-8495-437a-9e6b-7735ada58880-kube-api-access-v8trr\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:39:33.280956 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.280950 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d9ef2f40-8495-437a-9e6b-7735ada58880-feast-data\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:39:33.281210 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.280951 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d25f39bb-9452-4682-ad76-3fafb7cc75f2-feast-data" (OuterVolumeSpecName: "feast-data") pod "d25f39bb-9452-4682-ad76-3fafb7cc75f2" (UID: "d25f39bb-9452-4682-ad76-3fafb7cc75f2"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 10:39:33.281210 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.280961 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-ui-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:39:33.281210 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.281019 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-online-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:39:33.281210 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.281035 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d9ef2f40-8495-437a-9e6b-7735ada58880-offline-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:39:33.282651 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.282624 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "d25f39bb-9452-4682-ad76-3fafb7cc75f2" (UID: "d25f39bb-9452-4682-ad76-3fafb7cc75f2"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:39:33.282766 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.282652 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "d25f39bb-9452-4682-ad76-3fafb7cc75f2" (UID: "d25f39bb-9452-4682-ad76-3fafb7cc75f2"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:39:33.282766 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.282666 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-online-tls" (OuterVolumeSpecName: "online-tls") pod "d25f39bb-9452-4682-ad76-3fafb7cc75f2" (UID: "d25f39bb-9452-4682-ad76-3fafb7cc75f2"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:39:33.282978 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.282957 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d25f39bb-9452-4682-ad76-3fafb7cc75f2-kube-api-access-r64m9" (OuterVolumeSpecName: "kube-api-access-r64m9") pod "d25f39bb-9452-4682-ad76-3fafb7cc75f2" (UID: "d25f39bb-9452-4682-ad76-3fafb7cc75f2"). InnerVolumeSpecName "kube-api-access-r64m9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:39:33.283044 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.282996 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d25f39bb-9452-4682-ad76-3fafb7cc75f2" (UID: "d25f39bb-9452-4682-ad76-3fafb7cc75f2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:39:33.381892 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.381847 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-ui-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:39:33.381892 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.381881 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-registry-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:39:33.381892 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.381893 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r64m9\" (UniqueName: \"kubernetes.io/projected/d25f39bb-9452-4682-ad76-3fafb7cc75f2-kube-api-access-r64m9\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:39:33.381892 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.381901 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-online-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:39:33.382172 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.381914 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d25f39bb-9452-4682-ad76-3fafb7cc75f2-offline-tls\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:39:33.382172 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.381925 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d25f39bb-9452-4682-ad76-3fafb7cc75f2-feast-data\") on node \"ip-10-0-141-85.ec2.internal\" DevicePath \"\"" Sep 07 10:39:33.564053 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.563961 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" Sep 07 10:39:33.564053 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.563971 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7" event={"ID":"d9ef2f40-8495-437a-9e6b-7735ada58880","Type":"ContainerDied","Data":"0d43d91519c7006d3332006928ab19648e5199b1803f09eeb8a737b7c81277be"} Sep 07 10:39:33.564053 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.564027 2575 scope.go:117] "RemoveContainer" containerID="0e0cdfa4944e518e8acc49a70638e92bcf411b6c05116aed961b6bf5b56d1753" Sep 07 10:39:33.567035 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.567000 2575 generic.go:356] "Generic (PLEG): container finished" podID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerID="6eb1848f2d66dd65175120dba66e1605e25adac474a4834812857e716a4c20c3" exitCode=137 Sep 07 10:39:33.567035 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.567027 2575 generic.go:356] "Generic (PLEG): container finished" podID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerID="53fc8af5d527cc1e4069c3ee81879000a33d663efec14d95276c130ceda01825" exitCode=137 Sep 07 10:39:33.567205 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.567071 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" Sep 07 10:39:33.567205 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.567086 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" event={"ID":"d25f39bb-9452-4682-ad76-3fafb7cc75f2","Type":"ContainerDied","Data":"6eb1848f2d66dd65175120dba66e1605e25adac474a4834812857e716a4c20c3"} Sep 07 10:39:33.567205 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.567137 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" event={"ID":"d25f39bb-9452-4682-ad76-3fafb7cc75f2","Type":"ContainerDied","Data":"53fc8af5d527cc1e4069c3ee81879000a33d663efec14d95276c130ceda01825"} Sep 07 10:39:33.567205 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.567156 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64" event={"ID":"d25f39bb-9452-4682-ad76-3fafb7cc75f2","Type":"ContainerDied","Data":"6c1e3e2812bcd2396fec707be0702a46d698e52a438d68d089429948e35a0342"} Sep 07 10:39:33.578993 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.578963 2575 scope.go:117] "RemoveContainer" containerID="4b912382bdf63458f407a3a5ca45406541f61afc8352689a970af493ce365371" Sep 07 10:39:33.587751 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.587730 2575 scope.go:117] "RemoveContainer" containerID="fb9f1512afbfd4252318633d34515cc2b55a00f4cfadd68a2ab720f3f418ee96" Sep 07 10:39:33.595738 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.595713 2575 scope.go:117] "RemoveContainer" containerID="f2c67ec6ab87972b97b565aad0fd93d2642ff358cbfd02b225762e09fddb1318" Sep 07 10:39:33.596284 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.596259 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64"] Sep 07 10:39:33.602614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.602588 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-j2h64"] Sep 07 10:39:33.604810 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.604767 2575 scope.go:117] "RemoveContainer" containerID="9c3c028924e181ddc38fd6e655f59836cfda4b7c1fe9523b2865d89ce8e2bed4" Sep 07 10:39:33.612601 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.612582 2575 scope.go:117] "RemoveContainer" containerID="ed8ed32cbd2ef45888d1fe4c5fe8daf4fd2916e03c91548012c233f1a6ee64be" Sep 07 10:39:33.612971 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.612951 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7"] Sep 07 10:39:33.616490 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.616469 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jfgr7"] Sep 07 10:39:33.620233 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.620216 2575 scope.go:117] "RemoveContainer" containerID="6eb1848f2d66dd65175120dba66e1605e25adac474a4834812857e716a4c20c3" Sep 07 10:39:33.627583 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.627564 2575 scope.go:117] "RemoveContainer" containerID="a2d70d7520be57d2da47dde4e7f2f7e1bb5c2b26998ce867797feb2530bf3fd0" Sep 07 10:39:33.634617 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.634602 2575 scope.go:117] "RemoveContainer" containerID="53fc8af5d527cc1e4069c3ee81879000a33d663efec14d95276c130ceda01825" Sep 07 10:39:33.641767 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.641749 2575 scope.go:117] "RemoveContainer" containerID="de1cefac3a957ce4838b91c439e8e3de4dcd9f1f9f625b1df9a4c68f4303bf6b" Sep 07 10:39:33.648568 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.648551 2575 scope.go:117] "RemoveContainer" containerID="ed26a6ffa278dc9cb2b843074ba07e08e39a3706dc98b1c45301b74269e6ab7f" Sep 07 10:39:33.655573 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.655555 2575 scope.go:117] "RemoveContainer" containerID="ed8ed32cbd2ef45888d1fe4c5fe8daf4fd2916e03c91548012c233f1a6ee64be" Sep 07 10:39:33.655898 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:39:33.655877 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed8ed32cbd2ef45888d1fe4c5fe8daf4fd2916e03c91548012c233f1a6ee64be\": container with ID starting with ed8ed32cbd2ef45888d1fe4c5fe8daf4fd2916e03c91548012c233f1a6ee64be not found: ID does not exist" containerID="ed8ed32cbd2ef45888d1fe4c5fe8daf4fd2916e03c91548012c233f1a6ee64be" Sep 07 10:39:33.655982 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.655912 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed8ed32cbd2ef45888d1fe4c5fe8daf4fd2916e03c91548012c233f1a6ee64be"} err="failed to get container status \"ed8ed32cbd2ef45888d1fe4c5fe8daf4fd2916e03c91548012c233f1a6ee64be\": rpc error: code = NotFound desc = could not find container \"ed8ed32cbd2ef45888d1fe4c5fe8daf4fd2916e03c91548012c233f1a6ee64be\": container with ID starting with ed8ed32cbd2ef45888d1fe4c5fe8daf4fd2916e03c91548012c233f1a6ee64be not found: ID does not exist" Sep 07 10:39:33.655982 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.655938 2575 scope.go:117] "RemoveContainer" containerID="6eb1848f2d66dd65175120dba66e1605e25adac474a4834812857e716a4c20c3" Sep 07 10:39:33.656240 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:39:33.656219 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6eb1848f2d66dd65175120dba66e1605e25adac474a4834812857e716a4c20c3\": container with ID starting with 6eb1848f2d66dd65175120dba66e1605e25adac474a4834812857e716a4c20c3 not found: ID does not exist" containerID="6eb1848f2d66dd65175120dba66e1605e25adac474a4834812857e716a4c20c3" Sep 07 10:39:33.656281 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.656247 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eb1848f2d66dd65175120dba66e1605e25adac474a4834812857e716a4c20c3"} err="failed to get container status \"6eb1848f2d66dd65175120dba66e1605e25adac474a4834812857e716a4c20c3\": rpc error: code = NotFound desc = could not find container \"6eb1848f2d66dd65175120dba66e1605e25adac474a4834812857e716a4c20c3\": container with ID starting with 6eb1848f2d66dd65175120dba66e1605e25adac474a4834812857e716a4c20c3 not found: ID does not exist" Sep 07 10:39:33.656281 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.656263 2575 scope.go:117] "RemoveContainer" containerID="a2d70d7520be57d2da47dde4e7f2f7e1bb5c2b26998ce867797feb2530bf3fd0" Sep 07 10:39:33.656511 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:39:33.656491 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2d70d7520be57d2da47dde4e7f2f7e1bb5c2b26998ce867797feb2530bf3fd0\": container with ID starting with a2d70d7520be57d2da47dde4e7f2f7e1bb5c2b26998ce867797feb2530bf3fd0 not found: ID does not exist" containerID="a2d70d7520be57d2da47dde4e7f2f7e1bb5c2b26998ce867797feb2530bf3fd0" Sep 07 10:39:33.656570 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.656519 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2d70d7520be57d2da47dde4e7f2f7e1bb5c2b26998ce867797feb2530bf3fd0"} err="failed to get container status \"a2d70d7520be57d2da47dde4e7f2f7e1bb5c2b26998ce867797feb2530bf3fd0\": rpc error: code = NotFound desc = could not find container \"a2d70d7520be57d2da47dde4e7f2f7e1bb5c2b26998ce867797feb2530bf3fd0\": container with ID starting with a2d70d7520be57d2da47dde4e7f2f7e1bb5c2b26998ce867797feb2530bf3fd0 not found: ID does not exist" Sep 07 10:39:33.656570 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.656539 2575 scope.go:117] "RemoveContainer" containerID="53fc8af5d527cc1e4069c3ee81879000a33d663efec14d95276c130ceda01825" Sep 07 10:39:33.656755 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:39:33.656739 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53fc8af5d527cc1e4069c3ee81879000a33d663efec14d95276c130ceda01825\": container with ID starting with 53fc8af5d527cc1e4069c3ee81879000a33d663efec14d95276c130ceda01825 not found: ID does not exist" containerID="53fc8af5d527cc1e4069c3ee81879000a33d663efec14d95276c130ceda01825" Sep 07 10:39:33.656819 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.656759 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53fc8af5d527cc1e4069c3ee81879000a33d663efec14d95276c130ceda01825"} err="failed to get container status \"53fc8af5d527cc1e4069c3ee81879000a33d663efec14d95276c130ceda01825\": rpc error: code = NotFound desc = could not find container \"53fc8af5d527cc1e4069c3ee81879000a33d663efec14d95276c130ceda01825\": container with ID starting with 53fc8af5d527cc1e4069c3ee81879000a33d663efec14d95276c130ceda01825 not found: ID does not exist" Sep 07 10:39:33.656819 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.656774 2575 scope.go:117] "RemoveContainer" containerID="de1cefac3a957ce4838b91c439e8e3de4dcd9f1f9f625b1df9a4c68f4303bf6b" Sep 07 10:39:33.657072 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:39:33.657055 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de1cefac3a957ce4838b91c439e8e3de4dcd9f1f9f625b1df9a4c68f4303bf6b\": container with ID starting with de1cefac3a957ce4838b91c439e8e3de4dcd9f1f9f625b1df9a4c68f4303bf6b not found: ID does not exist" containerID="de1cefac3a957ce4838b91c439e8e3de4dcd9f1f9f625b1df9a4c68f4303bf6b" Sep 07 10:39:33.657139 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.657078 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de1cefac3a957ce4838b91c439e8e3de4dcd9f1f9f625b1df9a4c68f4303bf6b"} err="failed to get container status \"de1cefac3a957ce4838b91c439e8e3de4dcd9f1f9f625b1df9a4c68f4303bf6b\": rpc error: code = NotFound desc = could not find container \"de1cefac3a957ce4838b91c439e8e3de4dcd9f1f9f625b1df9a4c68f4303bf6b\": container with ID starting with de1cefac3a957ce4838b91c439e8e3de4dcd9f1f9f625b1df9a4c68f4303bf6b not found: ID does not exist" Sep 07 10:39:33.657139 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.657098 2575 scope.go:117] "RemoveContainer" containerID="ed26a6ffa278dc9cb2b843074ba07e08e39a3706dc98b1c45301b74269e6ab7f" Sep 07 10:39:33.657377 ip-10-0-141-85 kubenswrapper[2575]: E0907 10:39:33.657359 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed26a6ffa278dc9cb2b843074ba07e08e39a3706dc98b1c45301b74269e6ab7f\": container with ID starting with ed26a6ffa278dc9cb2b843074ba07e08e39a3706dc98b1c45301b74269e6ab7f not found: ID does not exist" containerID="ed26a6ffa278dc9cb2b843074ba07e08e39a3706dc98b1c45301b74269e6ab7f" Sep 07 10:39:33.657421 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.657387 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed26a6ffa278dc9cb2b843074ba07e08e39a3706dc98b1c45301b74269e6ab7f"} err="failed to get container status \"ed26a6ffa278dc9cb2b843074ba07e08e39a3706dc98b1c45301b74269e6ab7f\": rpc error: code = NotFound desc = could not find container \"ed26a6ffa278dc9cb2b843074ba07e08e39a3706dc98b1c45301b74269e6ab7f\": container with ID starting with ed26a6ffa278dc9cb2b843074ba07e08e39a3706dc98b1c45301b74269e6ab7f not found: ID does not exist" Sep 07 10:39:33.657421 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.657402 2575 scope.go:117] "RemoveContainer" containerID="ed8ed32cbd2ef45888d1fe4c5fe8daf4fd2916e03c91548012c233f1a6ee64be" Sep 07 10:39:33.657615 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.657598 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed8ed32cbd2ef45888d1fe4c5fe8daf4fd2916e03c91548012c233f1a6ee64be"} err="failed to get container status \"ed8ed32cbd2ef45888d1fe4c5fe8daf4fd2916e03c91548012c233f1a6ee64be\": rpc error: code = NotFound desc = could not find container \"ed8ed32cbd2ef45888d1fe4c5fe8daf4fd2916e03c91548012c233f1a6ee64be\": container with ID starting with ed8ed32cbd2ef45888d1fe4c5fe8daf4fd2916e03c91548012c233f1a6ee64be not found: ID does not exist" Sep 07 10:39:33.657683 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.657616 2575 scope.go:117] "RemoveContainer" containerID="6eb1848f2d66dd65175120dba66e1605e25adac474a4834812857e716a4c20c3" Sep 07 10:39:33.657860 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.657844 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eb1848f2d66dd65175120dba66e1605e25adac474a4834812857e716a4c20c3"} err="failed to get container status \"6eb1848f2d66dd65175120dba66e1605e25adac474a4834812857e716a4c20c3\": rpc error: code = NotFound desc = could not find container \"6eb1848f2d66dd65175120dba66e1605e25adac474a4834812857e716a4c20c3\": container with ID starting with 6eb1848f2d66dd65175120dba66e1605e25adac474a4834812857e716a4c20c3 not found: ID does not exist" Sep 07 10:39:33.657913 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.657862 2575 scope.go:117] "RemoveContainer" containerID="a2d70d7520be57d2da47dde4e7f2f7e1bb5c2b26998ce867797feb2530bf3fd0" Sep 07 10:39:33.658069 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.658052 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2d70d7520be57d2da47dde4e7f2f7e1bb5c2b26998ce867797feb2530bf3fd0"} err="failed to get container status \"a2d70d7520be57d2da47dde4e7f2f7e1bb5c2b26998ce867797feb2530bf3fd0\": rpc error: code = NotFound desc = could not find container \"a2d70d7520be57d2da47dde4e7f2f7e1bb5c2b26998ce867797feb2530bf3fd0\": container with ID starting with a2d70d7520be57d2da47dde4e7f2f7e1bb5c2b26998ce867797feb2530bf3fd0 not found: ID does not exist" Sep 07 10:39:33.658138 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.658070 2575 scope.go:117] "RemoveContainer" containerID="53fc8af5d527cc1e4069c3ee81879000a33d663efec14d95276c130ceda01825" Sep 07 10:39:33.658274 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.658259 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53fc8af5d527cc1e4069c3ee81879000a33d663efec14d95276c130ceda01825"} err="failed to get container status \"53fc8af5d527cc1e4069c3ee81879000a33d663efec14d95276c130ceda01825\": rpc error: code = NotFound desc = could not find container \"53fc8af5d527cc1e4069c3ee81879000a33d663efec14d95276c130ceda01825\": container with ID starting with 53fc8af5d527cc1e4069c3ee81879000a33d663efec14d95276c130ceda01825 not found: ID does not exist" Sep 07 10:39:33.658316 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.658274 2575 scope.go:117] "RemoveContainer" containerID="de1cefac3a957ce4838b91c439e8e3de4dcd9f1f9f625b1df9a4c68f4303bf6b" Sep 07 10:39:33.658482 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.658464 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de1cefac3a957ce4838b91c439e8e3de4dcd9f1f9f625b1df9a4c68f4303bf6b"} err="failed to get container status \"de1cefac3a957ce4838b91c439e8e3de4dcd9f1f9f625b1df9a4c68f4303bf6b\": rpc error: code = NotFound desc = could not find container \"de1cefac3a957ce4838b91c439e8e3de4dcd9f1f9f625b1df9a4c68f4303bf6b\": container with ID starting with de1cefac3a957ce4838b91c439e8e3de4dcd9f1f9f625b1df9a4c68f4303bf6b not found: ID does not exist" Sep 07 10:39:33.658482 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.658480 2575 scope.go:117] "RemoveContainer" containerID="ed26a6ffa278dc9cb2b843074ba07e08e39a3706dc98b1c45301b74269e6ab7f" Sep 07 10:39:33.658654 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:33.658640 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed26a6ffa278dc9cb2b843074ba07e08e39a3706dc98b1c45301b74269e6ab7f"} err="failed to get container status \"ed26a6ffa278dc9cb2b843074ba07e08e39a3706dc98b1c45301b74269e6ab7f\": rpc error: code = NotFound desc = could not find container \"ed26a6ffa278dc9cb2b843074ba07e08e39a3706dc98b1c45301b74269e6ab7f\": container with ID starting with ed26a6ffa278dc9cb2b843074ba07e08e39a3706dc98b1c45301b74269e6ab7f not found: ID does not exist" Sep 07 10:39:34.830870 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:34.830838 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" path="/var/lib/kubelet/pods/d25f39bb-9452-4682-ad76-3fafb7cc75f2/volumes" Sep 07 10:39:34.831471 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:39:34.831456 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" path="/var/lib/kubelet/pods/d9ef2f40-8495-437a-9e6b-7735ada58880/volumes" Sep 07 10:40:09.524357 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524316 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4plbr/must-gather-j775k"] Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524622 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="feast-init" Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524632 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="feast-init" Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524643 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="ui" Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524648 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="ui" Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524660 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="online" Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524666 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="online" Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524673 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="feast-apply" Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524678 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="feast-apply" Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524684 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="ui" Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524689 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="ui" Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524699 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="feast-init" Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524705 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="feast-init" Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524710 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="offline" Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524714 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="offline" Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524720 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="online" Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524726 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="online" Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524733 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="offline" Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524738 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="offline" Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524752 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="registry" Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524757 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="registry" Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524766 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="feast-apply" Sep 07 10:40:09.524782 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524771 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="feast-apply" Sep 07 10:40:09.525486 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524829 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="online" Sep 07 10:40:09.525486 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524838 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="registry" Sep 07 10:40:09.525486 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524844 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="online" Sep 07 10:40:09.525486 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524849 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="ui" Sep 07 10:40:09.525486 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524855 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="ui" Sep 07 10:40:09.525486 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524860 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9ef2f40-8495-437a-9e6b-7735ada58880" containerName="offline" Sep 07 10:40:09.525486 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.524867 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d25f39bb-9452-4682-ad76-3fafb7cc75f2" containerName="offline" Sep 07 10:40:09.528623 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.528605 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4plbr/must-gather-j775k" Sep 07 10:40:09.530868 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.530843 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4plbr\"/\"default-dockercfg-5l59n\"" Sep 07 10:40:09.531012 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.530921 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4plbr\"/\"openshift-service-ca.crt\"" Sep 07 10:40:09.531199 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.531177 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4plbr\"/\"kube-root-ca.crt\"" Sep 07 10:40:09.536228 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.536205 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4plbr/must-gather-j775k"] Sep 07 10:40:09.543696 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.543669 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ae8b74fd-eca9-47fc-b963-68befe29844c-must-gather-output\") pod \"must-gather-j775k\" (UID: \"ae8b74fd-eca9-47fc-b963-68befe29844c\") " pod="openshift-must-gather-4plbr/must-gather-j775k" Sep 07 10:40:09.543856 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.543711 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxshw\" (UniqueName: \"kubernetes.io/projected/ae8b74fd-eca9-47fc-b963-68befe29844c-kube-api-access-zxshw\") pod \"must-gather-j775k\" (UID: \"ae8b74fd-eca9-47fc-b963-68befe29844c\") " pod="openshift-must-gather-4plbr/must-gather-j775k" Sep 07 10:40:09.644440 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.644406 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ae8b74fd-eca9-47fc-b963-68befe29844c-must-gather-output\") pod \"must-gather-j775k\" (UID: \"ae8b74fd-eca9-47fc-b963-68befe29844c\") " pod="openshift-must-gather-4plbr/must-gather-j775k" Sep 07 10:40:09.644440 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.644454 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxshw\" (UniqueName: \"kubernetes.io/projected/ae8b74fd-eca9-47fc-b963-68befe29844c-kube-api-access-zxshw\") pod \"must-gather-j775k\" (UID: \"ae8b74fd-eca9-47fc-b963-68befe29844c\") " pod="openshift-must-gather-4plbr/must-gather-j775k" Sep 07 10:40:09.644856 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.644833 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ae8b74fd-eca9-47fc-b963-68befe29844c-must-gather-output\") pod \"must-gather-j775k\" (UID: \"ae8b74fd-eca9-47fc-b963-68befe29844c\") " pod="openshift-must-gather-4plbr/must-gather-j775k" Sep 07 10:40:09.655754 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.655719 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxshw\" (UniqueName: \"kubernetes.io/projected/ae8b74fd-eca9-47fc-b963-68befe29844c-kube-api-access-zxshw\") pod \"must-gather-j775k\" (UID: \"ae8b74fd-eca9-47fc-b963-68befe29844c\") " pod="openshift-must-gather-4plbr/must-gather-j775k" Sep 07 10:40:09.838216 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:09.838179 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4plbr/must-gather-j775k" Sep 07 10:40:10.105836 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:10.105778 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4plbr/must-gather-j775k"] Sep 07 10:40:10.108359 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:40:10.108330 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae8b74fd_eca9_47fc_b963_68befe29844c.slice/crio-1a13b02e9844e9279f04770a50bf9c5277a620bb2e7ce8a0463890fd83ca9304 WatchSource:0}: Error finding container 1a13b02e9844e9279f04770a50bf9c5277a620bb2e7ce8a0463890fd83ca9304: Status 404 returned error can't find the container with id 1a13b02e9844e9279f04770a50bf9c5277a620bb2e7ce8a0463890fd83ca9304 Sep 07 10:40:10.110426 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:10.110403 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 10:40:10.713328 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:10.713291 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4plbr/must-gather-j775k" event={"ID":"ae8b74fd-eca9-47fc-b963-68befe29844c","Type":"ContainerStarted","Data":"1a13b02e9844e9279f04770a50bf9c5277a620bb2e7ce8a0463890fd83ca9304"} Sep 07 10:40:11.720458 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:11.720415 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4plbr/must-gather-j775k" event={"ID":"ae8b74fd-eca9-47fc-b963-68befe29844c","Type":"ContainerStarted","Data":"55043d66f2ccb8f7e8e94c31c09fe56f809e0b09ba0ae1bece12f75fe76cdd2d"} Sep 07 10:40:11.720874 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:11.720469 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4plbr/must-gather-j775k" event={"ID":"ae8b74fd-eca9-47fc-b963-68befe29844c","Type":"ContainerStarted","Data":"592e54e51b4edffaee7ef7cd26ad59a67169254923320c9b1406b8a32046f48f"} Sep 07 10:40:11.736699 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:11.736633 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4plbr/must-gather-j775k" podStartSLOduration=1.7959642630000001 podStartE2EDuration="2.736613384s" totalImagesPullingTime="940.649121ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:40:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:40:11.734408635 +0000 UTC m=+1929.446650581" watchObservedRunningTime="2026-09-07 10:40:11.736613384 +0000 UTC m=+1929.448855330" Sep 07 10:40:12.347887 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:12.347855 2575 ???:1] "http: TLS handshake error from 10.0.131.125:34382: EOF" Sep 07 10:40:12.351306 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:12.351275 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6v64l_cff88048-74be-46c2-a8ac-113c5fc3a85d/global-pull-secret-syncer/0.log" Sep 07 10:40:12.542210 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:12.542164 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9tdjb_cb028709-a2f7-4338-8699-120a4879ffaf/konnectivity-agent/0.log" Sep 07 10:40:12.617295 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:12.617213 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-85.ec2.internal_f155589bfc7e2b2766af1b57a8ea8129/haproxy/0.log" Sep 07 10:40:16.484433 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:16.484396 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-dx9lz_502e0127-90b6-4388-860b-5dd7979db869/cluster-monitoring-operator/0.log" Sep 07 10:40:16.595399 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:16.595312 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7f5864f4f-6d2qt_ff48b6c0-a453-4d1d-8851-ded97f4ca8ed/metrics-server/0.log" Sep 07 10:40:16.622530 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:16.622498 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-kfqn5_b37db823-b8fb-448c-9556-a28cd0896239/monitoring-plugin/0.log" Sep 07 10:40:16.661394 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:16.661365 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6ctzx_fa50d828-c196-4e48-9742-8f6aaea0b9df/node-exporter/0.log" Sep 07 10:40:16.683104 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:16.683076 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6ctzx_fa50d828-c196-4e48-9742-8f6aaea0b9df/kube-rbac-proxy/0.log" Sep 07 10:40:16.707588 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:16.707563 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6ctzx_fa50d828-c196-4e48-9742-8f6aaea0b9df/init-textfile/0.log" Sep 07 10:40:16.891951 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:16.891867 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-975lp_8ce40ab0-a188-4403-b608-575dcef41060/kube-rbac-proxy-main/0.log" Sep 07 10:40:16.913643 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:16.913571 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-975lp_8ce40ab0-a188-4403-b608-575dcef41060/kube-rbac-proxy-self/0.log" Sep 07 10:40:16.936611 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:16.936550 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-975lp_8ce40ab0-a188-4403-b608-575dcef41060/openshift-state-metrics/0.log" Sep 07 10:40:17.217497 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:17.217409 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-djr8f_4d198470-13aa-4025-a865-7d1f9f8ca2cc/prometheus-operator/0.log" Sep 07 10:40:17.236557 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:17.236454 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-djr8f_4d198470-13aa-4025-a865-7d1f9f8ca2cc/kube-rbac-proxy/0.log" Sep 07 10:40:17.381783 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:17.381748 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b4454c4f5-9svvk_6268e89a-3997-4ad1-bcbb-ac4496d1997b/thanos-query/0.log" Sep 07 10:40:17.405655 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:17.405624 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b4454c4f5-9svvk_6268e89a-3997-4ad1-bcbb-ac4496d1997b/kube-rbac-proxy-web/0.log" Sep 07 10:40:17.427621 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:17.427585 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b4454c4f5-9svvk_6268e89a-3997-4ad1-bcbb-ac4496d1997b/kube-rbac-proxy/0.log" Sep 07 10:40:17.452847 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:17.452816 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b4454c4f5-9svvk_6268e89a-3997-4ad1-bcbb-ac4496d1997b/prom-label-proxy/0.log" Sep 07 10:40:17.477001 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:17.476919 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b4454c4f5-9svvk_6268e89a-3997-4ad1-bcbb-ac4496d1997b/kube-rbac-proxy-rules/0.log" Sep 07 10:40:17.505699 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:17.505669 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b4454c4f5-9svvk_6268e89a-3997-4ad1-bcbb-ac4496d1997b/kube-rbac-proxy-metrics/0.log" Sep 07 10:40:19.345072 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:19.345043 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-m6mmt_6aad73e9-87ee-49b0-8651-d3fd8390e4e4/console-operator/1.log" Sep 07 10:40:19.354326 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:19.354257 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-m6mmt_6aad73e9-87ee-49b0-8651-d3fd8390e4e4/console-operator/2.log" Sep 07 10:40:19.926907 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:19.926872 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r"] Sep 07 10:40:19.932729 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:19.932701 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" Sep 07 10:40:19.939567 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:19.939489 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r"] Sep 07 10:40:20.035375 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:20.035345 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/01c285d5-30a7-497d-9c45-13e7fc0fd5ce-podres\") pod \"perf-node-gather-daemonset-f4l7r\" (UID: \"01c285d5-30a7-497d-9c45-13e7fc0fd5ce\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" Sep 07 10:40:20.035375 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:20.035384 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/01c285d5-30a7-497d-9c45-13e7fc0fd5ce-sys\") pod \"perf-node-gather-daemonset-f4l7r\" (UID: \"01c285d5-30a7-497d-9c45-13e7fc0fd5ce\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" Sep 07 10:40:20.035598 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:20.035428 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/01c285d5-30a7-497d-9c45-13e7fc0fd5ce-proc\") pod \"perf-node-gather-daemonset-f4l7r\" (UID: \"01c285d5-30a7-497d-9c45-13e7fc0fd5ce\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" Sep 07 10:40:20.035598 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:20.035453 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bhlk\" (UniqueName: \"kubernetes.io/projected/01c285d5-30a7-497d-9c45-13e7fc0fd5ce-kube-api-access-7bhlk\") pod \"perf-node-gather-daemonset-f4l7r\" (UID: \"01c285d5-30a7-497d-9c45-13e7fc0fd5ce\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" Sep 07 10:40:20.035598 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:20.035496 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/01c285d5-30a7-497d-9c45-13e7fc0fd5ce-lib-modules\") pod \"perf-node-gather-daemonset-f4l7r\" (UID: \"01c285d5-30a7-497d-9c45-13e7fc0fd5ce\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" Sep 07 10:40:20.136596 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:20.136560 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/01c285d5-30a7-497d-9c45-13e7fc0fd5ce-podres\") pod \"perf-node-gather-daemonset-f4l7r\" (UID: \"01c285d5-30a7-497d-9c45-13e7fc0fd5ce\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" Sep 07 10:40:20.136596 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:20.136602 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/01c285d5-30a7-497d-9c45-13e7fc0fd5ce-sys\") pod \"perf-node-gather-daemonset-f4l7r\" (UID: \"01c285d5-30a7-497d-9c45-13e7fc0fd5ce\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" Sep 07 10:40:20.136906 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:20.136632 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/01c285d5-30a7-497d-9c45-13e7fc0fd5ce-proc\") pod \"perf-node-gather-daemonset-f4l7r\" (UID: \"01c285d5-30a7-497d-9c45-13e7fc0fd5ce\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" Sep 07 10:40:20.136906 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:20.136698 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/01c285d5-30a7-497d-9c45-13e7fc0fd5ce-proc\") pod \"perf-node-gather-daemonset-f4l7r\" (UID: \"01c285d5-30a7-497d-9c45-13e7fc0fd5ce\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" Sep 07 10:40:20.136906 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:20.136747 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bhlk\" (UniqueName: \"kubernetes.io/projected/01c285d5-30a7-497d-9c45-13e7fc0fd5ce-kube-api-access-7bhlk\") pod \"perf-node-gather-daemonset-f4l7r\" (UID: \"01c285d5-30a7-497d-9c45-13e7fc0fd5ce\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" Sep 07 10:40:20.136906 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:20.136760 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/01c285d5-30a7-497d-9c45-13e7fc0fd5ce-sys\") pod \"perf-node-gather-daemonset-f4l7r\" (UID: \"01c285d5-30a7-497d-9c45-13e7fc0fd5ce\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" Sep 07 10:40:20.136906 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:20.136784 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/01c285d5-30a7-497d-9c45-13e7fc0fd5ce-podres\") pod \"perf-node-gather-daemonset-f4l7r\" (UID: \"01c285d5-30a7-497d-9c45-13e7fc0fd5ce\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" Sep 07 10:40:20.136906 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:20.136849 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/01c285d5-30a7-497d-9c45-13e7fc0fd5ce-lib-modules\") pod \"perf-node-gather-daemonset-f4l7r\" (UID: \"01c285d5-30a7-497d-9c45-13e7fc0fd5ce\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" Sep 07 10:40:20.137182 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:20.136991 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/01c285d5-30a7-497d-9c45-13e7fc0fd5ce-lib-modules\") pod \"perf-node-gather-daemonset-f4l7r\" (UID: \"01c285d5-30a7-497d-9c45-13e7fc0fd5ce\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" Sep 07 10:40:20.146235 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:20.145555 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bhlk\" (UniqueName: \"kubernetes.io/projected/01c285d5-30a7-497d-9c45-13e7fc0fd5ce-kube-api-access-7bhlk\") pod \"perf-node-gather-daemonset-f4l7r\" (UID: \"01c285d5-30a7-497d-9c45-13e7fc0fd5ce\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" Sep 07 10:40:20.251168 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:20.250762 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" Sep 07 10:40:20.535900 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:20.535783 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r"] Sep 07 10:40:20.540182 ip-10-0-141-85 kubenswrapper[2575]: W0907 10:40:20.540153 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod01c285d5_30a7_497d_9c45_13e7fc0fd5ce.slice/crio-a5a729f4f3850f7c117674ef74399251be25f484fff6f31392d9be71fe992d8b WatchSource:0}: Error finding container a5a729f4f3850f7c117674ef74399251be25f484fff6f31392d9be71fe992d8b: Status 404 returned error can't find the container with id a5a729f4f3850f7c117674ef74399251be25f484fff6f31392d9be71fe992d8b Sep 07 10:40:20.775366 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:20.775317 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" event={"ID":"01c285d5-30a7-497d-9c45-13e7fc0fd5ce","Type":"ContainerStarted","Data":"640a891baa99b180c4b22d10a62ed18c7a60ab2fa7757dfac8d01649e41bd9ea"} Sep 07 10:40:20.775583 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:20.775375 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" event={"ID":"01c285d5-30a7-497d-9c45-13e7fc0fd5ce","Type":"ContainerStarted","Data":"a5a729f4f3850f7c117674ef74399251be25f484fff6f31392d9be71fe992d8b"} Sep 07 10:40:20.775583 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:20.775419 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" Sep 07 10:40:20.792689 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:20.792550 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" podStartSLOduration=1.792526818 podStartE2EDuration="1.792526818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:40:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:40:20.791822742 +0000 UTC m=+1938.504064688" watchObservedRunningTime="2026-09-07 10:40:20.792526818 +0000 UTC m=+1938.504768765" Sep 07 10:40:21.056761 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:21.056675 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nc7j5_b07b7355-ed97-4a04-958b-c675ec096d60/dns/0.log" Sep 07 10:40:21.115205 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:21.115173 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nc7j5_b07b7355-ed97-4a04-958b-c675ec096d60/kube-rbac-proxy/0.log" Sep 07 10:40:21.231734 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:21.231702 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w5w2p_19524835-4647-4786-bc0f-c222dd09c075/dns-node-resolver/0.log" Sep 07 10:40:21.592813 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:21.592758 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-86f7794bb5-pxqrg_a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc/registry/0.log" Sep 07 10:40:21.607375 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:21.607340 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-86f7794bb5-pxqrg_a0a13746-7fc9-4fdb-ad9a-17e2020bd2bc/registry/1.log" Sep 07 10:40:21.650978 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:21.650946 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t6ztt_a6a30db9-93e6-41b1-a0e6-28e5088df925/node-ca/0.log" Sep 07 10:40:22.325059 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:22.325029 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c558849d-hzpf7_c3c51a39-edc3-44e7-a0ed-843ac7ec5059/router/0.log" Sep 07 10:40:22.636645 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:22.636568 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jfq75_1352b671-2ab8-4c45-8f10-f82960e1ff1c/serve-healthcheck-canary/0.log" Sep 07 10:40:23.048147 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:23.048066 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gv4pf_51adbf04-234f-4a6c-8aec-c65483741ae6/kube-rbac-proxy/0.log" Sep 07 10:40:23.067898 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:23.067872 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gv4pf_51adbf04-234f-4a6c-8aec-c65483741ae6/exporter/0.log" Sep 07 10:40:23.088302 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:23.088274 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gv4pf_51adbf04-234f-4a6c-8aec-c65483741ae6/extractor/0.log" Sep 07 10:40:26.790334 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:26.790305 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-f4l7r" Sep 07 10:40:27.414086 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:27.414056 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-8bl4l_70c229fe-b5da-413c-b231-625155198200/kube-storage-version-migrator-operator/1.log" Sep 07 10:40:27.415721 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:27.415697 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-8bl4l_70c229fe-b5da-413c-b231-625155198200/kube-storage-version-migrator-operator/0.log" Sep 07 10:40:28.261966 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:28.261914 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6q6w4_448a5179-febc-460a-ab61-f7810c8bf09b/kube-multus/0.log" Sep 07 10:40:28.586964 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:28.586926 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nzg2l_9aeb0af4-1773-4b88-b17f-d1d4d6490101/kube-multus-additional-cni-plugins/0.log" Sep 07 10:40:28.607945 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:28.607910 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nzg2l_9aeb0af4-1773-4b88-b17f-d1d4d6490101/egress-router-binary-copy/0.log" Sep 07 10:40:28.628375 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:28.628341 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nzg2l_9aeb0af4-1773-4b88-b17f-d1d4d6490101/cni-plugins/0.log" Sep 07 10:40:28.648042 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:28.648001 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nzg2l_9aeb0af4-1773-4b88-b17f-d1d4d6490101/bond-cni-plugin/0.log" Sep 07 10:40:28.669246 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:28.669218 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nzg2l_9aeb0af4-1773-4b88-b17f-d1d4d6490101/routeoverride-cni/0.log" Sep 07 10:40:28.691998 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:28.691970 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nzg2l_9aeb0af4-1773-4b88-b17f-d1d4d6490101/whereabouts-cni-bincopy/0.log" Sep 07 10:40:28.714292 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:28.714258 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nzg2l_9aeb0af4-1773-4b88-b17f-d1d4d6490101/whereabouts-cni/0.log" Sep 07 10:40:28.799778 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:28.799745 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4wfhx_d11a5179-d3c5-48d5-a7c6-a7c367fd2584/network-metrics-daemon/0.log" Sep 07 10:40:28.819418 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:28.819386 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4wfhx_d11a5179-d3c5-48d5-a7c6-a7c367fd2584/kube-rbac-proxy/0.log" Sep 07 10:40:29.885152 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:29.885122 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/ovn-controller/0.log" Sep 07 10:40:29.902010 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:29.901976 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/ovn-acl-logging/1.log" Sep 07 10:40:29.902199 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:29.902113 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/ovn-acl-logging/0.log" Sep 07 10:40:29.921503 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:29.921474 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/kube-rbac-proxy-node/0.log" Sep 07 10:40:29.939989 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:29.939962 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/kube-rbac-proxy-ovn-metrics/0.log" Sep 07 10:40:29.955671 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:29.955646 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/northd/0.log" Sep 07 10:40:29.974070 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:29.974041 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/nbdb/0.log" Sep 07 10:40:29.994614 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:29.994581 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/sbdb/0.log" Sep 07 10:40:30.052737 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:30.052705 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-f6mnj_f245ed6c-716b-4e7f-89bf-2db47ba2e83c/ovnkube-controller/0.log" Sep 07 10:40:31.247982 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:31.247947 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-km29h_d4e7d380-df30-4ee5-9059-6fd7fb6e82b7/network-check-target-container/0.log" Sep 07 10:40:32.069242 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:32.069211 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-qss98_adcc0041-9a3b-4ff3-8a9a-25741c8e9baa/iptables-alerter/0.log" Sep 07 10:40:32.656188 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:32.656156 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-4wg78_a1a1ee72-afe3-4901-8a66-95c456e21ada/tuned/0.log" Sep 07 10:40:35.992261 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:35.992231 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-fr4jh_18c4a942-f1db-473e-aab6-d35c7bde40ae/csi-driver/0.log" Sep 07 10:40:36.010102 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:36.010074 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-fr4jh_18c4a942-f1db-473e-aab6-d35c7bde40ae/csi-node-driver-registrar/0.log" Sep 07 10:40:36.028165 ip-10-0-141-85 kubenswrapper[2575]: I0907 10:40:36.028134 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-fr4jh_18c4a942-f1db-473e-aab6-d35c7bde40ae/csi-liveness-probe/0.log"