Sep 17 11:36:46.126049 ip-10-0-140-196 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 17 11:36:46.126064 ip-10-0-140-196 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 17 11:36:46.126074 ip-10-0-140-196 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 17 11:36:46.126409 ip-10-0-140-196 systemd[1]: Failed to start Kubernetes Kubelet. Sep 17 11:36:56.151797 ip-10-0-140-196 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 17 11:36:56.151811 ip-10-0-140-196 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot ed080701fd684c44892d5b8b16fafd3f -- Sep 17 11:38:48.736474 ip-10-0-140-196 systemd[1]: Starting Kubernetes Kubelet... Sep 17 11:38:49.082448 ip-10-0-140-196 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 11:38:49.082448 ip-10-0-140-196 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 17 11:38:49.082448 ip-10-0-140-196 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 11:38:49.082448 ip-10-0-140-196 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 17 11:38:49.082448 ip-10-0-140-196 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 11:38:49.083252 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.083126 2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 17 11:38:49.089567 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089551 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 11:38:49.089567 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089567 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 11:38:49.089633 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089571 2581 feature_gate.go:328] unrecognized feature gate: Example Sep 17 11:38:49.089633 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089575 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 11:38:49.089633 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089578 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 11:38:49.089633 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089581 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 11:38:49.089633 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089583 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 11:38:49.089633 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089586 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 11:38:49.089633 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089589 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 11:38:49.089633 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089591 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 11:38:49.089633 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089594 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 11:38:49.089633 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089608 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 11:38:49.089633 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089611 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 11:38:49.089633 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089614 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 11:38:49.089633 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089616 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 11:38:49.089633 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089619 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 11:38:49.089633 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089622 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 11:38:49.089633 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089624 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 11:38:49.089633 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089626 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 11:38:49.089633 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089629 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 11:38:49.089633 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089632 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 11:38:49.089633 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089634 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 11:38:49.090104 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089637 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 11:38:49.090104 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089640 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 11:38:49.090104 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089643 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 11:38:49.090104 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089646 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 11:38:49.090104 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089649 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 11:38:49.090104 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089651 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 11:38:49.090104 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089654 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 11:38:49.090104 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089656 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 11:38:49.090104 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089658 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 11:38:49.090104 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089661 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 11:38:49.090104 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089663 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 11:38:49.090104 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089665 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 11:38:49.090104 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089668 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 11:38:49.090104 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089670 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 11:38:49.090104 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089673 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 11:38:49.090104 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089675 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 11:38:49.090104 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089678 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 11:38:49.090104 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089681 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 11:38:49.090104 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089683 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 11:38:49.090104 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089685 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 11:38:49.090604 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089688 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 11:38:49.090604 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089690 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 11:38:49.090604 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089693 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 11:38:49.090604 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089695 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 11:38:49.090604 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089697 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 11:38:49.090604 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089699 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 11:38:49.090604 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089702 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 11:38:49.090604 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089704 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 11:38:49.090604 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089706 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 11:38:49.090604 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089709 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 11:38:49.090604 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089711 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 11:38:49.090604 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089715 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 11:38:49.090604 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089718 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 11:38:49.090604 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089723 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 11:38:49.090604 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089726 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 11:38:49.090604 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089728 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 11:38:49.090604 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089731 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 11:38:49.090604 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089733 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 11:38:49.090604 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089736 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 11:38:49.091077 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089738 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 11:38:49.091077 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089741 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 11:38:49.091077 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089743 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 11:38:49.091077 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089746 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 11:38:49.091077 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089748 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 11:38:49.091077 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089751 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 11:38:49.091077 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089753 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 11:38:49.091077 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089756 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 11:38:49.091077 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089758 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 11:38:49.091077 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089762 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 11:38:49.091077 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089765 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 11:38:49.091077 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089768 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 11:38:49.091077 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089771 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 11:38:49.091077 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089773 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 11:38:49.091077 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089776 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 11:38:49.091077 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089778 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 11:38:49.091077 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089781 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 11:38:49.091077 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089784 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 11:38:49.091077 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089788 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 11:38:49.091077 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089790 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 11:38:49.091575 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089793 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 11:38:49.091575 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089795 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 11:38:49.091575 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089798 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 11:38:49.091575 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089800 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 11:38:49.091575 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089803 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 11:38:49.091575 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089805 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 11:38:49.091575 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089810 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 11:38:49.091575 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089814 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 11:38:49.091575 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089817 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 11:38:49.091575 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089819 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 11:38:49.091575 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089822 2581 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 11:38:49.091575 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089826 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 11:38:49.091575 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089829 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 11:38:49.091575 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089832 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 11:38:49.091575 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089835 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 11:38:49.091575 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089838 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 11:38:49.091575 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089840 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 11:38:49.091575 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.089843 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 11:38:49.091575 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090551 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 11:38:49.091575 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090558 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 11:38:49.092061 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090561 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 11:38:49.092061 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090564 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 11:38:49.092061 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090567 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 11:38:49.092061 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090571 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 11:38:49.092061 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090573 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 11:38:49.092061 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090576 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 11:38:49.092061 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090578 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 11:38:49.092061 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090581 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 11:38:49.092061 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090584 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 11:38:49.092061 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090586 2581 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 11:38:49.092061 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090589 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 11:38:49.092061 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090591 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 11:38:49.092061 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090593 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 11:38:49.092061 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090596 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 11:38:49.092061 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090598 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 11:38:49.092061 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090601 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 11:38:49.092061 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090604 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 11:38:49.092061 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090606 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 11:38:49.092061 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090610 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 11:38:49.092061 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090613 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 11:38:49.092631 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090616 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 11:38:49.092631 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090618 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 11:38:49.092631 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090621 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 11:38:49.092631 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090624 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 11:38:49.092631 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090626 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 11:38:49.092631 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090629 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 11:38:49.092631 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090632 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 11:38:49.092631 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090634 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 11:38:49.092631 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090636 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 11:38:49.092631 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090639 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 11:38:49.092631 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090641 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 11:38:49.092631 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090644 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 11:38:49.092631 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090646 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 11:38:49.092631 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090649 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 11:38:49.092631 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090651 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 11:38:49.092631 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090655 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 11:38:49.092631 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090659 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 11:38:49.092631 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090661 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 11:38:49.092631 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090664 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 11:38:49.092631 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090668 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 11:38:49.093137 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090672 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 11:38:49.093137 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090675 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 11:38:49.093137 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090679 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 11:38:49.093137 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090681 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 11:38:49.093137 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090684 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 11:38:49.093137 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090687 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 11:38:49.093137 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090689 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 11:38:49.093137 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090692 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 11:38:49.093137 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090694 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 11:38:49.093137 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090699 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 11:38:49.093137 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090702 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 11:38:49.093137 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090705 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 11:38:49.093137 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090707 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 11:38:49.093137 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090711 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 11:38:49.093137 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090714 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 11:38:49.093137 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090717 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 11:38:49.093137 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090720 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 11:38:49.093137 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090722 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 11:38:49.093137 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090725 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 11:38:49.093620 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090727 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 11:38:49.093620 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090730 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 11:38:49.093620 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090732 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 11:38:49.093620 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090735 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 11:38:49.093620 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090737 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 11:38:49.093620 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090739 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 11:38:49.093620 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090742 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 11:38:49.093620 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090744 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 11:38:49.093620 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090747 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 11:38:49.093620 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090750 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 11:38:49.093620 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090752 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 11:38:49.093620 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090755 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 11:38:49.093620 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090758 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 11:38:49.093620 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090760 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 11:38:49.093620 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090763 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 11:38:49.093620 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090765 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 11:38:49.093620 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090767 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 11:38:49.093620 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090770 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 11:38:49.093620 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090772 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 11:38:49.093620 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090775 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 11:38:49.094115 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090777 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 11:38:49.094115 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090780 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 11:38:49.094115 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090785 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 11:38:49.094115 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090788 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 11:38:49.094115 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090791 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 11:38:49.094115 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090793 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 11:38:49.094115 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090795 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 11:38:49.094115 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090798 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 11:38:49.094115 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090801 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 11:38:49.094115 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090803 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 11:38:49.094115 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090807 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 11:38:49.094115 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090810 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 11:38:49.094115 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090814 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 11:38:49.094115 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090816 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 11:38:49.094115 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090819 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 11:38:49.094115 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090821 2581 feature_gate.go:328] unrecognized feature gate: Example Sep 17 11:38:49.094115 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090824 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 11:38:49.094115 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.090826 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 11:38:49.094115 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091196 2581 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 11:38:49.094115 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091200 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 11:38:49.094632 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091203 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 11:38:49.094632 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091206 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 11:38:49.094632 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091209 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 11:38:49.094632 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091212 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 11:38:49.094632 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091215 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 11:38:49.094632 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091219 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 11:38:49.094632 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091223 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 11:38:49.094632 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091226 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 11:38:49.094632 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091229 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 11:38:49.094632 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091232 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 11:38:49.094632 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091235 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 11:38:49.094632 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091238 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 11:38:49.094632 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091240 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 11:38:49.094632 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091243 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 11:38:49.094632 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091248 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 11:38:49.094632 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091250 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 11:38:49.094632 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091253 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 11:38:49.094632 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091255 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 11:38:49.094632 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091258 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 11:38:49.095100 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091263 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 11:38:49.095100 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091266 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 11:38:49.095100 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091269 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 11:38:49.095100 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091272 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 11:38:49.095100 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091275 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 11:38:49.095100 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091278 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 11:38:49.095100 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091281 2581 feature_gate.go:328] unrecognized feature gate: Example Sep 17 11:38:49.095100 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091283 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 11:38:49.095100 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091286 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 11:38:49.095100 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091288 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 11:38:49.095100 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091290 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 11:38:49.095100 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091293 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 11:38:49.095100 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091295 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 11:38:49.095100 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091298 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 11:38:49.095100 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091300 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 11:38:49.095100 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091302 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 11:38:49.095100 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091305 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 11:38:49.095100 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091308 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 11:38:49.095100 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091310 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 11:38:49.095581 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091313 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 11:38:49.095581 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091315 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 11:38:49.095581 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091317 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 11:38:49.095581 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091320 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 11:38:49.095581 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091322 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 11:38:49.095581 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091324 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 11:38:49.095581 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091327 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 11:38:49.095581 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091329 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 11:38:49.095581 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091333 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 11:38:49.095581 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091336 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 11:38:49.095581 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091338 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 11:38:49.095581 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091340 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 11:38:49.095581 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091343 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 11:38:49.095581 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091345 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 11:38:49.095581 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091348 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 11:38:49.095581 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091352 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 11:38:49.095581 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091355 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 11:38:49.095581 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091358 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 11:38:49.095581 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091361 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 11:38:49.095581 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091364 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 11:38:49.096052 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091367 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 11:38:49.096052 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091369 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 11:38:49.096052 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091372 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 11:38:49.096052 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091376 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 11:38:49.096052 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091378 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 11:38:49.096052 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091380 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 11:38:49.096052 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091383 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 11:38:49.096052 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091385 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 11:38:49.096052 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091388 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 11:38:49.096052 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091390 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 11:38:49.096052 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091393 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 11:38:49.096052 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091395 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 11:38:49.096052 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091398 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 11:38:49.096052 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091400 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 11:38:49.096052 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091402 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 11:38:49.096052 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091405 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 11:38:49.096052 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091408 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 11:38:49.096052 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091410 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 11:38:49.096052 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091412 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 11:38:49.096052 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091415 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 11:38:49.096540 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091418 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 11:38:49.096540 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091421 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 11:38:49.096540 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091423 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 11:38:49.096540 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091426 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 11:38:49.096540 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091428 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 11:38:49.096540 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091431 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 11:38:49.096540 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091433 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 11:38:49.096540 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091436 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 11:38:49.096540 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091438 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 11:38:49.096540 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091445 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 11:38:49.096540 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091448 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 11:38:49.096540 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091450 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 11:38:49.096540 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091452 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 11:38:49.096540 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091455 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 11:38:49.096540 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091457 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 11:38:49.096540 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091460 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 11:38:49.096540 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091462 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 11:38:49.096540 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091464 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 11:38:49.096540 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.091467 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 11:38:49.096540 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091533 2581 flags.go:64] FLAG: --address="0.0.0.0" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091566 2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091573 2581 flags.go:64] FLAG: --anonymous-auth="true" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091577 2581 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091582 2581 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091585 2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091589 2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091593 2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091597 2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091599 2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091603 2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091606 2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091609 2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091613 2581 flags.go:64] FLAG: --cgroup-root="" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091616 2581 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091619 2581 flags.go:64] FLAG: --client-ca-file="" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091622 2581 flags.go:64] FLAG: --cloud-provider="external" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091624 2581 flags.go:64] FLAG: --cluster-dns="[]" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091628 2581 flags.go:64] FLAG: --cluster-domain="" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091631 2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091634 2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091637 2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091641 2581 flags.go:64] FLAG: --container-log-max-files="5" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091645 2581 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 17 11:38:49.097076 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091647 2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091650 2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091653 2581 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091656 2581 flags.go:64] FLAG: --contention-profiling="false" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091659 2581 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091662 2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091664 2581 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091667 2581 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091671 2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091674 2581 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091676 2581 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091679 2581 flags.go:64] FLAG: --enable-load-reader="false" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091682 2581 flags.go:64] FLAG: --enable-server="true" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091686 2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091690 2581 flags.go:64] FLAG: --event-burst="100" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091693 2581 flags.go:64] FLAG: --event-qps="50" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091696 2581 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091699 2581 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091702 2581 flags.go:64] FLAG: --eviction-hard="" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091710 2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091713 2581 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091717 2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091720 2581 flags.go:64] FLAG: --eviction-soft="" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091722 2581 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091725 2581 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 17 11:38:49.097660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091728 2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091731 2581 flags.go:64] FLAG: --experimental-mounter-path="" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091734 2581 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091737 2581 flags.go:64] FLAG: --fail-swap-on="true" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091740 2581 flags.go:64] FLAG: --feature-gates="" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091743 2581 flags.go:64] FLAG: --file-check-frequency="20s" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091747 2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091750 2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091753 2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091756 2581 flags.go:64] FLAG: --healthz-port="10248" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091759 2581 flags.go:64] FLAG: --help="false" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091761 2581 flags.go:64] FLAG: --hostname-override="ip-10-0-140-196.ec2.internal" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091764 2581 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091767 2581 flags.go:64] FLAG: --http-check-frequency="20s" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091770 2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091773 2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091776 2581 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091779 2581 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091782 2581 flags.go:64] FLAG: --image-service-endpoint="" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091784 2581 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091787 2581 flags.go:64] FLAG: --kube-api-burst="100" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091790 2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091793 2581 flags.go:64] FLAG: --kube-api-qps="50" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091796 2581 flags.go:64] FLAG: --kube-reserved="" Sep 17 11:38:49.098325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091799 2581 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091801 2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091804 2581 flags.go:64] FLAG: --kubelet-cgroups="" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091807 2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091810 2581 flags.go:64] FLAG: --lock-file="" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091812 2581 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091815 2581 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091818 2581 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091823 2581 flags.go:64] FLAG: --log-json-split-stream="false" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091826 2581 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091829 2581 flags.go:64] FLAG: --log-text-split-stream="false" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091832 2581 flags.go:64] FLAG: --logging-format="text" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091834 2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091838 2581 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091841 2581 flags.go:64] FLAG: --manifest-url="" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091843 2581 flags.go:64] FLAG: --manifest-url-header="" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091848 2581 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091850 2581 flags.go:64] FLAG: --max-open-files="1000000" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091854 2581 flags.go:64] FLAG: --max-pods="110" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091857 2581 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091860 2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091863 2581 flags.go:64] FLAG: --memory-manager-policy="None" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091866 2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091868 2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091871 2581 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 17 11:38:49.098940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091875 2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091881 2581 flags.go:64] FLAG: --node-status-max-images="50" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091884 2581 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091887 2581 flags.go:64] FLAG: --oom-score-adj="-999" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091890 2581 flags.go:64] FLAG: --pod-cidr="" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091893 2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091899 2581 flags.go:64] FLAG: --pod-manifest-path="" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091902 2581 flags.go:64] FLAG: --pod-max-pids="-1" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091905 2581 flags.go:64] FLAG: --pods-per-core="0" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091908 2581 flags.go:64] FLAG: --port="10250" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091911 2581 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091914 2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02aefcdc53d9c60f0" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091918 2581 flags.go:64] FLAG: --qos-reserved="" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091920 2581 flags.go:64] FLAG: --read-only-port="10255" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091923 2581 flags.go:64] FLAG: --register-node="true" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091926 2581 flags.go:64] FLAG: --register-with-taints="" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091929 2581 flags.go:64] FLAG: --registry-burst="10" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091932 2581 flags.go:64] FLAG: --registry-qps="5" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091935 2581 flags.go:64] FLAG: --reserved-cpus="" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091937 2581 flags.go:64] FLAG: --reserved-memory="" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091941 2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091944 2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091947 2581 flags.go:64] FLAG: --rotate-certificates="false" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091950 2581 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091953 2581 flags.go:64] FLAG: --runonce="false" Sep 17 11:38:49.099589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091955 2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091958 2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091961 2581 flags.go:64] FLAG: --seccomp-default="false" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091963 2581 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091966 2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091969 2581 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091972 2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091975 2581 flags.go:64] FLAG: --storage-driver-password="root" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091977 2581 flags.go:64] FLAG: --storage-driver-secure="false" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091980 2581 flags.go:64] FLAG: --storage-driver-table="stats" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091983 2581 flags.go:64] FLAG: --storage-driver-user="root" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091985 2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091988 2581 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091991 2581 flags.go:64] FLAG: --system-cgroups="" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091994 2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.091999 2581 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.092002 2581 flags.go:64] FLAG: --tls-cert-file="" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.092004 2581 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.092008 2581 flags.go:64] FLAG: --tls-min-version="" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.092012 2581 flags.go:64] FLAG: --tls-private-key-file="" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.092015 2581 flags.go:64] FLAG: --topology-manager-policy="none" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.092018 2581 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.092020 2581 flags.go:64] FLAG: --topology-manager-scope="container" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.092023 2581 flags.go:64] FLAG: --v="2" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.092027 2581 flags.go:64] FLAG: --version="false" Sep 17 11:38:49.100289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.092030 2581 flags.go:64] FLAG: --vmodule="" Sep 17 11:38:49.101292 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.092034 2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 17 11:38:49.101292 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.092039 2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 17 11:38:49.101292 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092149 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 11:38:49.101292 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092170 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 11:38:49.101292 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092174 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 11:38:49.101292 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092178 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 11:38:49.101292 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092180 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 11:38:49.101292 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092183 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 11:38:49.101292 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092186 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 11:38:49.101292 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092189 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 11:38:49.101292 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092191 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 11:38:49.101292 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092194 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 11:38:49.101292 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092196 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 11:38:49.101292 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092199 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 11:38:49.101292 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092203 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 11:38:49.101292 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092207 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 11:38:49.101292 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092210 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 11:38:49.101292 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092212 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 11:38:49.101292 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092215 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 11:38:49.101919 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092218 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 11:38:49.101919 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092221 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 11:38:49.101919 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092224 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 11:38:49.101919 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092227 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 11:38:49.101919 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092230 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 11:38:49.101919 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092232 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 11:38:49.101919 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092236 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 11:38:49.101919 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092239 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 11:38:49.101919 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092242 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 11:38:49.101919 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092245 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 11:38:49.101919 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092248 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 11:38:49.101919 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092250 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 11:38:49.101919 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092253 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 11:38:49.101919 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092256 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 11:38:49.101919 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092259 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 11:38:49.101919 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092270 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 11:38:49.101919 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092273 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 11:38:49.101919 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092276 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 11:38:49.101919 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092279 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 11:38:49.101919 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092281 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 11:38:49.102460 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092284 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 11:38:49.102460 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092286 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 11:38:49.102460 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092289 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 11:38:49.102460 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092292 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 11:38:49.102460 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092294 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 11:38:49.102460 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092297 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 11:38:49.102460 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092300 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 11:38:49.102460 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092302 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 11:38:49.102460 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092304 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 11:38:49.102460 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092307 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 11:38:49.102460 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092310 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 11:38:49.102460 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092312 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 11:38:49.102460 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092315 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 11:38:49.102460 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092317 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 11:38:49.102460 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092323 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 11:38:49.102460 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092326 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 11:38:49.102460 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092328 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 11:38:49.102460 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092331 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 11:38:49.102460 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092335 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 11:38:49.102460 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092338 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 11:38:49.102959 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092341 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 11:38:49.102959 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092343 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 11:38:49.102959 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092346 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 11:38:49.102959 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092348 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 11:38:49.102959 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092351 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 11:38:49.102959 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092368 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 11:38:49.102959 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092373 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 11:38:49.102959 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092378 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 11:38:49.102959 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092381 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 11:38:49.102959 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092384 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 11:38:49.102959 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092386 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 11:38:49.102959 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092389 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 11:38:49.102959 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092392 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 11:38:49.102959 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092394 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 11:38:49.102959 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092397 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 11:38:49.102959 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092400 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 11:38:49.102959 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092402 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 11:38:49.102959 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092405 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 11:38:49.102959 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092407 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 11:38:49.103452 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092410 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 11:38:49.103452 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092412 2581 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 11:38:49.103452 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092415 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 11:38:49.103452 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092417 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 11:38:49.103452 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092420 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 11:38:49.103452 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092423 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 11:38:49.103452 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092425 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 11:38:49.103452 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092428 2581 feature_gate.go:328] unrecognized feature gate: Example Sep 17 11:38:49.103452 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092431 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 11:38:49.103452 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092433 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 11:38:49.103452 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092436 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 11:38:49.103452 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092439 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 11:38:49.103452 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092442 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 11:38:49.103452 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092444 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 11:38:49.103452 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092447 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 11:38:49.103452 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092449 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 11:38:49.103452 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092452 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 11:38:49.103452 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092454 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 11:38:49.103452 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092457 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 11:38:49.103452 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092461 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 11:38:49.103937 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092464 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 11:38:49.103937 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092466 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 11:38:49.103937 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.092469 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 11:38:49.103937 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.092994 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 11:38:49.103937 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.101570 2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 17 11:38:49.103937 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.101585 2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 17 11:38:49.103937 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.102239 2581 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 17 11:38:49.103937 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.102250 2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 17 11:38:49.103937 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102308 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 11:38:49.103937 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102313 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 11:38:49.103937 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102316 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 11:38:49.103937 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102319 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 11:38:49.103937 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102322 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 11:38:49.103937 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102326 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 11:38:49.103937 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102328 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 11:38:49.104327 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102331 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 11:38:49.104327 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102334 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 11:38:49.104327 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102336 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 11:38:49.104327 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102339 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 11:38:49.104327 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102342 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 11:38:49.104327 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102344 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 11:38:49.104327 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102347 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 11:38:49.104327 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102350 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 11:38:49.104327 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102352 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 11:38:49.104327 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102356 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 11:38:49.104327 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102359 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 11:38:49.104327 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102361 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 11:38:49.104327 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102365 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 11:38:49.104327 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102368 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 11:38:49.104327 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102371 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 11:38:49.104327 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102373 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 11:38:49.104327 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102376 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 11:38:49.104327 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102379 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 11:38:49.104327 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102382 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 11:38:49.104327 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102384 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 11:38:49.104871 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102387 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 11:38:49.104871 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102389 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 11:38:49.104871 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102393 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 11:38:49.104871 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102396 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 11:38:49.104871 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102399 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 11:38:49.104871 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102401 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 11:38:49.104871 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102404 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 11:38:49.104871 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102407 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 11:38:49.104871 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102409 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 11:38:49.104871 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102412 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 11:38:49.104871 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102414 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 11:38:49.104871 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102417 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 11:38:49.104871 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102419 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 11:38:49.104871 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102422 2581 feature_gate.go:328] unrecognized feature gate: Example Sep 17 11:38:49.104871 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102424 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 11:38:49.104871 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102427 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 11:38:49.104871 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102429 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 11:38:49.104871 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102432 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 11:38:49.104871 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102434 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 11:38:49.104871 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102438 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 11:38:49.105502 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102440 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 11:38:49.105502 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102443 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 11:38:49.105502 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102445 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 11:38:49.105502 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102448 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 11:38:49.105502 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102450 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 11:38:49.105502 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102453 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 11:38:49.105502 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102456 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 11:38:49.105502 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102458 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 11:38:49.105502 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102461 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 11:38:49.105502 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102463 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 11:38:49.105502 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102466 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 11:38:49.105502 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102469 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 11:38:49.105502 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102472 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 11:38:49.105502 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102476 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 11:38:49.105502 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102481 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 11:38:49.105502 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102485 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 11:38:49.105502 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102488 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 11:38:49.105502 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102492 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 11:38:49.105935 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102495 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 11:38:49.105935 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102497 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 11:38:49.105935 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102500 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 11:38:49.105935 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102503 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 11:38:49.105935 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102505 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 11:38:49.105935 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102509 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 11:38:49.105935 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102513 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 11:38:49.105935 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102515 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 11:38:49.105935 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102518 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 11:38:49.105935 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102521 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 11:38:49.105935 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102523 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 11:38:49.105935 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102525 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 11:38:49.105935 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102528 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 11:38:49.105935 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102530 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 11:38:49.105935 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102533 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 11:38:49.105935 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102536 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 11:38:49.105935 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102539 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 11:38:49.105935 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102541 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 11:38:49.105935 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102544 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 11:38:49.106427 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102546 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 11:38:49.106427 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102549 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 11:38:49.106427 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102552 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 11:38:49.106427 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102554 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 11:38:49.106427 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102556 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 11:38:49.106427 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102559 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 11:38:49.106427 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102561 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 11:38:49.106427 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102564 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 11:38:49.106427 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102567 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 11:38:49.106427 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102570 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 11:38:49.106427 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102573 2581 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 11:38:49.106427 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102575 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 11:38:49.106427 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102578 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 11:38:49.106427 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102580 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 11:38:49.106427 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102583 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 11:38:49.106790 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.102588 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 11:38:49.106790 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102699 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 11:38:49.106790 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102704 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 11:38:49.106790 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102707 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 11:38:49.106790 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102711 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 11:38:49.106790 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102713 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 11:38:49.106790 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102716 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 11:38:49.106790 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102719 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 11:38:49.106790 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102721 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 11:38:49.106790 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102724 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 11:38:49.106790 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102726 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 11:38:49.106790 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102729 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 11:38:49.106790 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102731 2581 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 11:38:49.106790 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102735 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 11:38:49.106790 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102738 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 11:38:49.107179 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102741 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 11:38:49.107179 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102743 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 11:38:49.107179 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102745 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 11:38:49.107179 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102748 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 11:38:49.107179 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102751 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 11:38:49.107179 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102753 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 11:38:49.107179 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102755 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 11:38:49.107179 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102758 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 11:38:49.107179 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102760 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 11:38:49.107179 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102763 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 11:38:49.107179 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102765 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 11:38:49.107179 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102769 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 11:38:49.107179 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102772 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 11:38:49.107179 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102774 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 11:38:49.107179 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102777 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 11:38:49.107179 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102780 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 11:38:49.107179 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102782 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 11:38:49.107179 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102785 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 11:38:49.107179 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102787 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 11:38:49.107179 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102790 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 11:38:49.107703 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102792 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 11:38:49.107703 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102795 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 11:38:49.107703 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102797 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 11:38:49.107703 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102800 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 11:38:49.107703 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102802 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 11:38:49.107703 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102805 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 11:38:49.107703 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102807 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 11:38:49.107703 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102810 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 11:38:49.107703 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102812 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 11:38:49.107703 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102814 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 11:38:49.107703 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102817 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 11:38:49.107703 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102820 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 11:38:49.107703 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102822 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 11:38:49.107703 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102825 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 11:38:49.107703 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102827 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 11:38:49.107703 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102830 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 11:38:49.107703 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102832 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 11:38:49.107703 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102835 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 11:38:49.107703 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102837 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 11:38:49.107703 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102840 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 11:38:49.108198 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102842 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 11:38:49.108198 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102845 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 11:38:49.108198 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102848 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 11:38:49.108198 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102850 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 11:38:49.108198 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102853 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 11:38:49.108198 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102856 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 11:38:49.108198 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102859 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 11:38:49.108198 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102861 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 11:38:49.108198 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102864 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 11:38:49.108198 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102866 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 11:38:49.108198 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102869 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 11:38:49.108198 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102872 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 11:38:49.108198 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102874 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 11:38:49.108198 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102876 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 11:38:49.108198 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102879 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 11:38:49.108198 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102882 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 11:38:49.108198 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102885 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 11:38:49.108198 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102887 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 11:38:49.108198 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102890 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 11:38:49.108691 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102892 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 11:38:49.108691 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102895 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 11:38:49.108691 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102897 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 11:38:49.108691 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102900 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 11:38:49.108691 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102902 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 11:38:49.108691 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102906 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 11:38:49.108691 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102908 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 11:38:49.108691 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102912 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 11:38:49.108691 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102915 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 11:38:49.108691 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102918 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 11:38:49.108691 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102921 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 11:38:49.108691 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102924 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 11:38:49.108691 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102926 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 11:38:49.108691 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102930 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 11:38:49.108691 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102933 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 11:38:49.108691 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102936 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 11:38:49.108691 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102939 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 11:38:49.108691 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102942 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 11:38:49.108691 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102945 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 11:38:49.109163 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102947 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 11:38:49.109163 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102950 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 11:38:49.109163 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102953 2581 feature_gate.go:328] unrecognized feature gate: Example Sep 17 11:38:49.109163 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102956 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 11:38:49.109163 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102959 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 11:38:49.109163 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102961 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 11:38:49.109163 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:49.102965 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 11:38:49.109163 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.102970 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 11:38:49.109163 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.103071 2581 server.go:962] "Client rotation is on, will bootstrap in background" Sep 17 11:38:49.109163 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.105250 2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 17 11:38:49.109163 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.106093 2581 server.go:1019] "Starting client certificate rotation" Sep 17 11:38:49.109163 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.106184 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 11:38:49.109163 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.106902 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 11:38:49.109499 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.108591 2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 11:38:49.113314 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.113300 2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 11:38:49.121555 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.121537 2581 log.go:25] "Validated CRI v1 runtime API" Sep 17 11:38:49.124892 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.124871 2581 log.go:25] "Validated CRI v1 image API" Sep 17 11:38:49.125790 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.125765 2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 17 11:38:49.128370 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.128351 2581 fs.go:135] Filesystem UUIDs: map[01e2d8a4-0b8d-4b8f-a3be-7a9fc8eb2ac7:/dev/nvme0n1p4 139d4afb-979d-4e6f-8c4e-83111b9ae94d:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Sep 17 11:38:49.128432 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.128370 2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 17 11:38:49.133634 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.133521 2581 manager.go:216] Machine: {Timestamp:2026-09-17 11:38:49.132144618 +0000 UTC m=+0.289552770 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3069714 MemoryCapacity:32812154880 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20f445ebb6d268782fc0e0d30ae6d0 SystemUUID:ec20f445-ebb6-d268-782f-c0e0d30ae6d0 BootID:ed080701-fd68-4c44-892d-5b8b16fafd3f Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ee:dc:42:6b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ee:dc:42:6b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:f6:d4:9f:dc:75 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812154880 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 17 11:38:49.133634 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.133627 2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 17 11:38:49.133759 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.133703 2581 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 17 11:38:49.137015 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.136980 2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 17 11:38:49.137190 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.137020 2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-196.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 17 11:38:49.137925 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.137913 2581 topology_manager.go:138] "Creating topology manager with none policy" Sep 17 11:38:49.137983 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.137932 2581 container_manager_linux.go:309] "Creating device plugin manager" Sep 17 11:38:49.137983 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.137945 2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 11:38:49.138089 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.138074 2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 11:38:49.139443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.139338 2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 17 11:38:49.139443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.139363 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 11:38:49.144268 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.144252 2581 state_mem.go:36] "Initialized new in-memory state store" Sep 17 11:38:49.144423 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.144411 2581 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 17 11:38:49.144492 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.144463 2581 kubelet.go:486] "Attempting to sync node with API server" Sep 17 11:38:49.144492 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.144479 2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 17 11:38:49.144585 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.144500 2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 17 11:38:49.144585 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.144513 2581 kubelet.go:398] "Adding apiserver pod source" Sep 17 11:38:49.144585 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.144526 2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 17 11:38:49.144910 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.144896 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 11:38:49.144968 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.144919 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 11:38:49.147285 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.147148 2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 17 11:38:49.150390 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.150377 2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 17 11:38:49.150435 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.150407 2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 17 11:38:49.152391 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.152379 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 17 11:38:49.152458 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.152397 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 17 11:38:49.152458 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.152403 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 17 11:38:49.152458 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.152409 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 17 11:38:49.152458 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.152418 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 17 11:38:49.152458 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.152423 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 17 11:38:49.152458 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.152441 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 17 11:38:49.152458 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.152447 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 17 11:38:49.152458 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.152454 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 17 11:38:49.152458 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.152460 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 17 11:38:49.152691 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.152468 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 17 11:38:49.152691 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.152473 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 17 11:38:49.153380 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.153370 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 17 11:38:49.153380 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.153380 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 17 11:38:49.154193 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.154148 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 17 11:38:49.154193 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.154148 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-196.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 17 11:38:49.156628 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.156614 2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-196.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 17 11:38:49.157130 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.157118 2581 server.go:1268] "Started kubelet" Sep 17 11:38:49.157214 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.157192 2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 17 11:38:49.157833 ip-10-0-140-196 systemd[1]: Started Kubernetes Kubelet. Sep 17 11:38:49.158208 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.158132 2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 17 11:38:49.158289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.158264 2581 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 17 11:38:49.158620 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.158523 2581 server.go:310] "Adding debug handlers to kubelet server" Sep 17 11:38:49.159674 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.159658 2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 17 11:38:49.161644 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.161622 2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 17 11:38:49.165469 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.164333 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-196.ec2.internal.18d618c3556a88d5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-196.ec2.internal,UID:ip-10-0-140-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-196.ec2.internal,},FirstTimestamp:2026-09-17 11:38:49.157085397 +0000 UTC m=+0.314493548,LastTimestamp:2026-09-17 11:38:49.157085397 +0000 UTC m=+0.314493548,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-196.ec2.internal,}" Sep 17 11:38:49.165953 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.165932 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 17 11:38:49.166674 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.166658 2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 17 11:38:49.167000 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.166848 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-196.ec2.internal\" not found" Sep 17 11:38:49.167457 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.167441 2581 factory.go:55] Registering systemd factory Sep 17 11:38:49.167457 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.167458 2581 factory.go:223] Registration of the systemd container factory successfully Sep 17 11:38:49.167568 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.167513 2581 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 17 11:38:49.167568 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.167519 2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 17 11:38:49.167568 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.167523 2581 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 17 11:38:49.167711 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.167651 2581 reconstruct.go:97] "Volume reconstruction finished" Sep 17 11:38:49.167711 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.167658 2581 reconciler.go:29] "Reconciler: start to sync state" Sep 17 11:38:49.167711 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.167666 2581 factory.go:153] Registering CRI-O factory Sep 17 11:38:49.167711 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.167677 2581 factory.go:223] Registration of the crio container factory successfully Sep 17 11:38:49.167864 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.167726 2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 17 11:38:49.167864 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.167745 2581 factory.go:103] Registering Raw factory Sep 17 11:38:49.167864 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.167762 2581 manager.go:1196] Started watching for new ooms in manager Sep 17 11:38:49.168137 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.168125 2581 manager.go:349] Starting recovery of all containers Sep 17 11:38:49.173123 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.173093 2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-196.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 17 11:38:49.174134 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.174083 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 17 11:38:49.181552 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.181533 2581 manager.go:354] Recovery completed Sep 17 11:38:49.185502 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.185490 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 11:38:49.188187 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.188171 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeHasSufficientMemory" Sep 17 11:38:49.188246 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.188200 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 11:38:49.188246 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.188211 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeHasSufficientPID" Sep 17 11:38:49.197260 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.197232 2581 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 17 11:38:49.197260 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.197247 2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 17 11:38:49.197260 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.197248 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 11:38:49.197260 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.197265 2581 state_mem.go:36] "Initialized new in-memory state store" Sep 17 11:38:49.197440 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.197133 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-196.ec2.internal.18d618c357451a4d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-196.ec2.internal,UID:ip-10-0-140-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-196.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-196.ec2.internal,},FirstTimestamp:2026-09-17 11:38:49.188186701 +0000 UTC m=+0.345594853,LastTimestamp:2026-09-17 11:38:49.188186701 +0000 UTC m=+0.345594853,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-196.ec2.internal,}" Sep 17 11:38:49.198134 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.198115 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeHasSufficientMemory" Sep 17 11:38:49.198280 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.198147 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 11:38:49.198280 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.198178 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeHasSufficientPID" Sep 17 11:38:49.199523 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.199510 2581 policy_none.go:49] "None policy: Start" Sep 17 11:38:49.199595 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.199529 2581 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 17 11:38:49.199595 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.199542 2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 17 11:38:49.201586 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.201573 2581 policy_none.go:47] "Start" Sep 17 11:38:49.208878 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.208819 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-196.ec2.internal.18d618c357456379 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-196.ec2.internal,UID:ip-10-0-140-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-196.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-196.ec2.internal,},FirstTimestamp:2026-09-17 11:38:49.188205433 +0000 UTC m=+0.345613586,LastTimestamp:2026-09-17 11:38:49.188205433 +0000 UTC m=+0.345613586,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-196.ec2.internal,}" Sep 17 11:38:49.220228 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.220128 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-196.ec2.internal.18d618c3574588c8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-196.ec2.internal,UID:ip-10-0-140-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-196.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-196.ec2.internal,},FirstTimestamp:2026-09-17 11:38:49.188214984 +0000 UTC m=+0.345623136,LastTimestamp:2026-09-17 11:38:49.188214984 +0000 UTC m=+0.345623136,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-196.ec2.internal,}" Sep 17 11:38:49.231543 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.231485 2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-196.ec2.internal.18d618c357451a4d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-196.ec2.internal.18d618c357451a4d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-196.ec2.internal,UID:ip-10-0-140-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-196.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-196.ec2.internal,},FirstTimestamp:2026-09-17 11:38:49.188186701 +0000 UTC m=+0.345594853,LastTimestamp:2026-09-17 11:38:49.198133655 +0000 UTC m=+0.355541808,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-196.ec2.internal,}" Sep 17 11:38:49.252241 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.242192 2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-196.ec2.internal.18d618c357456379\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-196.ec2.internal.18d618c357456379 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-196.ec2.internal,UID:ip-10-0-140-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-196.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-196.ec2.internal,},FirstTimestamp:2026-09-17 11:38:49.188205433 +0000 UTC m=+0.345613586,LastTimestamp:2026-09-17 11:38:49.198169997 +0000 UTC m=+0.355578194,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-196.ec2.internal,}" Sep 17 11:38:49.252241 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.245800 2581 manager.go:337] "Starting Device Plugin manager" Sep 17 11:38:49.252241 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.245840 2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 17 11:38:49.252241 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.245852 2581 server.go:87] "Starting device plugin registration server" Sep 17 11:38:49.252241 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.246222 2581 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 17 11:38:49.252241 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.246233 2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 17 11:38:49.252241 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.246328 2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 17 11:38:49.252241 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.246408 2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 17 11:38:49.252241 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.246419 2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 17 11:38:49.252241 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.246994 2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 17 11:38:49.252241 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.247041 2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-196.ec2.internal\" not found" Sep 17 11:38:49.252241 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.251669 2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-196.ec2.internal.18d618c3574588c8\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-196.ec2.internal.18d618c3574588c8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-196.ec2.internal,UID:ip-10-0-140-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-196.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-196.ec2.internal,},FirstTimestamp:2026-09-17 11:38:49.188214984 +0000 UTC m=+0.345623136,LastTimestamp:2026-09-17 11:38:49.198185879 +0000 UTC m=+0.355594031,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-196.ec2.internal,}" Sep 17 11:38:49.259834 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.259780 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-196.ec2.internal.18d618c35ad97b17 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-196.ec2.internal,UID:ip-10-0-140-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-140-196.ec2.internal,},FirstTimestamp:2026-09-17 11:38:49.248242455 +0000 UTC m=+0.405650594,LastTimestamp:2026-09-17 11:38:49.248242455 +0000 UTC m=+0.405650594,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-196.ec2.internal,}" Sep 17 11:38:49.271089 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.271052 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 17 11:38:49.272370 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.272352 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 17 11:38:49.272440 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.272396 2581 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 17 11:38:49.272440 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.272417 2581 kubelet.go:2448] "Starting kubelet main sync loop" Sep 17 11:38:49.272524 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.272455 2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 17 11:38:49.281015 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.280994 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Sep 17 11:38:49.287209 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.287193 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-26xmf" Sep 17 11:38:49.295261 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.295243 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-26xmf" Sep 17 11:38:49.346589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.346530 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 11:38:49.347409 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.347396 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeHasSufficientMemory" Sep 17 11:38:49.347470 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.347422 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 11:38:49.347470 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.347431 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeHasSufficientPID" Sep 17 11:38:49.347470 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.347451 2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-196.ec2.internal" Sep 17 11:38:49.364265 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.364244 2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-196.ec2.internal" Sep 17 11:38:49.364351 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.364273 2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-196.ec2.internal\": node \"ip-10-0-140-196.ec2.internal\" not found" Sep 17 11:38:49.372674 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.372632 2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-196.ec2.internal"] Sep 17 11:38:49.372764 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.372697 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 11:38:49.374030 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.374016 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeHasSufficientMemory" Sep 17 11:38:49.374115 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.374044 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 11:38:49.374115 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.374058 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeHasSufficientPID" Sep 17 11:38:49.376341 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.376327 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 11:38:49.376467 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.376452 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal" Sep 17 11:38:49.376549 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.376480 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 11:38:49.377125 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.377110 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeHasSufficientMemory" Sep 17 11:38:49.377203 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.377139 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 11:38:49.377203 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.377149 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeHasSufficientPID" Sep 17 11:38:49.377203 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.377115 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeHasSufficientMemory" Sep 17 11:38:49.377203 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.377188 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 11:38:49.377203 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.377200 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeHasSufficientPID" Sep 17 11:38:49.379447 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.379432 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-196.ec2.internal" Sep 17 11:38:49.379528 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.379456 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 11:38:49.380089 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.380074 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeHasSufficientMemory" Sep 17 11:38:49.380200 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.380097 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 11:38:49.380200 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.380109 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeHasSufficientPID" Sep 17 11:38:49.389496 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.389476 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-196.ec2.internal\" not found" Sep 17 11:38:49.403787 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.403770 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-196.ec2.internal\" not found" node="ip-10-0-140-196.ec2.internal" Sep 17 11:38:49.407266 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.407250 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-196.ec2.internal\" not found" node="ip-10-0-140-196.ec2.internal" Sep 17 11:38:49.469620 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.469595 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/73800decfd2c4467378c70aaf90595a7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal\" (UID: \"73800decfd2c4467378c70aaf90595a7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal" Sep 17 11:38:49.469755 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.469635 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73800decfd2c4467378c70aaf90595a7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal\" (UID: \"73800decfd2c4467378c70aaf90595a7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal" Sep 17 11:38:49.469755 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.469704 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e5a272a923bce547f142a7efecbeccfb-config\") pod \"kube-apiserver-proxy-ip-10-0-140-196.ec2.internal\" (UID: \"e5a272a923bce547f142a7efecbeccfb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-196.ec2.internal" Sep 17 11:38:49.489581 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.489566 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-196.ec2.internal\" not found" Sep 17 11:38:49.569836 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.569815 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e5a272a923bce547f142a7efecbeccfb-config\") pod \"kube-apiserver-proxy-ip-10-0-140-196.ec2.internal\" (UID: \"e5a272a923bce547f142a7efecbeccfb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-196.ec2.internal" Sep 17 11:38:49.569971 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.569841 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/73800decfd2c4467378c70aaf90595a7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal\" (UID: \"73800decfd2c4467378c70aaf90595a7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal" Sep 17 11:38:49.569971 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.569857 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73800decfd2c4467378c70aaf90595a7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal\" (UID: \"73800decfd2c4467378c70aaf90595a7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal" Sep 17 11:38:49.569971 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.569884 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73800decfd2c4467378c70aaf90595a7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal\" (UID: \"73800decfd2c4467378c70aaf90595a7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal" Sep 17 11:38:49.569971 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.569917 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e5a272a923bce547f142a7efecbeccfb-config\") pod \"kube-apiserver-proxy-ip-10-0-140-196.ec2.internal\" (UID: \"e5a272a923bce547f142a7efecbeccfb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-196.ec2.internal" Sep 17 11:38:49.569971 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.569934 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/73800decfd2c4467378c70aaf90595a7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal\" (UID: \"73800decfd2c4467378c70aaf90595a7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal" Sep 17 11:38:49.590135 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.590118 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-196.ec2.internal\" not found" Sep 17 11:38:49.690979 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.690897 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-196.ec2.internal\" not found" Sep 17 11:38:49.707081 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.707057 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal" Sep 17 11:38:49.710236 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:49.710221 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-196.ec2.internal" Sep 17 11:38:49.791669 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.791644 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-196.ec2.internal\" not found" Sep 17 11:38:49.892140 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.892111 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-196.ec2.internal\" not found" Sep 17 11:38:49.992734 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:49.992653 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-196.ec2.internal\" not found" Sep 17 11:38:50.026738 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:50.026695 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 11:38:50.093703 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:50.093670 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-196.ec2.internal\" not found" Sep 17 11:38:50.105816 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:50.105794 2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 17 11:38:50.105932 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:50.105915 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 17 11:38:50.179796 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:50.179766 2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 11:38:50.267694 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:50.267635 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-196.ec2.internal" Sep 17 11:38:50.278794 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:50.278775 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 17 11:38:50.280014 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:50.280002 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal" Sep 17 11:38:50.291961 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:50.291937 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 17 11:38:50.299234 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:50.299208 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 11:33:50 +0000 UTC" deadline="2028-06-15 23:45:40.84226134 +0000 UTC" Sep 17 11:38:50.299316 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:50.299235 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15300h6m50.543029442s" Sep 17 11:38:50.364013 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:50.363986 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 11:38:50.431981 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:50.431954 2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 11:38:50.450976 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:50.450949 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5a272a923bce547f142a7efecbeccfb.slice/crio-4a0f066e9de307d8a836d791a49fa9253095aed7433ecce654601ed17b6ee25b WatchSource:0}: Error finding container 4a0f066e9de307d8a836d791a49fa9253095aed7433ecce654601ed17b6ee25b: Status 404 returned error can't find the container with id 4a0f066e9de307d8a836d791a49fa9253095aed7433ecce654601ed17b6ee25b Sep 17 11:38:50.451369 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:50.451350 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73800decfd2c4467378c70aaf90595a7.slice/crio-44f9c0917ef0e7f715586fb24417a864f4776c9059fa9c35edf987dc8dc4a590 WatchSource:0}: Error finding container 44f9c0917ef0e7f715586fb24417a864f4776c9059fa9c35edf987dc8dc4a590: Status 404 returned error can't find the container with id 44f9c0917ef0e7f715586fb24417a864f4776c9059fa9c35edf987dc8dc4a590 Sep 17 11:38:50.456389 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:50.456372 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 11:38:52.296492 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.296001 2581 apiserver.go:52] "Watching apiserver" Sep 17 11:38:52.303294 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.303247 2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 17 11:38:52.305943 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.305912 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-sdxbb","openshift-multus/multus-additional-cni-plugins-z5n4q","openshift-multus/network-metrics-daemon-f5svv","openshift-network-diagnostics/network-check-target-pk9p9","kube-system/kube-apiserver-proxy-ip-10-0-140-196.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg","openshift-dns/node-resolver-wk6sh","openshift-image-registry/node-ca-g6kpv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal","openshift-multus/multus-mljj2","openshift-network-operator/iptables-alerter-qbxfn","openshift-ovn-kubernetes/ovnkube-node-nn75c","kube-system/global-pull-secret-syncer-6jmh4","kube-system/konnectivity-agent-z8klg"] Sep 17 11:38:52.308407 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.308381 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wk6sh" Sep 17 11:38:52.310797 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.310780 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qbxfn" Sep 17 11:38:52.311475 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.311456 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 17 11:38:52.311662 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.311648 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cswwl\"" Sep 17 11:38:52.311844 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.311831 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 17 11:38:52.314312 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.312885 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 17 11:38:52.314312 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.313103 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 17 11:38:52.314312 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.313448 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 17 11:38:52.314312 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.313706 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-sqvhh\"" Sep 17 11:38:52.314312 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.313994 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:38:52.314312 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:52.314081 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5svv" podUID="757a11fc-3750-4ef9-94f6-6c6da0d5c95c" Sep 17 11:38:52.315605 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.315582 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:38:52.315708 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:52.315665 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pk9p9" podUID="78be981b-e482-4dcf-a146-ea8549a3a847" Sep 17 11:38:52.318222 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.318201 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 17 11:38:52.319279 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.319259 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z8klg" Sep 17 11:38:52.321977 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.321481 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 17 11:38:52.321977 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.321815 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-c76j8\"" Sep 17 11:38:52.321977 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.321935 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 17 11:38:52.322220 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.322146 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.327955 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.325431 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 17 11:38:52.327955 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.325756 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hd9g8\"" Sep 17 11:38:52.327955 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.325960 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 17 11:38:52.327955 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.326248 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 17 11:38:52.327955 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.326650 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g6kpv" Sep 17 11:38:52.328711 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.328619 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bc2cd604-c635-41ec-a14e-cb5574805469-device-dir\") pod \"aws-ebs-csi-driver-node-pzqqg\" (UID: \"bc2cd604-c635-41ec-a14e-cb5574805469\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.328711 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.328662 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/26f77893-4701-4ac8-bd67-4df96b57deea-host-slash\") pod \"iptables-alerter-qbxfn\" (UID: \"26f77893-4701-4ac8-bd67-4df96b57deea\") " pod="openshift-network-operator/iptables-alerter-qbxfn" Sep 17 11:38:52.328872 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.328837 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:38:52.328948 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.328894 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bc2cd604-c635-41ec-a14e-cb5574805469-registration-dir\") pod \"aws-ebs-csi-driver-node-pzqqg\" (UID: \"bc2cd604-c635-41ec-a14e-cb5574805469\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.328948 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.328932 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bc2cd604-c635-41ec-a14e-cb5574805469-sys-fs\") pod \"aws-ebs-csi-driver-node-pzqqg\" (UID: \"bc2cd604-c635-41ec-a14e-cb5574805469\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.328948 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:52.328931 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jmh4" podUID="7a49d705-8c21-411b-8988-260b0158a9be" Sep 17 11:38:52.329102 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.328964 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e1152cde-f077-4e81-a902-d0cd724f0495-hosts-file\") pod \"node-resolver-wk6sh\" (UID: \"e1152cde-f077-4e81-a902-d0cd724f0495\") " pod="openshift-dns/node-resolver-wk6sh" Sep 17 11:38:52.329102 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.328998 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bc2cd604-c635-41ec-a14e-cb5574805469-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pzqqg\" (UID: \"bc2cd604-c635-41ec-a14e-cb5574805469\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.329102 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.329042 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mzjj\" (UniqueName: \"kubernetes.io/projected/bc2cd604-c635-41ec-a14e-cb5574805469-kube-api-access-5mzjj\") pod \"aws-ebs-csi-driver-node-pzqqg\" (UID: \"bc2cd604-c635-41ec-a14e-cb5574805469\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.329102 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.329071 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1152cde-f077-4e81-a902-d0cd724f0495-tmp-dir\") pod \"node-resolver-wk6sh\" (UID: \"e1152cde-f077-4e81-a902-d0cd724f0495\") " pod="openshift-dns/node-resolver-wk6sh" Sep 17 11:38:52.329309 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.329101 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl2x8\" (UniqueName: \"kubernetes.io/projected/e1152cde-f077-4e81-a902-d0cd724f0495-kube-api-access-nl2x8\") pod \"node-resolver-wk6sh\" (UID: \"e1152cde-f077-4e81-a902-d0cd724f0495\") " pod="openshift-dns/node-resolver-wk6sh" Sep 17 11:38:52.329309 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.329156 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/17023d9f-cb91-44aa-a873-3c5218fa56ea-konnectivity-ca\") pod \"konnectivity-agent-z8klg\" (UID: \"17023d9f-cb91-44aa-a873-3c5218fa56ea\") " pod="kube-system/konnectivity-agent-z8klg" Sep 17 11:38:52.329309 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.329189 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bc2cd604-c635-41ec-a14e-cb5574805469-socket-dir\") pod \"aws-ebs-csi-driver-node-pzqqg\" (UID: \"bc2cd604-c635-41ec-a14e-cb5574805469\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.329309 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.329227 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bc2cd604-c635-41ec-a14e-cb5574805469-etc-selinux\") pod \"aws-ebs-csi-driver-node-pzqqg\" (UID: \"bc2cd604-c635-41ec-a14e-cb5574805469\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.329309 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.329268 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/26f77893-4701-4ac8-bd67-4df96b57deea-iptables-alerter-script\") pod \"iptables-alerter-qbxfn\" (UID: \"26f77893-4701-4ac8-bd67-4df96b57deea\") " pod="openshift-network-operator/iptables-alerter-qbxfn" Sep 17 11:38:52.329309 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.329298 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqhwc\" (UniqueName: \"kubernetes.io/projected/26f77893-4701-4ac8-bd67-4df96b57deea-kube-api-access-dqhwc\") pod \"iptables-alerter-qbxfn\" (UID: \"26f77893-4701-4ac8-bd67-4df96b57deea\") " pod="openshift-network-operator/iptables-alerter-qbxfn" Sep 17 11:38:52.329593 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.329327 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs\") pod \"network-metrics-daemon-f5svv\" (UID: \"757a11fc-3750-4ef9-94f6-6c6da0d5c95c\") " pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:38:52.329593 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.329357 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84jxz\" (UniqueName: \"kubernetes.io/projected/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-kube-api-access-84jxz\") pod \"network-metrics-daemon-f5svv\" (UID: \"757a11fc-3750-4ef9-94f6-6c6da0d5c95c\") " pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:38:52.329593 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.329384 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxqzg\" (UniqueName: \"kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg\") pod \"network-check-target-pk9p9\" (UID: \"78be981b-e482-4dcf-a146-ea8549a3a847\") " pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:38:52.329593 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.329430 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/17023d9f-cb91-44aa-a873-3c5218fa56ea-agent-certs\") pod \"konnectivity-agent-z8klg\" (UID: \"17023d9f-cb91-44aa-a873-3c5218fa56ea\") " pod="kube-system/konnectivity-agent-z8klg" Sep 17 11:38:52.333148 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.330155 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 17 11:38:52.333148 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.330255 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 17 11:38:52.333148 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.330493 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 17 11:38:52.333148 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.330590 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qhchk\"" Sep 17 11:38:52.333148 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.330988 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.333148 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.333109 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 11:38:52.333565 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.333525 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.334592 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.334571 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nc9k4\"" Sep 17 11:38:52.334683 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.334657 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 17 11:38:52.334892 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.334874 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 17 11:38:52.334966 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.334940 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 17 11:38:52.335203 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.335180 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 17 11:38:52.335203 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.335200 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 17 11:38:52.336464 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.336448 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 17 11:38:52.336737 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.336541 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.336997 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.336968 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zxmzt\"" Sep 17 11:38:52.339075 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.339056 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.339271 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.339250 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 17 11:38:52.339630 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.339614 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bttjq\"" Sep 17 11:38:52.339812 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.339693 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 17 11:38:52.341414 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.341394 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 17 11:38:52.341800 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.341783 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 17 11:38:52.342512 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.342184 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 17 11:38:52.342512 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.342317 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 17 11:38:52.342512 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.342378 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x5cs7\"" Sep 17 11:38:52.342512 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.342408 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 17 11:38:52.342770 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.342716 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 17 11:38:52.401358 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.401333 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xfbmr" Sep 17 11:38:52.408397 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.408376 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xfbmr" Sep 17 11:38:52.418293 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.418276 2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 17 11:38:52.430603 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.430578 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-run-openvswitch\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.430712 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.430618 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-run-ovn\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.430712 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.430652 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bc2cd604-c635-41ec-a14e-cb5574805469-registration-dir\") pod \"aws-ebs-csi-driver-node-pzqqg\" (UID: \"bc2cd604-c635-41ec-a14e-cb5574805469\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.430712 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.430679 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e1152cde-f077-4e81-a902-d0cd724f0495-hosts-file\") pod \"node-resolver-wk6sh\" (UID: \"e1152cde-f077-4e81-a902-d0cd724f0495\") " pod="openshift-dns/node-resolver-wk6sh" Sep 17 11:38:52.430870 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.430732 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-host-run-multus-certs\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.430870 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.430750 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e1152cde-f077-4e81-a902-d0cd724f0495-hosts-file\") pod \"node-resolver-wk6sh\" (UID: \"e1152cde-f077-4e81-a902-d0cd724f0495\") " pod="openshift-dns/node-resolver-wk6sh" Sep 17 11:38:52.430870 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.430736 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bc2cd604-c635-41ec-a14e-cb5574805469-registration-dir\") pod \"aws-ebs-csi-driver-node-pzqqg\" (UID: \"bc2cd604-c635-41ec-a14e-cb5574805469\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.430870 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.430766 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-etc-modprobe-d\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.430870 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.430806 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-run\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.430870 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.430837 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bc2cd604-c635-41ec-a14e-cb5574805469-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pzqqg\" (UID: \"bc2cd604-c635-41ec-a14e-cb5574805469\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.430870 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.430860 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/67153d8b-662e-4d11-b932-2ec9f96be172-host\") pod \"node-ca-g6kpv\" (UID: \"67153d8b-662e-4d11-b932-2ec9f96be172\") " pod="openshift-image-registry/node-ca-g6kpv" Sep 17 11:38:52.431262 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.430886 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-host-run-netns\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.431262 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.430944 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bc2cd604-c635-41ec-a14e-cb5574805469-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pzqqg\" (UID: \"bc2cd604-c635-41ec-a14e-cb5574805469\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.431262 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.430980 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-host-var-lib-cni-bin\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.431262 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431006 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-host-run-ovn-kubernetes\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.431262 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431033 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-host-cni-netd\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.431262 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431055 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-etc-kubernetes\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.431262 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431082 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bc2cd604-c635-41ec-a14e-cb5574805469-etc-selinux\") pod \"aws-ebs-csi-driver-node-pzqqg\" (UID: \"bc2cd604-c635-41ec-a14e-cb5574805469\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.431262 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431104 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/67153d8b-662e-4d11-b932-2ec9f96be172-serviceca\") pod \"node-ca-g6kpv\" (UID: \"67153d8b-662e-4d11-b932-2ec9f96be172\") " pod="openshift-image-registry/node-ca-g6kpv" Sep 17 11:38:52.431262 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431147 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-multus-cni-dir\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.431262 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431190 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-tmp\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.431262 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431217 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bc2cd604-c635-41ec-a14e-cb5574805469-etc-selinux\") pod \"aws-ebs-csi-driver-node-pzqqg\" (UID: \"bc2cd604-c635-41ec-a14e-cb5574805469\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.431262 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431219 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eb0d7c07-75fb-4d19-b732-a1f9c737540a-env-overrides\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.431820 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431288 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/26f77893-4701-4ac8-bd67-4df96b57deea-iptables-alerter-script\") pod \"iptables-alerter-qbxfn\" (UID: \"26f77893-4701-4ac8-bd67-4df96b57deea\") " pod="openshift-network-operator/iptables-alerter-qbxfn" Sep 17 11:38:52.431820 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431314 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqhwc\" (UniqueName: \"kubernetes.io/projected/26f77893-4701-4ac8-bd67-4df96b57deea-kube-api-access-dqhwc\") pod \"iptables-alerter-qbxfn\" (UID: \"26f77893-4701-4ac8-bd67-4df96b57deea\") " pod="openshift-network-operator/iptables-alerter-qbxfn" Sep 17 11:38:52.431820 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431337 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84jxz\" (UniqueName: \"kubernetes.io/projected/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-kube-api-access-84jxz\") pod \"network-metrics-daemon-f5svv\" (UID: \"757a11fc-3750-4ef9-94f6-6c6da0d5c95c\") " pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:38:52.431820 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431365 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq5p7\" (UniqueName: \"kubernetes.io/projected/185c4318-ad3f-4a65-bdb4-4d8cbb554069-kube-api-access-rq5p7\") pod \"multus-additional-cni-plugins-z5n4q\" (UID: \"185c4318-ad3f-4a65-bdb4-4d8cbb554069\") " pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.431820 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431390 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-cnibin\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.431820 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431373 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-196.ec2.internal" event={"ID":"e5a272a923bce547f142a7efecbeccfb","Type":"ContainerStarted","Data":"4a0f066e9de307d8a836d791a49fa9253095aed7433ecce654601ed17b6ee25b"} Sep 17 11:38:52.431820 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431417 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7a49d705-8c21-411b-8988-260b0158a9be-kubelet-config\") pod \"global-pull-secret-syncer-6jmh4\" (UID: \"7a49d705-8c21-411b-8988-260b0158a9be\") " pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:38:52.431820 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431445 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7a49d705-8c21-411b-8988-260b0158a9be-dbus\") pod \"global-pull-secret-syncer-6jmh4\" (UID: \"7a49d705-8c21-411b-8988-260b0158a9be\") " pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:38:52.431820 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431482 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bc2cd604-c635-41ec-a14e-cb5574805469-sys-fs\") pod \"aws-ebs-csi-driver-node-pzqqg\" (UID: \"bc2cd604-c635-41ec-a14e-cb5574805469\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.431820 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431510 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snsnp\" (UniqueName: \"kubernetes.io/projected/4c40a380-1eb9-46c1-a161-a9973c9b916d-kube-api-access-snsnp\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.431820 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431571 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bc2cd604-c635-41ec-a14e-cb5574805469-sys-fs\") pod \"aws-ebs-csi-driver-node-pzqqg\" (UID: \"bc2cd604-c635-41ec-a14e-cb5574805469\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.431820 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431619 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-etc-sysctl-d\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.431820 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431670 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-lib-modules\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.431820 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431709 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.431820 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431738 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1152cde-f077-4e81-a902-d0cd724f0495-tmp-dir\") pod \"node-resolver-wk6sh\" (UID: \"e1152cde-f077-4e81-a902-d0cd724f0495\") " pod="openshift-dns/node-resolver-wk6sh" Sep 17 11:38:52.431820 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431771 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl2x8\" (UniqueName: \"kubernetes.io/projected/e1152cde-f077-4e81-a902-d0cd724f0495-kube-api-access-nl2x8\") pod \"node-resolver-wk6sh\" (UID: \"e1152cde-f077-4e81-a902-d0cd724f0495\") " pod="openshift-dns/node-resolver-wk6sh" Sep 17 11:38:52.431820 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431794 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-multus-conf-dir\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.432691 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431821 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn9vj\" (UniqueName: \"kubernetes.io/projected/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-kube-api-access-qn9vj\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.432691 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431843 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-run-systemd\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.432691 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431867 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-host-run-k8s-cni-cncf-io\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.432691 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431888 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-etc-sysctl-conf\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.432691 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431911 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-sys\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.432691 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431931 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-host\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.432691 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431952 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-log-socket\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.432691 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.431997 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/185c4318-ad3f-4a65-bdb4-4d8cbb554069-system-cni-dir\") pod \"multus-additional-cni-plugins-z5n4q\" (UID: \"185c4318-ad3f-4a65-bdb4-4d8cbb554069\") " pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.432691 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.432027 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs\") pod \"network-metrics-daemon-f5svv\" (UID: \"757a11fc-3750-4ef9-94f6-6c6da0d5c95c\") " pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:38:52.432691 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.432091 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1152cde-f077-4e81-a902-d0cd724f0495-tmp-dir\") pod \"node-resolver-wk6sh\" (UID: \"e1152cde-f077-4e81-a902-d0cd724f0495\") " pod="openshift-dns/node-resolver-wk6sh" Sep 17 11:38:52.432691 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.432313 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bc2cd604-c635-41ec-a14e-cb5574805469-device-dir\") pod \"aws-ebs-csi-driver-node-pzqqg\" (UID: \"bc2cd604-c635-41ec-a14e-cb5574805469\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.432691 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.432417 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/26f77893-4701-4ac8-bd67-4df96b57deea-iptables-alerter-script\") pod \"iptables-alerter-qbxfn\" (UID: \"26f77893-4701-4ac8-bd67-4df96b57deea\") " pod="openshift-network-operator/iptables-alerter-qbxfn" Sep 17 11:38:52.432691 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.432521 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bc2cd604-c635-41ec-a14e-cb5574805469-device-dir\") pod \"aws-ebs-csi-driver-node-pzqqg\" (UID: \"bc2cd604-c635-41ec-a14e-cb5574805469\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.432691 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.432556 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb0d7c07-75fb-4d19-b732-a1f9c737540a-ovn-node-metrics-cert\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.432691 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.432616 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/185c4318-ad3f-4a65-bdb4-4d8cbb554069-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z5n4q\" (UID: \"185c4318-ad3f-4a65-bdb4-4d8cbb554069\") " pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.432691 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.432670 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-hostroot\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.432691 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.432695 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-etc-systemd\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.433391 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:52.432706 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 11:38:52.433391 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.432716 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-systemd-units\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.433391 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.432756 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-var-lib-openvswitch\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.433391 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:52.432791 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs podName:757a11fc-3750-4ef9-94f6-6c6da0d5c95c nodeName:}" failed. No retries permitted until 2026-09-17 11:38:52.932759865 +0000 UTC m=+2.940206890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs") pod "network-metrics-daemon-f5svv" (UID: "757a11fc-3750-4ef9-94f6-6c6da0d5c95c") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 11:38:52.433391 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.432828 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdmts\" (UniqueName: \"kubernetes.io/projected/eb0d7c07-75fb-4d19-b732-a1f9c737540a-kube-api-access-qdmts\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.433391 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.432855 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-host-kubelet\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.433391 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.432885 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-node-log\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.433391 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.432931 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-etc-sysconfig\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.433391 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.432978 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bc2cd604-c635-41ec-a14e-cb5574805469-socket-dir\") pod \"aws-ebs-csi-driver-node-pzqqg\" (UID: \"bc2cd604-c635-41ec-a14e-cb5574805469\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.433391 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433000 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/185c4318-ad3f-4a65-bdb4-4d8cbb554069-cni-binary-copy\") pod \"multus-additional-cni-plugins-z5n4q\" (UID: \"185c4318-ad3f-4a65-bdb4-4d8cbb554069\") " pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.433391 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433016 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-system-cni-dir\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.433391 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433045 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4c40a380-1eb9-46c1-a161-a9973c9b916d-multus-daemon-config\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.433391 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433066 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-etc-kubernetes\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.433391 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433096 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-var-lib-kubelet\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.433391 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433149 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c40a380-1eb9-46c1-a161-a9973c9b916d-cni-binary-copy\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.433391 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433171 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bc2cd604-c635-41ec-a14e-cb5574805469-socket-dir\") pod \"aws-ebs-csi-driver-node-pzqqg\" (UID: \"bc2cd604-c635-41ec-a14e-cb5574805469\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.434137 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433179 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-host-var-lib-kubelet\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.434137 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433206 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/17023d9f-cb91-44aa-a873-3c5218fa56ea-agent-certs\") pod \"konnectivity-agent-z8klg\" (UID: \"17023d9f-cb91-44aa-a873-3c5218fa56ea\") " pod="kube-system/konnectivity-agent-z8klg" Sep 17 11:38:52.434137 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433245 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t9z9\" (UniqueName: \"kubernetes.io/projected/67153d8b-662e-4d11-b932-2ec9f96be172-kube-api-access-8t9z9\") pod \"node-ca-g6kpv\" (UID: \"67153d8b-662e-4d11-b932-2ec9f96be172\") " pod="openshift-image-registry/node-ca-g6kpv" Sep 17 11:38:52.434137 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433348 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/185c4318-ad3f-4a65-bdb4-4d8cbb554069-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z5n4q\" (UID: \"185c4318-ad3f-4a65-bdb4-4d8cbb554069\") " pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.434137 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433393 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-host-run-netns\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.434137 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433420 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-host-cni-bin\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.434137 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433465 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/26f77893-4701-4ac8-bd67-4df96b57deea-host-slash\") pod \"iptables-alerter-qbxfn\" (UID: \"26f77893-4701-4ac8-bd67-4df96b57deea\") " pod="openshift-network-operator/iptables-alerter-qbxfn" Sep 17 11:38:52.434137 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433522 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/26f77893-4701-4ac8-bd67-4df96b57deea-host-slash\") pod \"iptables-alerter-qbxfn\" (UID: \"26f77893-4701-4ac8-bd67-4df96b57deea\") " pod="openshift-network-operator/iptables-alerter-qbxfn" Sep 17 11:38:52.434137 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433523 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/185c4318-ad3f-4a65-bdb4-4d8cbb554069-cnibin\") pod \"multus-additional-cni-plugins-z5n4q\" (UID: \"185c4318-ad3f-4a65-bdb4-4d8cbb554069\") " pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.434137 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433556 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-host-slash\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.434137 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433587 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mzjj\" (UniqueName: \"kubernetes.io/projected/bc2cd604-c635-41ec-a14e-cb5574805469-kube-api-access-5mzjj\") pod \"aws-ebs-csi-driver-node-pzqqg\" (UID: \"bc2cd604-c635-41ec-a14e-cb5574805469\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.434137 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433649 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-host-var-lib-cni-multus\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.434137 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433702 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eb0d7c07-75fb-4d19-b732-a1f9c737540a-ovnkube-config\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.434137 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433730 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eb0d7c07-75fb-4d19-b732-a1f9c737540a-ovnkube-script-lib\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.434137 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433754 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret\") pod \"global-pull-secret-syncer-6jmh4\" (UID: \"7a49d705-8c21-411b-8988-260b0158a9be\") " pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:38:52.434137 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433814 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/17023d9f-cb91-44aa-a873-3c5218fa56ea-konnectivity-ca\") pod \"konnectivity-agent-z8klg\" (UID: \"17023d9f-cb91-44aa-a873-3c5218fa56ea\") " pod="kube-system/konnectivity-agent-z8klg" Sep 17 11:38:52.434746 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433847 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-multus-socket-dir-parent\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.434746 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433872 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-etc-openvswitch\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.434746 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433898 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxqzg\" (UniqueName: \"kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg\") pod \"network-check-target-pk9p9\" (UID: \"78be981b-e482-4dcf-a146-ea8549a3a847\") " pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:38:52.434746 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433923 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/185c4318-ad3f-4a65-bdb4-4d8cbb554069-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z5n4q\" (UID: \"185c4318-ad3f-4a65-bdb4-4d8cbb554069\") " pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.434746 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433967 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-etc-tuned\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.434746 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.433894 2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 17 11:38:52.434746 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.434372 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/17023d9f-cb91-44aa-a873-3c5218fa56ea-konnectivity-ca\") pod \"konnectivity-agent-z8klg\" (UID: \"17023d9f-cb91-44aa-a873-3c5218fa56ea\") " pod="kube-system/konnectivity-agent-z8klg" Sep 17 11:38:52.439462 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.439437 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal" event={"ID":"73800decfd2c4467378c70aaf90595a7","Type":"ContainerStarted","Data":"44f9c0917ef0e7f715586fb24417a864f4776c9059fa9c35edf987dc8dc4a590"} Sep 17 11:38:52.442304 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.442283 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/17023d9f-cb91-44aa-a873-3c5218fa56ea-agent-certs\") pod \"konnectivity-agent-z8klg\" (UID: \"17023d9f-cb91-44aa-a873-3c5218fa56ea\") " pod="kube-system/konnectivity-agent-z8klg" Sep 17 11:38:52.443062 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:52.443041 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 11:38:52.443168 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:52.443070 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 11:38:52.443168 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:52.443083 2581 projected.go:196] Error preparing data for projected volume kube-api-access-cxqzg for pod openshift-network-diagnostics/network-check-target-pk9p9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 11:38:52.443292 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:52.443170 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg podName:78be981b-e482-4dcf-a146-ea8549a3a847 nodeName:}" failed. No retries permitted until 2026-09-17 11:38:52.943149801 +0000 UTC m=+2.950596846 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cxqzg" (UniqueName: "kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg") pod "network-check-target-pk9p9" (UID: "78be981b-e482-4dcf-a146-ea8549a3a847") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 11:38:52.443983 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.443940 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqhwc\" (UniqueName: \"kubernetes.io/projected/26f77893-4701-4ac8-bd67-4df96b57deea-kube-api-access-dqhwc\") pod \"iptables-alerter-qbxfn\" (UID: \"26f77893-4701-4ac8-bd67-4df96b57deea\") " pod="openshift-network-operator/iptables-alerter-qbxfn" Sep 17 11:38:52.445396 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.445376 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl2x8\" (UniqueName: \"kubernetes.io/projected/e1152cde-f077-4e81-a902-d0cd724f0495-kube-api-access-nl2x8\") pod \"node-resolver-wk6sh\" (UID: \"e1152cde-f077-4e81-a902-d0cd724f0495\") " pod="openshift-dns/node-resolver-wk6sh" Sep 17 11:38:52.448552 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.448514 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84jxz\" (UniqueName: \"kubernetes.io/projected/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-kube-api-access-84jxz\") pod \"network-metrics-daemon-f5svv\" (UID: \"757a11fc-3750-4ef9-94f6-6c6da0d5c95c\") " pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:38:52.451020 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.451001 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mzjj\" (UniqueName: \"kubernetes.io/projected/bc2cd604-c635-41ec-a14e-cb5574805469-kube-api-access-5mzjj\") pod \"aws-ebs-csi-driver-node-pzqqg\" (UID: \"bc2cd604-c635-41ec-a14e-cb5574805469\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.534899 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.534866 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/185c4318-ad3f-4a65-bdb4-4d8cbb554069-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z5n4q\" (UID: \"185c4318-ad3f-4a65-bdb4-4d8cbb554069\") " pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.535088 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.534910 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-host-run-netns\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.535088 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.534937 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-host-cni-bin\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.535088 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.534963 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/185c4318-ad3f-4a65-bdb4-4d8cbb554069-cnibin\") pod \"multus-additional-cni-plugins-z5n4q\" (UID: \"185c4318-ad3f-4a65-bdb4-4d8cbb554069\") " pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.535088 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.534985 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-host-slash\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.535088 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535012 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-host-var-lib-cni-multus\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.535088 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535036 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eb0d7c07-75fb-4d19-b732-a1f9c737540a-ovnkube-config\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.535088 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535058 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eb0d7c07-75fb-4d19-b732-a1f9c737540a-ovnkube-script-lib\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.535088 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535082 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret\") pod \"global-pull-secret-syncer-6jmh4\" (UID: \"7a49d705-8c21-411b-8988-260b0158a9be\") " pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:38:52.535443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535106 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-multus-socket-dir-parent\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.535443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535151 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-etc-openvswitch\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.535443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535189 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/185c4318-ad3f-4a65-bdb4-4d8cbb554069-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z5n4q\" (UID: \"185c4318-ad3f-4a65-bdb4-4d8cbb554069\") " pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.535443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535203 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-host-slash\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.535443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535214 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-etc-tuned\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.535443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535214 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-host-run-netns\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.535443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535266 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-host-var-lib-cni-multus\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.535443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535278 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-host-cni-bin\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.535443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535297 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-run-openvswitch\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.535443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535307 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/185c4318-ad3f-4a65-bdb4-4d8cbb554069-cnibin\") pod \"multus-additional-cni-plugins-z5n4q\" (UID: \"185c4318-ad3f-4a65-bdb4-4d8cbb554069\") " pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.535443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535316 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-run-ovn\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.535443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535333 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-host-run-multus-certs\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.535443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535349 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-etc-modprobe-d\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.535443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535363 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-run\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.535443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535377 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/67153d8b-662e-4d11-b932-2ec9f96be172-host\") pod \"node-ca-g6kpv\" (UID: \"67153d8b-662e-4d11-b932-2ec9f96be172\") " pod="openshift-image-registry/node-ca-g6kpv" Sep 17 11:38:52.535443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535391 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-host-run-netns\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.535443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535407 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-host-var-lib-cni-bin\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.535443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535422 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-host-run-ovn-kubernetes\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.536498 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535435 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-host-cni-netd\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.536498 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535449 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-etc-kubernetes\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.536498 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535463 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/67153d8b-662e-4d11-b932-2ec9f96be172-serviceca\") pod \"node-ca-g6kpv\" (UID: \"67153d8b-662e-4d11-b932-2ec9f96be172\") " pod="openshift-image-registry/node-ca-g6kpv" Sep 17 11:38:52.536498 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535478 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-multus-cni-dir\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.536498 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535493 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-tmp\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.536498 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535507 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eb0d7c07-75fb-4d19-b732-a1f9c737540a-env-overrides\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.536498 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535524 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq5p7\" (UniqueName: \"kubernetes.io/projected/185c4318-ad3f-4a65-bdb4-4d8cbb554069-kube-api-access-rq5p7\") pod \"multus-additional-cni-plugins-z5n4q\" (UID: \"185c4318-ad3f-4a65-bdb4-4d8cbb554069\") " pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.536498 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535539 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-cnibin\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.536498 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535553 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7a49d705-8c21-411b-8988-260b0158a9be-kubelet-config\") pod \"global-pull-secret-syncer-6jmh4\" (UID: \"7a49d705-8c21-411b-8988-260b0158a9be\") " pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:38:52.536498 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535567 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7a49d705-8c21-411b-8988-260b0158a9be-dbus\") pod \"global-pull-secret-syncer-6jmh4\" (UID: \"7a49d705-8c21-411b-8988-260b0158a9be\") " pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:38:52.536498 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535583 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snsnp\" (UniqueName: \"kubernetes.io/projected/4c40a380-1eb9-46c1-a161-a9973c9b916d-kube-api-access-snsnp\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.536498 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535603 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-etc-sysctl-d\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.536498 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535617 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-lib-modules\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.536498 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535644 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.536498 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535661 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-run-openvswitch\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.536498 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535674 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-multus-conf-dir\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.536498 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535700 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn9vj\" (UniqueName: \"kubernetes.io/projected/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-kube-api-access-qn9vj\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.537314 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535708 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-run-ovn\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.537314 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535729 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-run-systemd\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.537314 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535754 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-host-run-k8s-cni-cncf-io\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.537314 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535759 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/185c4318-ad3f-4a65-bdb4-4d8cbb554069-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z5n4q\" (UID: \"185c4318-ad3f-4a65-bdb4-4d8cbb554069\") " pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.537314 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535779 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-etc-sysctl-conf\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.537314 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535796 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-etc-modprobe-d\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.537314 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535801 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-sys\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.537314 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535818 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-host\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.537314 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535832 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-log-socket\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.537314 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535848 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/185c4318-ad3f-4a65-bdb4-4d8cbb554069-system-cni-dir\") pod \"multus-additional-cni-plugins-z5n4q\" (UID: \"185c4318-ad3f-4a65-bdb4-4d8cbb554069\") " pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.537314 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535863 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-run\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.537314 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535877 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb0d7c07-75fb-4d19-b732-a1f9c737540a-ovn-node-metrics-cert\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.537314 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535885 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/67153d8b-662e-4d11-b932-2ec9f96be172-host\") pod \"node-ca-g6kpv\" (UID: \"67153d8b-662e-4d11-b932-2ec9f96be172\") " pod="openshift-image-registry/node-ca-g6kpv" Sep 17 11:38:52.537314 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535892 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/185c4318-ad3f-4a65-bdb4-4d8cbb554069-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z5n4q\" (UID: \"185c4318-ad3f-4a65-bdb4-4d8cbb554069\") " pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.537314 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535907 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-host-run-netns\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.537314 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535911 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-hostroot\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.537314 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535926 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-etc-systemd\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.537314 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535929 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-host-var-lib-cni-bin\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.538334 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535967 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-etc-systemd\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.538334 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535948 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-systemd-units\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.538334 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535996 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-var-lib-openvswitch\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.538334 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536013 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdmts\" (UniqueName: \"kubernetes.io/projected/eb0d7c07-75fb-4d19-b732-a1f9c737540a-kube-api-access-qdmts\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.538334 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536029 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-host-kubelet\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.538334 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536043 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-node-log\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.538334 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536058 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-etc-sysconfig\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.538334 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536076 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/185c4318-ad3f-4a65-bdb4-4d8cbb554069-cni-binary-copy\") pod \"multus-additional-cni-plugins-z5n4q\" (UID: \"185c4318-ad3f-4a65-bdb4-4d8cbb554069\") " pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.538334 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536092 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-system-cni-dir\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.538334 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536106 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4c40a380-1eb9-46c1-a161-a9973c9b916d-multus-daemon-config\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.538334 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536157 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eb0d7c07-75fb-4d19-b732-a1f9c737540a-ovnkube-config\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.538334 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536189 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-etc-kubernetes\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.538334 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536206 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-var-lib-kubelet\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.538334 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536228 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c40a380-1eb9-46c1-a161-a9973c9b916d-cni-binary-copy\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.538334 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536258 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-host-var-lib-kubelet\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.538334 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536286 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t9z9\" (UniqueName: \"kubernetes.io/projected/67153d8b-662e-4d11-b932-2ec9f96be172-kube-api-access-8t9z9\") pod \"node-ca-g6kpv\" (UID: \"67153d8b-662e-4d11-b932-2ec9f96be172\") " pod="openshift-image-registry/node-ca-g6kpv" Sep 17 11:38:52.538334 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536324 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/67153d8b-662e-4d11-b932-2ec9f96be172-serviceca\") pod \"node-ca-g6kpv\" (UID: \"67153d8b-662e-4d11-b932-2ec9f96be172\") " pod="openshift-image-registry/node-ca-g6kpv" Sep 17 11:38:52.538334 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536374 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-multus-cni-dir\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.538900 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536534 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-host-run-ovn-kubernetes\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.538900 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536565 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-host-cni-netd\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.538900 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536589 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-systemd-units\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.538900 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536614 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-var-lib-openvswitch\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.538900 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536609 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eb0d7c07-75fb-4d19-b732-a1f9c737540a-ovnkube-script-lib\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.538900 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:52.536699 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 11:38:52.538900 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536722 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-host-kubelet\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.538900 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536748 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-node-log\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.538900 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:52.536753 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret podName:7a49d705-8c21-411b-8988-260b0158a9be nodeName:}" failed. No retries permitted until 2026-09-17 11:38:53.036737001 +0000 UTC m=+3.044184015 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret") pod "global-pull-secret-syncer-6jmh4" (UID: "7a49d705-8c21-411b-8988-260b0158a9be") : object "kube-system"/"original-pull-secret" not registered Sep 17 11:38:52.538900 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536810 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-multus-socket-dir-parent\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.538900 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536855 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-var-lib-kubelet\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.538900 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.536885 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-etc-openvswitch\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.538900 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537061 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eb0d7c07-75fb-4d19-b732-a1f9c737540a-env-overrides\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.538900 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537203 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-system-cni-dir\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.538900 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537211 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/185c4318-ad3f-4a65-bdb4-4d8cbb554069-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z5n4q\" (UID: \"185c4318-ad3f-4a65-bdb4-4d8cbb554069\") " pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.538900 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537222 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c40a380-1eb9-46c1-a161-a9973c9b916d-cni-binary-copy\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.538900 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537263 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-host-var-lib-kubelet\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.539559 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537326 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-lib-modules\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.539559 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537358 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-cnibin\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.539559 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537368 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-etc-kubernetes\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.539559 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537414 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.539559 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537442 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-multus-conf-dir\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.539559 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537479 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-etc-sysconfig\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.539559 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537554 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-etc-sysctl-d\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.539559 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537643 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7a49d705-8c21-411b-8988-260b0158a9be-dbus\") pod \"global-pull-secret-syncer-6jmh4\" (UID: \"7a49d705-8c21-411b-8988-260b0158a9be\") " pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:38:52.539559 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537677 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-sys\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.539559 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537704 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-run-systemd\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.539559 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.535732 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-host-run-multus-certs\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.539559 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537732 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-host-run-k8s-cni-cncf-io\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.539559 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537759 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4c40a380-1eb9-46c1-a161-a9973c9b916d-multus-daemon-config\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.539559 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537804 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7a49d705-8c21-411b-8988-260b0158a9be-kubelet-config\") pod \"global-pull-secret-syncer-6jmh4\" (UID: \"7a49d705-8c21-411b-8988-260b0158a9be\") " pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:38:52.539559 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537812 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-etc-sysctl-conf\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.539559 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537815 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/185c4318-ad3f-4a65-bdb4-4d8cbb554069-cni-binary-copy\") pod \"multus-additional-cni-plugins-z5n4q\" (UID: \"185c4318-ad3f-4a65-bdb4-4d8cbb554069\") " pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.539559 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537828 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-etc-kubernetes\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.539559 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537844 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/185c4318-ad3f-4a65-bdb4-4d8cbb554069-system-cni-dir\") pod \"multus-additional-cni-plugins-z5n4q\" (UID: \"185c4318-ad3f-4a65-bdb4-4d8cbb554069\") " pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.540400 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537857 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4c40a380-1eb9-46c1-a161-a9973c9b916d-hostroot\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.540400 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537874 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-host\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.540400 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537898 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eb0d7c07-75fb-4d19-b732-a1f9c737540a-log-socket\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.540400 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.537919 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/185c4318-ad3f-4a65-bdb4-4d8cbb554069-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z5n4q\" (UID: \"185c4318-ad3f-4a65-bdb4-4d8cbb554069\") " pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.540400 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.538453 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-etc-tuned\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.540400 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.540133 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb0d7c07-75fb-4d19-b732-a1f9c737540a-ovn-node-metrics-cert\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.540400 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.540301 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-tmp\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.546008 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.545965 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdmts\" (UniqueName: \"kubernetes.io/projected/eb0d7c07-75fb-4d19-b732-a1f9c737540a-kube-api-access-qdmts\") pod \"ovnkube-node-nn75c\" (UID: \"eb0d7c07-75fb-4d19-b732-a1f9c737540a\") " pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.546561 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.546509 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snsnp\" (UniqueName: \"kubernetes.io/projected/4c40a380-1eb9-46c1-a161-a9973c9b916d-kube-api-access-snsnp\") pod \"multus-mljj2\" (UID: \"4c40a380-1eb9-46c1-a161-a9973c9b916d\") " pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.547060 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.547036 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t9z9\" (UniqueName: \"kubernetes.io/projected/67153d8b-662e-4d11-b932-2ec9f96be172-kube-api-access-8t9z9\") pod \"node-ca-g6kpv\" (UID: \"67153d8b-662e-4d11-b932-2ec9f96be172\") " pod="openshift-image-registry/node-ca-g6kpv" Sep 17 11:38:52.547791 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.547769 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn9vj\" (UniqueName: \"kubernetes.io/projected/29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0-kube-api-access-qn9vj\") pod \"tuned-sdxbb\" (UID: \"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.549082 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.549063 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq5p7\" (UniqueName: \"kubernetes.io/projected/185c4318-ad3f-4a65-bdb4-4d8cbb554069-kube-api-access-rq5p7\") pod \"multus-additional-cni-plugins-z5n4q\" (UID: \"185c4318-ad3f-4a65-bdb4-4d8cbb554069\") " pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.626037 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.626002 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wk6sh" Sep 17 11:38:52.637079 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.637055 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qbxfn" Sep 17 11:38:52.639436 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:52.639407 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1152cde_f077_4e81_a902_d0cd724f0495.slice/crio-07c27af3a5d82de82412566cb4572a861e74bad9f105a462d609a010538f3225 WatchSource:0}: Error finding container 07c27af3a5d82de82412566cb4572a861e74bad9f105a462d609a010538f3225: Status 404 returned error can't find the container with id 07c27af3a5d82de82412566cb4572a861e74bad9f105a462d609a010538f3225 Sep 17 11:38:52.644725 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.644703 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z8klg" Sep 17 11:38:52.647220 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:52.647189 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26f77893_4701_4ac8_bd67_4df96b57deea.slice/crio-0e2d170e2b7e2f462b35b13bc55669685facf5f39dfec3c03d4af2b96738aa57 WatchSource:0}: Error finding container 0e2d170e2b7e2f462b35b13bc55669685facf5f39dfec3c03d4af2b96738aa57: Status 404 returned error can't find the container with id 0e2d170e2b7e2f462b35b13bc55669685facf5f39dfec3c03d4af2b96738aa57 Sep 17 11:38:52.653098 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.653047 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" Sep 17 11:38:52.654236 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:52.654214 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17023d9f_cb91_44aa_a873_3c5218fa56ea.slice/crio-1c5863f027d77bb5e6ce2ac40f6d2c1f199af1b6e1ca209960760ebd163c4168 WatchSource:0}: Error finding container 1c5863f027d77bb5e6ce2ac40f6d2c1f199af1b6e1ca209960760ebd163c4168: Status 404 returned error can't find the container with id 1c5863f027d77bb5e6ce2ac40f6d2c1f199af1b6e1ca209960760ebd163c4168 Sep 17 11:38:52.661781 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.661616 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g6kpv" Sep 17 11:38:52.662569 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:52.662537 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc2cd604_c635_41ec_a14e_cb5574805469.slice/crio-fe12cf61ae379b0f370561d3113170243357a7b5a295b790a0434f63d895afb9 WatchSource:0}: Error finding container fe12cf61ae379b0f370561d3113170243357a7b5a295b790a0434f63d895afb9: Status 404 returned error can't find the container with id fe12cf61ae379b0f370561d3113170243357a7b5a295b790a0434f63d895afb9 Sep 17 11:38:52.668505 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.668479 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z5n4q" Sep 17 11:38:52.673320 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:52.673289 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67153d8b_662e_4d11_b932_2ec9f96be172.slice/crio-24b127c3aa8f55e3954c649fd5ae29306529891b470a0116088cadd204a539e1 WatchSource:0}: Error finding container 24b127c3aa8f55e3954c649fd5ae29306529891b470a0116088cadd204a539e1: Status 404 returned error can't find the container with id 24b127c3aa8f55e3954c649fd5ae29306529891b470a0116088cadd204a539e1 Sep 17 11:38:52.677024 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.676925 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mljj2" Sep 17 11:38:52.678586 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:52.678548 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod185c4318_ad3f_4a65_bdb4_4d8cbb554069.slice/crio-3c16d40fe63b873eeb9f65c97a96e8d747837a67834b33f70e4fff1fd9eece42 WatchSource:0}: Error finding container 3c16d40fe63b873eeb9f65c97a96e8d747837a67834b33f70e4fff1fd9eece42: Status 404 returned error can't find the container with id 3c16d40fe63b873eeb9f65c97a96e8d747837a67834b33f70e4fff1fd9eece42 Sep 17 11:38:52.685385 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.685222 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" Sep 17 11:38:52.687307 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:52.687283 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c40a380_1eb9_46c1_a161_a9973c9b916d.slice/crio-c7b7ddafc8a82b271373b286f404b12ea45dfff35daef30c09bc2e8a16fe1c61 WatchSource:0}: Error finding container c7b7ddafc8a82b271373b286f404b12ea45dfff35daef30c09bc2e8a16fe1c61: Status 404 returned error can't find the container with id c7b7ddafc8a82b271373b286f404b12ea45dfff35daef30c09bc2e8a16fe1c61 Sep 17 11:38:52.693611 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.693588 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:38:52.695202 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:52.695175 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29bdca3f_fa73_4fad_a99e_1d8dcf4a59f0.slice/crio-76cb9efc70fcec783bc41d2b091f2d6536708948168a2821c1930717d8e2eb52 WatchSource:0}: Error finding container 76cb9efc70fcec783bc41d2b091f2d6536708948168a2821c1930717d8e2eb52: Status 404 returned error can't find the container with id 76cb9efc70fcec783bc41d2b091f2d6536708948168a2821c1930717d8e2eb52 Sep 17 11:38:52.703763 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:38:52.703742 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb0d7c07_75fb_4d19_b732_a1f9c737540a.slice/crio-afcc30d9caf572dc6144f285aeb12ba6ac3ae85189917228d299b583e6a115b3 WatchSource:0}: Error finding container afcc30d9caf572dc6144f285aeb12ba6ac3ae85189917228d299b583e6a115b3: Status 404 returned error can't find the container with id afcc30d9caf572dc6144f285aeb12ba6ac3ae85189917228d299b583e6a115b3 Sep 17 11:38:52.939221 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:52.939112 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs\") pod \"network-metrics-daemon-f5svv\" (UID: \"757a11fc-3750-4ef9-94f6-6c6da0d5c95c\") " pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:38:52.939362 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:52.939255 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 11:38:52.939362 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:52.939344 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs podName:757a11fc-3750-4ef9-94f6-6c6da0d5c95c nodeName:}" failed. No retries permitted until 2026-09-17 11:38:53.939323942 +0000 UTC m=+3.946770964 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs") pod "network-metrics-daemon-f5svv" (UID: "757a11fc-3750-4ef9-94f6-6c6da0d5c95c") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 11:38:53.040239 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:53.040196 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret\") pod \"global-pull-secret-syncer-6jmh4\" (UID: \"7a49d705-8c21-411b-8988-260b0158a9be\") " pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:38:53.040239 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:53.040244 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxqzg\" (UniqueName: \"kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg\") pod \"network-check-target-pk9p9\" (UID: \"78be981b-e482-4dcf-a146-ea8549a3a847\") " pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:38:53.040437 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:53.040368 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 11:38:53.040437 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:53.040383 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 11:38:53.040437 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:53.040392 2581 projected.go:196] Error preparing data for projected volume kube-api-access-cxqzg for pod openshift-network-diagnostics/network-check-target-pk9p9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 11:38:53.040437 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:53.040390 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 11:38:53.040437 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:53.040437 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg podName:78be981b-e482-4dcf-a146-ea8549a3a847 nodeName:}" failed. No retries permitted until 2026-09-17 11:38:54.040424616 +0000 UTC m=+4.047871624 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxqzg" (UniqueName: "kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg") pod "network-check-target-pk9p9" (UID: "78be981b-e482-4dcf-a146-ea8549a3a847") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 11:38:53.040629 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:53.040451 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret podName:7a49d705-8c21-411b-8988-260b0158a9be nodeName:}" failed. No retries permitted until 2026-09-17 11:38:54.040445629 +0000 UTC m=+4.047892637 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret") pod "global-pull-secret-syncer-6jmh4" (UID: "7a49d705-8c21-411b-8988-260b0158a9be") : object "kube-system"/"original-pull-secret" not registered Sep 17 11:38:53.411687 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:53.411569 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 11:33:52 +0000 UTC" deadline="2028-02-26 05:35:11.445232352 +0000 UTC" Sep 17 11:38:53.411687 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:53.411643 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12641h56m18.033593977s" Sep 17 11:38:53.443231 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:53.443149 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mljj2" event={"ID":"4c40a380-1eb9-46c1-a161-a9973c9b916d","Type":"ContainerStarted","Data":"c7b7ddafc8a82b271373b286f404b12ea45dfff35daef30c09bc2e8a16fe1c61"} Sep 17 11:38:53.445199 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:53.445160 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5n4q" event={"ID":"185c4318-ad3f-4a65-bdb4-4d8cbb554069","Type":"ContainerStarted","Data":"3c16d40fe63b873eeb9f65c97a96e8d747837a67834b33f70e4fff1fd9eece42"} Sep 17 11:38:53.447238 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:53.447216 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g6kpv" event={"ID":"67153d8b-662e-4d11-b932-2ec9f96be172","Type":"ContainerStarted","Data":"24b127c3aa8f55e3954c649fd5ae29306529891b470a0116088cadd204a539e1"} Sep 17 11:38:53.448768 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:53.448740 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" event={"ID":"bc2cd604-c635-41ec-a14e-cb5574805469","Type":"ContainerStarted","Data":"fe12cf61ae379b0f370561d3113170243357a7b5a295b790a0434f63d895afb9"} Sep 17 11:38:53.452199 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:53.452176 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z8klg" event={"ID":"17023d9f-cb91-44aa-a873-3c5218fa56ea","Type":"ContainerStarted","Data":"1c5863f027d77bb5e6ce2ac40f6d2c1f199af1b6e1ca209960760ebd163c4168"} Sep 17 11:38:53.457318 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:53.454325 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qbxfn" event={"ID":"26f77893-4701-4ac8-bd67-4df96b57deea","Type":"ContainerStarted","Data":"0e2d170e2b7e2f462b35b13bc55669685facf5f39dfec3c03d4af2b96738aa57"} Sep 17 11:38:53.458342 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:53.458230 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" event={"ID":"eb0d7c07-75fb-4d19-b732-a1f9c737540a","Type":"ContainerStarted","Data":"afcc30d9caf572dc6144f285aeb12ba6ac3ae85189917228d299b583e6a115b3"} Sep 17 11:38:53.460021 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:53.459993 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" event={"ID":"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0","Type":"ContainerStarted","Data":"76cb9efc70fcec783bc41d2b091f2d6536708948168a2821c1930717d8e2eb52"} Sep 17 11:38:53.461574 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:53.461529 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wk6sh" event={"ID":"e1152cde-f077-4e81-a902-d0cd724f0495","Type":"ContainerStarted","Data":"07c27af3a5d82de82412566cb4572a861e74bad9f105a462d609a010538f3225"} Sep 17 11:38:53.949550 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:53.948889 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs\") pod \"network-metrics-daemon-f5svv\" (UID: \"757a11fc-3750-4ef9-94f6-6c6da0d5c95c\") " pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:38:53.949550 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:53.949040 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 11:38:53.949550 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:53.949130 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs podName:757a11fc-3750-4ef9-94f6-6c6da0d5c95c nodeName:}" failed. No retries permitted until 2026-09-17 11:38:55.949096388 +0000 UTC m=+5.956543410 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs") pod "network-metrics-daemon-f5svv" (UID: "757a11fc-3750-4ef9-94f6-6c6da0d5c95c") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 11:38:54.049415 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:54.049374 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret\") pod \"global-pull-secret-syncer-6jmh4\" (UID: \"7a49d705-8c21-411b-8988-260b0158a9be\") " pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:38:54.049703 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:54.049427 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxqzg\" (UniqueName: \"kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg\") pod \"network-check-target-pk9p9\" (UID: \"78be981b-e482-4dcf-a146-ea8549a3a847\") " pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:38:54.049703 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:54.049580 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 11:38:54.049703 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:54.049599 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 11:38:54.049703 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:54.049612 2581 projected.go:196] Error preparing data for projected volume kube-api-access-cxqzg for pod openshift-network-diagnostics/network-check-target-pk9p9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 11:38:54.049703 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:54.049672 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg podName:78be981b-e482-4dcf-a146-ea8549a3a847 nodeName:}" failed. No retries permitted until 2026-09-17 11:38:56.049653512 +0000 UTC m=+6.057100537 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxqzg" (UniqueName: "kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg") pod "network-check-target-pk9p9" (UID: "78be981b-e482-4dcf-a146-ea8549a3a847") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 11:38:54.051208 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:54.050076 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 11:38:54.051208 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:54.050147 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret podName:7a49d705-8c21-411b-8988-260b0158a9be nodeName:}" failed. No retries permitted until 2026-09-17 11:38:56.050131867 +0000 UTC m=+6.057578893 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret") pod "global-pull-secret-syncer-6jmh4" (UID: "7a49d705-8c21-411b-8988-260b0158a9be") : object "kube-system"/"original-pull-secret" not registered Sep 17 11:38:54.412673 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:54.412564 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 11:33:52 +0000 UTC" deadline="2028-05-30 14:54:40.6124213 +0000 UTC" Sep 17 11:38:54.412673 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:54.412626 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14907h15m46.19979887s" Sep 17 11:38:54.425450 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:54.425422 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:38:54.425592 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:54.425558 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5svv" podUID="757a11fc-3750-4ef9-94f6-6c6da0d5c95c" Sep 17 11:38:54.425967 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:54.425946 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:38:54.426069 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:54.426045 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jmh4" podUID="7a49d705-8c21-411b-8988-260b0158a9be" Sep 17 11:38:54.426181 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:54.426136 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:38:54.426275 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:54.426207 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pk9p9" podUID="78be981b-e482-4dcf-a146-ea8549a3a847" Sep 17 11:38:54.447816 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:54.447733 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 11:38:55.481418 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:55.481381 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal" event={"ID":"73800decfd2c4467378c70aaf90595a7","Type":"ContainerStarted","Data":"5eb01da46b9206a402f2b17214469f7bcc0a4e86cae51a63b57be8119fd21613"} Sep 17 11:38:55.488782 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:55.488749 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-196.ec2.internal" event={"ID":"e5a272a923bce547f142a7efecbeccfb","Type":"ContainerStarted","Data":"a4e2ea11a48dde373a90d8dc760798ca0d8ff93d5d445c2f07524c5bb4915793"} Sep 17 11:38:55.963377 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:55.963305 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs\") pod \"network-metrics-daemon-f5svv\" (UID: \"757a11fc-3750-4ef9-94f6-6c6da0d5c95c\") " pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:38:55.963564 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:55.963507 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 11:38:55.963620 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:55.963575 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs podName:757a11fc-3750-4ef9-94f6-6c6da0d5c95c nodeName:}" failed. No retries permitted until 2026-09-17 11:38:59.963556312 +0000 UTC m=+9.971003348 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs") pod "network-metrics-daemon-f5svv" (UID: "757a11fc-3750-4ef9-94f6-6c6da0d5c95c") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 11:38:56.064214 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:56.064167 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret\") pod \"global-pull-secret-syncer-6jmh4\" (UID: \"7a49d705-8c21-411b-8988-260b0158a9be\") " pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:38:56.064397 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:56.064223 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxqzg\" (UniqueName: \"kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg\") pod \"network-check-target-pk9p9\" (UID: \"78be981b-e482-4dcf-a146-ea8549a3a847\") " pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:38:56.064473 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:56.064402 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 11:38:56.064473 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:56.064424 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 11:38:56.064473 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:56.064438 2581 projected.go:196] Error preparing data for projected volume kube-api-access-cxqzg for pod openshift-network-diagnostics/network-check-target-pk9p9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 11:38:56.064614 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:56.064523 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg podName:78be981b-e482-4dcf-a146-ea8549a3a847 nodeName:}" failed. No retries permitted until 2026-09-17 11:39:00.064500302 +0000 UTC m=+10.071947326 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxqzg" (UniqueName: "kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg") pod "network-check-target-pk9p9" (UID: "78be981b-e482-4dcf-a146-ea8549a3a847") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 11:38:56.064930 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:56.064910 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 11:38:56.065004 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:56.064965 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret podName:7a49d705-8c21-411b-8988-260b0158a9be nodeName:}" failed. No retries permitted until 2026-09-17 11:39:00.064950627 +0000 UTC m=+10.072397641 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret") pod "global-pull-secret-syncer-6jmh4" (UID: "7a49d705-8c21-411b-8988-260b0158a9be") : object "kube-system"/"original-pull-secret" not registered Sep 17 11:38:56.437351 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:56.437264 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:38:56.437501 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:56.437452 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5svv" podUID="757a11fc-3750-4ef9-94f6-6c6da0d5c95c" Sep 17 11:38:56.437905 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:56.437886 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:38:56.438024 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:56.438009 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jmh4" podUID="7a49d705-8c21-411b-8988-260b0158a9be" Sep 17 11:38:56.438099 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:56.438074 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:38:56.438234 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:56.438173 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pk9p9" podUID="78be981b-e482-4dcf-a146-ea8549a3a847" Sep 17 11:38:57.496746 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:57.496713 2581 generic.go:356] "Generic (PLEG): container finished" podID="73800decfd2c4467378c70aaf90595a7" containerID="5eb01da46b9206a402f2b17214469f7bcc0a4e86cae51a63b57be8119fd21613" exitCode=0 Sep 17 11:38:57.497110 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:57.496761 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal" event={"ID":"73800decfd2c4467378c70aaf90595a7","Type":"ContainerDied","Data":"5eb01da46b9206a402f2b17214469f7bcc0a4e86cae51a63b57be8119fd21613"} Sep 17 11:38:57.569171 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:57.569092 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-196.ec2.internal" podStartSLOduration=6.569071861 podStartE2EDuration="6.569071861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 11:38:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:38:55.547389378 +0000 UTC m=+5.554836411" watchObservedRunningTime="2026-09-17 11:38:57.569071861 +0000 UTC m=+7.576518891" Sep 17 11:38:58.423093 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:58.423057 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:38:58.423293 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:58.423215 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5svv" podUID="757a11fc-3750-4ef9-94f6-6c6da0d5c95c" Sep 17 11:38:58.423710 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:58.423573 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:38:58.423710 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:58.423664 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jmh4" podUID="7a49d705-8c21-411b-8988-260b0158a9be" Sep 17 11:38:58.423878 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:58.423811 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:38:58.423935 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:58.423912 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pk9p9" podUID="78be981b-e482-4dcf-a146-ea8549a3a847" Sep 17 11:38:59.995428 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:38:59.995191 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs\") pod \"network-metrics-daemon-f5svv\" (UID: \"757a11fc-3750-4ef9-94f6-6c6da0d5c95c\") " pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:38:59.995855 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:59.995407 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 11:38:59.995855 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:38:59.995523 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs podName:757a11fc-3750-4ef9-94f6-6c6da0d5c95c nodeName:}" failed. No retries permitted until 2026-09-17 11:39:07.99550128 +0000 UTC m=+18.002948290 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs") pod "network-metrics-daemon-f5svv" (UID: "757a11fc-3750-4ef9-94f6-6c6da0d5c95c") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 11:39:00.096267 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:00.096237 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret\") pod \"global-pull-secret-syncer-6jmh4\" (UID: \"7a49d705-8c21-411b-8988-260b0158a9be\") " pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:39:00.096459 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:00.096277 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxqzg\" (UniqueName: \"kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg\") pod \"network-check-target-pk9p9\" (UID: \"78be981b-e482-4dcf-a146-ea8549a3a847\") " pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:39:00.096459 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:00.096434 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 11:39:00.096459 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:00.096453 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 11:39:00.096590 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:00.096464 2581 projected.go:196] Error preparing data for projected volume kube-api-access-cxqzg for pod openshift-network-diagnostics/network-check-target-pk9p9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 11:39:00.096590 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:00.096507 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg podName:78be981b-e482-4dcf-a146-ea8549a3a847 nodeName:}" failed. No retries permitted until 2026-09-17 11:39:08.096494503 +0000 UTC m=+18.103941512 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxqzg" (UniqueName: "kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg") pod "network-check-target-pk9p9" (UID: "78be981b-e482-4dcf-a146-ea8549a3a847") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 11:39:00.096789 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:00.096763 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 11:39:00.096961 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:00.096832 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret podName:7a49d705-8c21-411b-8988-260b0158a9be nodeName:}" failed. No retries permitted until 2026-09-17 11:39:08.096819414 +0000 UTC m=+18.104266423 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret") pod "global-pull-secret-syncer-6jmh4" (UID: "7a49d705-8c21-411b-8988-260b0158a9be") : object "kube-system"/"original-pull-secret" not registered Sep 17 11:39:00.423506 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:00.423421 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:39:00.423666 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:00.423533 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:39:00.423666 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:00.423547 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5svv" podUID="757a11fc-3750-4ef9-94f6-6c6da0d5c95c" Sep 17 11:39:00.423666 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:00.423650 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jmh4" podUID="7a49d705-8c21-411b-8988-260b0158a9be" Sep 17 11:39:00.423832 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:00.423698 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:39:00.423832 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:00.423763 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pk9p9" podUID="78be981b-e482-4dcf-a146-ea8549a3a847" Sep 17 11:39:02.425661 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:02.425627 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:39:02.426097 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:02.425745 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5svv" podUID="757a11fc-3750-4ef9-94f6-6c6da0d5c95c" Sep 17 11:39:02.426097 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:02.425812 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:39:02.426097 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:02.425864 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jmh4" podUID="7a49d705-8c21-411b-8988-260b0158a9be" Sep 17 11:39:02.426097 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:02.425911 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:39:02.426097 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:02.425976 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pk9p9" podUID="78be981b-e482-4dcf-a146-ea8549a3a847" Sep 17 11:39:04.423144 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:04.423077 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:39:04.423624 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:04.423082 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:39:04.423624 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:04.423222 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jmh4" podUID="7a49d705-8c21-411b-8988-260b0158a9be" Sep 17 11:39:04.423624 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:04.423254 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:39:04.423624 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:04.423350 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5svv" podUID="757a11fc-3750-4ef9-94f6-6c6da0d5c95c" Sep 17 11:39:04.423624 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:04.423418 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pk9p9" podUID="78be981b-e482-4dcf-a146-ea8549a3a847" Sep 17 11:39:06.423566 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:06.423534 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:39:06.423919 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:06.423534 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:39:06.423919 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:06.423650 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5svv" podUID="757a11fc-3750-4ef9-94f6-6c6da0d5c95c" Sep 17 11:39:06.423919 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:06.423761 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jmh4" podUID="7a49d705-8c21-411b-8988-260b0158a9be" Sep 17 11:39:06.423919 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:06.423816 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:39:06.423919 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:06.423891 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pk9p9" podUID="78be981b-e482-4dcf-a146-ea8549a3a847" Sep 17 11:39:08.059154 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:08.059108 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs\") pod \"network-metrics-daemon-f5svv\" (UID: \"757a11fc-3750-4ef9-94f6-6c6da0d5c95c\") " pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:39:08.059636 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:08.059288 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 11:39:08.059636 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:08.059355 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs podName:757a11fc-3750-4ef9-94f6-6c6da0d5c95c nodeName:}" failed. No retries permitted until 2026-09-17 11:39:24.059337951 +0000 UTC m=+34.066784965 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs") pod "network-metrics-daemon-f5svv" (UID: "757a11fc-3750-4ef9-94f6-6c6da0d5c95c") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 11:39:08.159660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:08.159628 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret\") pod \"global-pull-secret-syncer-6jmh4\" (UID: \"7a49d705-8c21-411b-8988-260b0158a9be\") " pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:39:08.159840 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:08.159669 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxqzg\" (UniqueName: \"kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg\") pod \"network-check-target-pk9p9\" (UID: \"78be981b-e482-4dcf-a146-ea8549a3a847\") " pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:39:08.159840 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:08.159784 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 11:39:08.159840 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:08.159807 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 11:39:08.159840 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:08.159821 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 11:39:08.159840 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:08.159835 2581 projected.go:196] Error preparing data for projected volume kube-api-access-cxqzg for pod openshift-network-diagnostics/network-check-target-pk9p9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 11:39:08.160016 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:08.159852 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret podName:7a49d705-8c21-411b-8988-260b0158a9be nodeName:}" failed. No retries permitted until 2026-09-17 11:39:24.15983485 +0000 UTC m=+34.167281876 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret") pod "global-pull-secret-syncer-6jmh4" (UID: "7a49d705-8c21-411b-8988-260b0158a9be") : object "kube-system"/"original-pull-secret" not registered Sep 17 11:39:08.160016 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:08.159871 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg podName:78be981b-e482-4dcf-a146-ea8549a3a847 nodeName:}" failed. No retries permitted until 2026-09-17 11:39:24.159859939 +0000 UTC m=+34.167306954 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxqzg" (UniqueName: "kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg") pod "network-check-target-pk9p9" (UID: "78be981b-e482-4dcf-a146-ea8549a3a847") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 11:39:08.423260 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:08.423172 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:39:08.423260 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:08.423198 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:39:08.423260 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:08.423170 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:39:08.423530 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:08.423303 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pk9p9" podUID="78be981b-e482-4dcf-a146-ea8549a3a847" Sep 17 11:39:08.423530 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:08.423366 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jmh4" podUID="7a49d705-8c21-411b-8988-260b0158a9be" Sep 17 11:39:08.423530 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:08.423410 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5svv" podUID="757a11fc-3750-4ef9-94f6-6c6da0d5c95c" Sep 17 11:39:10.424041 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:10.424015 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:39:10.424512 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:10.424211 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:39:10.424512 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:10.424213 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5svv" podUID="757a11fc-3750-4ef9-94f6-6c6da0d5c95c" Sep 17 11:39:10.424512 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:10.424349 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jmh4" podUID="7a49d705-8c21-411b-8988-260b0158a9be" Sep 17 11:39:10.424512 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:10.424420 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:39:10.424512 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:10.424486 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pk9p9" podUID="78be981b-e482-4dcf-a146-ea8549a3a847" Sep 17 11:39:11.524613 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:11.524437 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal" event={"ID":"73800decfd2c4467378c70aaf90595a7","Type":"ContainerStarted","Data":"2cf9ffd9ba15e897ad473afe5ffe4246b63c04d12a0e4139f56a2e6c544381ab"} Sep 17 11:39:11.526088 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:11.526056 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mljj2" event={"ID":"4c40a380-1eb9-46c1-a161-a9973c9b916d","Type":"ContainerStarted","Data":"4ba46382ee070958b7ee11b57b19fb338f7725597895367d43ac9374f95ffc66"} Sep 17 11:39:11.527496 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:11.527475 2581 generic.go:356] "Generic (PLEG): container finished" podID="185c4318-ad3f-4a65-bdb4-4d8cbb554069" containerID="e370285d3ffb24e5be53717b4a46f11cc3275eae02f15c844fdd00a096666ad8" exitCode=0 Sep 17 11:39:11.527575 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:11.527537 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5n4q" event={"ID":"185c4318-ad3f-4a65-bdb4-4d8cbb554069","Type":"ContainerDied","Data":"e370285d3ffb24e5be53717b4a46f11cc3275eae02f15c844fdd00a096666ad8"} Sep 17 11:39:11.529188 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:11.529152 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g6kpv" event={"ID":"67153d8b-662e-4d11-b932-2ec9f96be172","Type":"ContainerStarted","Data":"1b7513a5da7654da5df5c766972e9ad5f6a3af24de71167c031c80a22c5396f1"} Sep 17 11:39:11.530766 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:11.530742 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" event={"ID":"bc2cd604-c635-41ec-a14e-cb5574805469","Type":"ContainerStarted","Data":"d847cc17ada6fcb171f8ebcdb32c13b351238e2d26426c9a04a5d8af65688a2e"} Sep 17 11:39:11.532194 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:11.532175 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z8klg" event={"ID":"17023d9f-cb91-44aa-a873-3c5218fa56ea","Type":"ContainerStarted","Data":"00d1332ffd78c3109f455790b3e5f81e0c9bacb56c3b469530895e97d405c8de"} Sep 17 11:39:11.534903 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:11.534884 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" event={"ID":"eb0d7c07-75fb-4d19-b732-a1f9c737540a","Type":"ContainerStarted","Data":"7ee460a1a4125640b807b65320da500d6ddc00427438d07b0e388f2df369c758"} Sep 17 11:39:11.534998 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:11.534956 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" event={"ID":"eb0d7c07-75fb-4d19-b732-a1f9c737540a","Type":"ContainerStarted","Data":"cb2b11e899bc5ebfba3180b35b76b255cacd5893fc0fb6173aac2119556c2d10"} Sep 17 11:39:11.535065 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:11.535009 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" event={"ID":"eb0d7c07-75fb-4d19-b732-a1f9c737540a","Type":"ContainerStarted","Data":"f0e9fb19f671769c7462b613ffca130e551c7819af6fff01925eb3d8637dbfcf"} Sep 17 11:39:11.535065 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:11.535058 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" event={"ID":"eb0d7c07-75fb-4d19-b732-a1f9c737540a","Type":"ContainerStarted","Data":"522cb4f5e53863e3d2a3f0774e95eacd574f30b0b8a93658355a71ee0eef4092"} Sep 17 11:39:11.536308 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:11.536284 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" event={"ID":"29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0","Type":"ContainerStarted","Data":"f81771e1a7703e7f4a6214b74d46f1cd98217c1a8b2026cacbd7abe05281bb8c"} Sep 17 11:39:11.537531 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:11.537510 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wk6sh" event={"ID":"e1152cde-f077-4e81-a902-d0cd724f0495","Type":"ContainerStarted","Data":"e081c0db986fef4d0b65724446e5302a423a6486763218e4fa1f0465f65a0bdc"} Sep 17 11:39:11.542164 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:11.542109 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-196.ec2.internal" podStartSLOduration=20.542098329 podStartE2EDuration="20.542098329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 11:38:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:39:11.541901429 +0000 UTC m=+21.549348459" watchObservedRunningTime="2026-09-17 11:39:11.542098329 +0000 UTC m=+21.549545355" Sep 17 11:39:11.562112 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:11.562045 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-z8klg" podStartSLOduration=3.161115399 podStartE2EDuration="21.562031231s" totalImagesPullingTime="18.400915832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:39:11.561869657 +0000 UTC m=+21.569316706" watchObservedRunningTime="2026-09-17 11:39:11.562031231 +0000 UTC m=+21.569478260" Sep 17 11:39:11.597761 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:11.597724 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-g6kpv" podStartSLOduration=3.21544842 podStartE2EDuration="21.597696799s" totalImagesPullingTime="18.382248379s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 11:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:39:11.597685674 +0000 UTC m=+21.605132703" watchObservedRunningTime="2026-09-17 11:39:11.597696799 +0000 UTC m=+21.605143831" Sep 17 11:39:11.597861 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:11.597819 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mljj2" podStartSLOduration=3.168010533 podStartE2EDuration="21.59781261s" totalImagesPullingTime="18.429802077s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 11:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:39:11.582292426 +0000 UTC m=+21.589739458" watchObservedRunningTime="2026-09-17 11:39:11.59781261 +0000 UTC m=+21.605259639" Sep 17 11:39:11.643030 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:11.642988 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-sdxbb" podStartSLOduration=3.282483682 podStartE2EDuration="21.642972356s" totalImagesPullingTime="18.360488674s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 11:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:39:11.642853 +0000 UTC m=+21.650300030" watchObservedRunningTime="2026-09-17 11:39:11.642972356 +0000 UTC m=+21.650419385" Sep 17 11:39:11.668792 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:11.668754 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wk6sh" podStartSLOduration=3.251125236 podStartE2EDuration="21.668734784s" totalImagesPullingTime="18.417609548s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 11:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:39:11.668658122 +0000 UTC m=+21.676105152" watchObservedRunningTime="2026-09-17 11:39:11.668734784 +0000 UTC m=+21.676181815" Sep 17 11:39:12.423135 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:12.422885 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:39:12.423307 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:12.422892 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:39:12.423307 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:12.423234 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5svv" podUID="757a11fc-3750-4ef9-94f6-6c6da0d5c95c" Sep 17 11:39:12.423307 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:12.423264 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jmh4" podUID="7a49d705-8c21-411b-8988-260b0158a9be" Sep 17 11:39:12.423307 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:12.422898 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:39:12.423466 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:12.423335 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pk9p9" podUID="78be981b-e482-4dcf-a146-ea8549a3a847" Sep 17 11:39:12.541750 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:12.541709 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qbxfn" event={"ID":"26f77893-4701-4ac8-bd67-4df96b57deea","Type":"ContainerStarted","Data":"2d72702fcf7f827d1b8edc1fd837bb7877a34dc5d74f4cc60b4e149303e47441"} Sep 17 11:39:12.545394 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:12.545366 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" event={"ID":"eb0d7c07-75fb-4d19-b732-a1f9c737540a","Type":"ContainerStarted","Data":"4806efa2f3e8edc378ed5a20765b2b4808466b3f3778785326c4ccdd355fee46"} Sep 17 11:39:12.545533 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:12.545440 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" event={"ID":"eb0d7c07-75fb-4d19-b732-a1f9c737540a","Type":"ContainerStarted","Data":"668067a0f27d6067d92d293a8f57d860b1d4a2c02cddd31e9fb19b1850019059"} Sep 17 11:39:12.792740 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:12.792716 2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 17 11:39:13.346209 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:13.346177 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-z8klg" Sep 17 11:39:13.410742 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:13.410611 2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T11:39:12.792736726Z","UUID":"c6e9700c-5962-4a99-bfba-f07820cd5873","Handler":null,"Name":"","Endpoint":""} Sep 17 11:39:13.411940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:13.411913 2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 17 11:39:13.411940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:13.411942 2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 17 11:39:13.549555 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:13.549516 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" event={"ID":"bc2cd604-c635-41ec-a14e-cb5574805469","Type":"ContainerStarted","Data":"490c18f8698a573676e8e4d920f5cb68bb6eb786e56a343d4d09c4df25f3abdc"} Sep 17 11:39:14.423632 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:14.423555 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:39:14.423800 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:14.423556 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:39:14.423800 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:14.423678 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pk9p9" podUID="78be981b-e482-4dcf-a146-ea8549a3a847" Sep 17 11:39:14.423800 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:14.423714 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:39:14.423965 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:14.423818 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jmh4" podUID="7a49d705-8c21-411b-8988-260b0158a9be" Sep 17 11:39:14.423965 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:14.423944 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5svv" podUID="757a11fc-3750-4ef9-94f6-6c6da0d5c95c" Sep 17 11:39:14.554877 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:14.554838 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" event={"ID":"bc2cd604-c635-41ec-a14e-cb5574805469","Type":"ContainerStarted","Data":"5512095e89439c2d83a3d3de81be6bdaf0e7540cd0b155628cd3db609517bd07"} Sep 17 11:39:14.559026 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:14.558991 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" event={"ID":"eb0d7c07-75fb-4d19-b732-a1f9c737540a","Type":"ContainerStarted","Data":"0886b3a73d5fb88500966e77dac2a71a0d1a2b8f257ac434ff02f3c8f396706f"} Sep 17 11:39:14.578111 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:14.578050 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qbxfn" podStartSLOduration=6.169056883 podStartE2EDuration="24.578035228s" totalImagesPullingTime="18.408978345s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 11:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:39:12.561505796 +0000 UTC m=+22.568952846" watchObservedRunningTime="2026-09-17 11:39:14.578035228 +0000 UTC m=+24.585482258" Sep 17 11:39:14.611013 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:14.610978 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-z8klg" Sep 17 11:39:14.611717 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:14.611695 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-z8klg" Sep 17 11:39:14.630871 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:14.630825 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzqqg" podStartSLOduration=3.719850546 podStartE2EDuration="24.630807134s" totalImagesPullingTime="20.910956588s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 11:38:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:39:14.577865312 +0000 UTC m=+24.585312344" watchObservedRunningTime="2026-09-17 11:39:14.630807134 +0000 UTC m=+24.638254165" Sep 17 11:39:15.561876 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:15.561849 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-z8klg" Sep 17 11:39:16.423549 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:16.423329 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:39:16.423701 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:16.423658 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jmh4" podUID="7a49d705-8c21-411b-8988-260b0158a9be" Sep 17 11:39:16.423701 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:16.423370 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:39:16.423810 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:16.423750 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pk9p9" podUID="78be981b-e482-4dcf-a146-ea8549a3a847" Sep 17 11:39:16.423810 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:16.423328 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:39:16.423885 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:16.423837 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5svv" podUID="757a11fc-3750-4ef9-94f6-6c6da0d5c95c" Sep 17 11:39:16.567009 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:16.566960 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" event={"ID":"eb0d7c07-75fb-4d19-b732-a1f9c737540a","Type":"ContainerStarted","Data":"f27af8e83e35ec63179861050628bcbafa2c2001c6f1a78bab060d78b9904003"} Sep 17 11:39:16.567506 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:16.567226 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:39:16.567506 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:16.567314 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:39:16.568877 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:16.568855 2581 generic.go:356] "Generic (PLEG): container finished" podID="185c4318-ad3f-4a65-bdb4-4d8cbb554069" containerID="c9ef6c3ad7217b92c5f15727ec4cb7b902f90c8e0b6070ed48ccccc9d2bd4241" exitCode=0 Sep 17 11:39:16.569002 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:16.568934 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5n4q" event={"ID":"185c4318-ad3f-4a65-bdb4-4d8cbb554069","Type":"ContainerDied","Data":"c9ef6c3ad7217b92c5f15727ec4cb7b902f90c8e0b6070ed48ccccc9d2bd4241"} Sep 17 11:39:16.583341 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:16.583322 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:39:16.598702 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:16.598621 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" podStartSLOduration=8.187577332 podStartE2EDuration="26.598606604s" totalImagesPullingTime="18.411029272s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 11:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:39:16.598443491 +0000 UTC m=+26.605890520" watchObservedRunningTime="2026-09-17 11:39:16.598606604 +0000 UTC m=+26.606053633" Sep 17 11:39:17.573316 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:17.573283 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:39:17.590143 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:17.590098 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:39:18.264191 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:18.263974 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6jmh4"] Sep 17 11:39:18.264371 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:18.264300 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:39:18.264444 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:18.264420 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jmh4" podUID="7a49d705-8c21-411b-8988-260b0158a9be" Sep 17 11:39:18.267046 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:18.267021 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pk9p9"] Sep 17 11:39:18.267188 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:18.267143 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:39:18.267244 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:18.267228 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pk9p9" podUID="78be981b-e482-4dcf-a146-ea8549a3a847" Sep 17 11:39:18.267657 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:18.267637 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f5svv"] Sep 17 11:39:18.267751 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:18.267739 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:39:18.267866 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:18.267849 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5svv" podUID="757a11fc-3750-4ef9-94f6-6c6da0d5c95c" Sep 17 11:39:18.576960 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:18.576927 2581 generic.go:356] "Generic (PLEG): container finished" podID="185c4318-ad3f-4a65-bdb4-4d8cbb554069" containerID="05c7ff5ccfcd1fcd17fbd77e30aba65865efe83efda7312a95724d764a951f54" exitCode=0 Sep 17 11:39:18.577520 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:18.577019 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5n4q" event={"ID":"185c4318-ad3f-4a65-bdb4-4d8cbb554069","Type":"ContainerDied","Data":"05c7ff5ccfcd1fcd17fbd77e30aba65865efe83efda7312a95724d764a951f54"} Sep 17 11:39:19.423078 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:19.423046 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:39:19.423263 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:19.423057 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:39:19.423263 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:19.423184 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jmh4" podUID="7a49d705-8c21-411b-8988-260b0158a9be" Sep 17 11:39:19.423339 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:19.423256 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5svv" podUID="757a11fc-3750-4ef9-94f6-6c6da0d5c95c" Sep 17 11:39:19.581448 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:19.581350 2581 generic.go:356] "Generic (PLEG): container finished" podID="185c4318-ad3f-4a65-bdb4-4d8cbb554069" containerID="51cdcf0fac0e00554197771064ce69f3c62947c26dfbdb2f2c47a62b64a0ac23" exitCode=0 Sep 17 11:39:19.581448 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:19.581426 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5n4q" event={"ID":"185c4318-ad3f-4a65-bdb4-4d8cbb554069","Type":"ContainerDied","Data":"51cdcf0fac0e00554197771064ce69f3c62947c26dfbdb2f2c47a62b64a0ac23"} Sep 17 11:39:20.423411 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:20.423382 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:39:20.423600 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:20.423473 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pk9p9" podUID="78be981b-e482-4dcf-a146-ea8549a3a847" Sep 17 11:39:21.423630 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:21.423551 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:39:21.424081 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:21.423680 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5svv" podUID="757a11fc-3750-4ef9-94f6-6c6da0d5c95c" Sep 17 11:39:21.424081 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:21.423728 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:39:21.424081 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:21.423877 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jmh4" podUID="7a49d705-8c21-411b-8988-260b0158a9be" Sep 17 11:39:22.422875 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:22.422840 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:39:22.423058 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:22.422970 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pk9p9" podUID="78be981b-e482-4dcf-a146-ea8549a3a847" Sep 17 11:39:23.422906 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:23.422870 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:39:23.422906 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:23.422917 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:39:23.423462 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:23.423018 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5svv" podUID="757a11fc-3750-4ef9-94f6-6c6da0d5c95c" Sep 17 11:39:23.423462 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:23.423110 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jmh4" podUID="7a49d705-8c21-411b-8988-260b0158a9be" Sep 17 11:39:24.077479 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.077427 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs\") pod \"network-metrics-daemon-f5svv\" (UID: \"757a11fc-3750-4ef9-94f6-6c6da0d5c95c\") " pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:39:24.077672 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:24.077600 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 11:39:24.077734 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:24.077684 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs podName:757a11fc-3750-4ef9-94f6-6c6da0d5c95c nodeName:}" failed. No retries permitted until 2026-09-17 11:39:56.077661478 +0000 UTC m=+66.085108492 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs") pod "network-metrics-daemon-f5svv" (UID: "757a11fc-3750-4ef9-94f6-6c6da0d5c95c") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 11:39:24.178644 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.178604 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret\") pod \"global-pull-secret-syncer-6jmh4\" (UID: \"7a49d705-8c21-411b-8988-260b0158a9be\") " pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:39:24.178644 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.178644 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxqzg\" (UniqueName: \"kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg\") pod \"network-check-target-pk9p9\" (UID: \"78be981b-e482-4dcf-a146-ea8549a3a847\") " pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:39:24.178876 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:24.178774 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 11:39:24.178876 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:24.178785 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 11:39:24.178876 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:24.178803 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 11:39:24.178876 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:24.178812 2581 projected.go:196] Error preparing data for projected volume kube-api-access-cxqzg for pod openshift-network-diagnostics/network-check-target-pk9p9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 11:39:24.178876 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:24.178853 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret podName:7a49d705-8c21-411b-8988-260b0158a9be nodeName:}" failed. No retries permitted until 2026-09-17 11:39:56.178833077 +0000 UTC m=+66.186280091 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret") pod "global-pull-secret-syncer-6jmh4" (UID: "7a49d705-8c21-411b-8988-260b0158a9be") : object "kube-system"/"original-pull-secret" not registered Sep 17 11:39:24.178876 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:24.178873 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg podName:78be981b-e482-4dcf-a146-ea8549a3a847 nodeName:}" failed. No retries permitted until 2026-09-17 11:39:56.178864354 +0000 UTC m=+66.186311369 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxqzg" (UniqueName: "kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg") pod "network-check-target-pk9p9" (UID: "78be981b-e482-4dcf-a146-ea8549a3a847") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 11:39:24.422779 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.422707 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:39:24.422942 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:24.422807 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pk9p9" podUID="78be981b-e482-4dcf-a146-ea8549a3a847" Sep 17 11:39:24.817409 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.817249 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-196.ec2.internal" event="NodeReady" Sep 17 11:39:24.817541 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.817531 2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 17 11:39:24.877261 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.877232 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8b6dg"] Sep 17 11:39:24.895082 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.895052 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nqcnx"] Sep 17 11:39:24.895625 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.895596 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8b6dg" Sep 17 11:39:24.898831 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.898810 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 17 11:39:24.898939 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.898808 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 17 11:39:24.898939 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.898814 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hs2cv\"" Sep 17 11:39:24.909010 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.908992 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nqcnx"] Sep 17 11:39:24.909092 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.909019 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8b6dg"] Sep 17 11:39:24.909154 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.909098 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nqcnx" Sep 17 11:39:24.915340 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.915302 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 17 11:39:24.915340 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.915332 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 17 11:39:24.915491 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.915338 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 17 11:39:24.915491 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.915395 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-k5pqb\"" Sep 17 11:39:24.984999 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.984928 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0d623f4-748b-4d82-9a70-41671b79d346-tmp-dir\") pod \"dns-default-8b6dg\" (UID: \"a0d623f4-748b-4d82-9a70-41671b79d346\") " pod="openshift-dns/dns-default-8b6dg" Sep 17 11:39:24.985105 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.985016 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5khfn\" (UniqueName: \"kubernetes.io/projected/a0d623f4-748b-4d82-9a70-41671b79d346-kube-api-access-5khfn\") pod \"dns-default-8b6dg\" (UID: \"a0d623f4-748b-4d82-9a70-41671b79d346\") " pod="openshift-dns/dns-default-8b6dg" Sep 17 11:39:24.985105 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.985040 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfxd5\" (UniqueName: \"kubernetes.io/projected/d4657dd9-0b00-4a86-af8e-10392c1704f9-kube-api-access-xfxd5\") pod \"ingress-canary-nqcnx\" (UID: \"d4657dd9-0b00-4a86-af8e-10392c1704f9\") " pod="openshift-ingress-canary/ingress-canary-nqcnx" Sep 17 11:39:24.985105 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.985080 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a0d623f4-748b-4d82-9a70-41671b79d346-config-volume\") pod \"dns-default-8b6dg\" (UID: \"a0d623f4-748b-4d82-9a70-41671b79d346\") " pod="openshift-dns/dns-default-8b6dg" Sep 17 11:39:24.985105 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.985097 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls\") pod \"dns-default-8b6dg\" (UID: \"a0d623f4-748b-4d82-9a70-41671b79d346\") " pod="openshift-dns/dns-default-8b6dg" Sep 17 11:39:24.985278 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:24.985111 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4657dd9-0b00-4a86-af8e-10392c1704f9-cert\") pod \"ingress-canary-nqcnx\" (UID: \"d4657dd9-0b00-4a86-af8e-10392c1704f9\") " pod="openshift-ingress-canary/ingress-canary-nqcnx" Sep 17 11:39:25.085766 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.085724 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5khfn\" (UniqueName: \"kubernetes.io/projected/a0d623f4-748b-4d82-9a70-41671b79d346-kube-api-access-5khfn\") pod \"dns-default-8b6dg\" (UID: \"a0d623f4-748b-4d82-9a70-41671b79d346\") " pod="openshift-dns/dns-default-8b6dg" Sep 17 11:39:25.085766 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.085754 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfxd5\" (UniqueName: \"kubernetes.io/projected/d4657dd9-0b00-4a86-af8e-10392c1704f9-kube-api-access-xfxd5\") pod \"ingress-canary-nqcnx\" (UID: \"d4657dd9-0b00-4a86-af8e-10392c1704f9\") " pod="openshift-ingress-canary/ingress-canary-nqcnx" Sep 17 11:39:25.085948 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.085779 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a0d623f4-748b-4d82-9a70-41671b79d346-config-volume\") pod \"dns-default-8b6dg\" (UID: \"a0d623f4-748b-4d82-9a70-41671b79d346\") " pod="openshift-dns/dns-default-8b6dg" Sep 17 11:39:25.085948 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.085805 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls\") pod \"dns-default-8b6dg\" (UID: \"a0d623f4-748b-4d82-9a70-41671b79d346\") " pod="openshift-dns/dns-default-8b6dg" Sep 17 11:39:25.085948 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.085828 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4657dd9-0b00-4a86-af8e-10392c1704f9-cert\") pod \"ingress-canary-nqcnx\" (UID: \"d4657dd9-0b00-4a86-af8e-10392c1704f9\") " pod="openshift-ingress-canary/ingress-canary-nqcnx" Sep 17 11:39:25.085948 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.085882 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0d623f4-748b-4d82-9a70-41671b79d346-tmp-dir\") pod \"dns-default-8b6dg\" (UID: \"a0d623f4-748b-4d82-9a70-41671b79d346\") " pod="openshift-dns/dns-default-8b6dg" Sep 17 11:39:25.085948 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:25.085930 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 11:39:25.086170 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:25.085997 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls podName:a0d623f4-748b-4d82-9a70-41671b79d346 nodeName:}" failed. No retries permitted until 2026-09-17 11:39:25.585976254 +0000 UTC m=+35.593423277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls") pod "dns-default-8b6dg" (UID: "a0d623f4-748b-4d82-9a70-41671b79d346") : secret "dns-default-metrics-tls" not found Sep 17 11:39:25.086449 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.086433 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0d623f4-748b-4d82-9a70-41671b79d346-tmp-dir\") pod \"dns-default-8b6dg\" (UID: \"a0d623f4-748b-4d82-9a70-41671b79d346\") " pod="openshift-dns/dns-default-8b6dg" Sep 17 11:39:25.086660 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.086640 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a0d623f4-748b-4d82-9a70-41671b79d346-config-volume\") pod \"dns-default-8b6dg\" (UID: \"a0d623f4-748b-4d82-9a70-41671b79d346\") " pod="openshift-dns/dns-default-8b6dg" Sep 17 11:39:25.090066 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.090045 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4657dd9-0b00-4a86-af8e-10392c1704f9-cert\") pod \"ingress-canary-nqcnx\" (UID: \"d4657dd9-0b00-4a86-af8e-10392c1704f9\") " pod="openshift-ingress-canary/ingress-canary-nqcnx" Sep 17 11:39:25.095109 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.095089 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5khfn\" (UniqueName: \"kubernetes.io/projected/a0d623f4-748b-4d82-9a70-41671b79d346-kube-api-access-5khfn\") pod \"dns-default-8b6dg\" (UID: \"a0d623f4-748b-4d82-9a70-41671b79d346\") " pod="openshift-dns/dns-default-8b6dg" Sep 17 11:39:25.095357 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.095336 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfxd5\" (UniqueName: \"kubernetes.io/projected/d4657dd9-0b00-4a86-af8e-10392c1704f9-kube-api-access-xfxd5\") pod \"ingress-canary-nqcnx\" (UID: \"d4657dd9-0b00-4a86-af8e-10392c1704f9\") " pod="openshift-ingress-canary/ingress-canary-nqcnx" Sep 17 11:39:25.217042 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.217004 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nqcnx" Sep 17 11:39:25.422523 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.422496 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:39:25.422683 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.422494 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:39:25.425832 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.425803 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 17 11:39:25.426459 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.425943 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 11:39:25.426459 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.425974 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l8fq5\"" Sep 17 11:39:25.551585 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.551518 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nqcnx"] Sep 17 11:39:25.555260 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:39:25.555228 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4657dd9_0b00_4a86_af8e_10392c1704f9.slice/crio-3a0506067c85e0139308ff496ce7afd7bc65f7dfa038ac587addd7d6a612d6a1 WatchSource:0}: Error finding container 3a0506067c85e0139308ff496ce7afd7bc65f7dfa038ac587addd7d6a612d6a1: Status 404 returned error can't find the container with id 3a0506067c85e0139308ff496ce7afd7bc65f7dfa038ac587addd7d6a612d6a1 Sep 17 11:39:25.590016 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.589992 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls\") pod \"dns-default-8b6dg\" (UID: \"a0d623f4-748b-4d82-9a70-41671b79d346\") " pod="openshift-dns/dns-default-8b6dg" Sep 17 11:39:25.590147 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:25.590104 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 11:39:25.590204 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:25.590182 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls podName:a0d623f4-748b-4d82-9a70-41671b79d346 nodeName:}" failed. No retries permitted until 2026-09-17 11:39:26.59016319 +0000 UTC m=+36.597610208 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls") pod "dns-default-8b6dg" (UID: "a0d623f4-748b-4d82-9a70-41671b79d346") : secret "dns-default-metrics-tls" not found Sep 17 11:39:25.598199 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.598171 2581 generic.go:356] "Generic (PLEG): container finished" podID="185c4318-ad3f-4a65-bdb4-4d8cbb554069" containerID="d138abe6eb46313c1af91de23011af6d6d2d1933c71672564a572e39227d1e52" exitCode=0 Sep 17 11:39:25.598282 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.598258 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5n4q" event={"ID":"185c4318-ad3f-4a65-bdb4-4d8cbb554069","Type":"ContainerDied","Data":"d138abe6eb46313c1af91de23011af6d6d2d1933c71672564a572e39227d1e52"} Sep 17 11:39:25.599483 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:25.599455 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nqcnx" event={"ID":"d4657dd9-0b00-4a86-af8e-10392c1704f9","Type":"ContainerStarted","Data":"3a0506067c85e0139308ff496ce7afd7bc65f7dfa038ac587addd7d6a612d6a1"} Sep 17 11:39:26.423600 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:26.423569 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:39:26.426528 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:26.426499 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 17 11:39:26.427162 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:26.427137 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 17 11:39:26.427293 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:26.427160 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nrbmw\"" Sep 17 11:39:26.597159 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:26.597110 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls\") pod \"dns-default-8b6dg\" (UID: \"a0d623f4-748b-4d82-9a70-41671b79d346\") " pod="openshift-dns/dns-default-8b6dg" Sep 17 11:39:26.597342 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:26.597258 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 11:39:26.597342 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:26.597331 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls podName:a0d623f4-748b-4d82-9a70-41671b79d346 nodeName:}" failed. No retries permitted until 2026-09-17 11:39:28.59730927 +0000 UTC m=+38.604756279 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls") pod "dns-default-8b6dg" (UID: "a0d623f4-748b-4d82-9a70-41671b79d346") : secret "dns-default-metrics-tls" not found Sep 17 11:39:26.605677 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:26.605650 2581 generic.go:356] "Generic (PLEG): container finished" podID="185c4318-ad3f-4a65-bdb4-4d8cbb554069" containerID="644b738b2ddb75a6a0a9b668b8669b955e99c265e8000956443ffcc85cfbe7db" exitCode=0 Sep 17 11:39:26.605814 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:26.605691 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5n4q" event={"ID":"185c4318-ad3f-4a65-bdb4-4d8cbb554069","Type":"ContainerDied","Data":"644b738b2ddb75a6a0a9b668b8669b955e99c265e8000956443ffcc85cfbe7db"} Sep 17 11:39:27.612700 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:27.612504 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z5n4q" event={"ID":"185c4318-ad3f-4a65-bdb4-4d8cbb554069","Type":"ContainerStarted","Data":"1d771be44a5653ea6604a0d3047fcc312f2bcaa45e7b04a9f627ced785b1d9da"} Sep 17 11:39:27.635883 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:27.635825 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-z5n4q" podStartSLOduration=7.858515047 podStartE2EDuration="37.635806595s" totalImagesPullingTime="29.777291548s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 11:38:50 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:39:27.634413906 +0000 UTC m=+37.641860938" watchObservedRunningTime="2026-09-17 11:39:27.635806595 +0000 UTC m=+37.643253625" Sep 17 11:39:28.611002 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:28.610966 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls\") pod \"dns-default-8b6dg\" (UID: \"a0d623f4-748b-4d82-9a70-41671b79d346\") " pod="openshift-dns/dns-default-8b6dg" Sep 17 11:39:28.611181 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:28.611143 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 11:39:28.611228 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:28.611210 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls podName:a0d623f4-748b-4d82-9a70-41671b79d346 nodeName:}" failed. No retries permitted until 2026-09-17 11:39:32.611193482 +0000 UTC m=+42.618640489 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls") pod "dns-default-8b6dg" (UID: "a0d623f4-748b-4d82-9a70-41671b79d346") : secret "dns-default-metrics-tls" not found Sep 17 11:39:28.616063 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:28.616037 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nqcnx" event={"ID":"d4657dd9-0b00-4a86-af8e-10392c1704f9","Type":"ContainerStarted","Data":"b382c7b64fc35e9303d654f8ad823d86438da0d74b22cfe5cbef06e75b013c19"} Sep 17 11:39:28.633198 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:28.633161 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nqcnx" podStartSLOduration=2.201240266 podStartE2EDuration="4.633147682s" totalImagesPullingTime="2.431907416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:39:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:39:28.632831576 +0000 UTC m=+38.640278603" watchObservedRunningTime="2026-09-17 11:39:28.633147682 +0000 UTC m=+38.640594706" Sep 17 11:39:32.638185 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:32.638151 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls\") pod \"dns-default-8b6dg\" (UID: \"a0d623f4-748b-4d82-9a70-41671b79d346\") " pod="openshift-dns/dns-default-8b6dg" Sep 17 11:39:32.638591 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:32.638289 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 11:39:32.638591 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:32.638347 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls podName:a0d623f4-748b-4d82-9a70-41671b79d346 nodeName:}" failed. No retries permitted until 2026-09-17 11:39:40.638332129 +0000 UTC m=+50.645779140 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls") pod "dns-default-8b6dg" (UID: "a0d623f4-748b-4d82-9a70-41671b79d346") : secret "dns-default-metrics-tls" not found Sep 17 11:39:40.548058 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.548029 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-skhl2"] Sep 17 11:39:40.558684 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.558665 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-skhl2" Sep 17 11:39:40.568077 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.568056 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-skhl2"] Sep 17 11:39:40.571045 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.571027 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 17 11:39:40.571346 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.571329 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 17 11:39:40.571346 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.571336 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 17 11:39:40.575940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.575924 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-vqwvf\"" Sep 17 11:39:40.576024 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.575931 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 17 11:39:40.597620 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.597601 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 17 11:39:40.693768 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.693731 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87h45\" (UniqueName: \"kubernetes.io/projected/344d4ef6-f336-46de-a0bd-0735aba220ec-kube-api-access-87h45\") pod \"insights-operator-5844b856db-skhl2\" (UID: \"344d4ef6-f336-46de-a0bd-0735aba220ec\") " pod="openshift-insights/insights-operator-5844b856db-skhl2" Sep 17 11:39:40.693878 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.693772 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/344d4ef6-f336-46de-a0bd-0735aba220ec-trusted-ca-bundle\") pod \"insights-operator-5844b856db-skhl2\" (UID: \"344d4ef6-f336-46de-a0bd-0735aba220ec\") " pod="openshift-insights/insights-operator-5844b856db-skhl2" Sep 17 11:39:40.693878 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.693797 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/344d4ef6-f336-46de-a0bd-0735aba220ec-serving-cert\") pod \"insights-operator-5844b856db-skhl2\" (UID: \"344d4ef6-f336-46de-a0bd-0735aba220ec\") " pod="openshift-insights/insights-operator-5844b856db-skhl2" Sep 17 11:39:40.693878 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.693819 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls\") pod \"dns-default-8b6dg\" (UID: \"a0d623f4-748b-4d82-9a70-41671b79d346\") " pod="openshift-dns/dns-default-8b6dg" Sep 17 11:39:40.693878 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.693844 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/344d4ef6-f336-46de-a0bd-0735aba220ec-tmp\") pod \"insights-operator-5844b856db-skhl2\" (UID: \"344d4ef6-f336-46de-a0bd-0735aba220ec\") " pod="openshift-insights/insights-operator-5844b856db-skhl2" Sep 17 11:39:40.693878 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.693867 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/344d4ef6-f336-46de-a0bd-0735aba220ec-service-ca-bundle\") pod \"insights-operator-5844b856db-skhl2\" (UID: \"344d4ef6-f336-46de-a0bd-0735aba220ec\") " pod="openshift-insights/insights-operator-5844b856db-skhl2" Sep 17 11:39:40.694057 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.693887 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/344d4ef6-f336-46de-a0bd-0735aba220ec-snapshots\") pod \"insights-operator-5844b856db-skhl2\" (UID: \"344d4ef6-f336-46de-a0bd-0735aba220ec\") " pod="openshift-insights/insights-operator-5844b856db-skhl2" Sep 17 11:39:40.694057 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:40.693982 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 11:39:40.694057 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:40.694043 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls podName:a0d623f4-748b-4d82-9a70-41671b79d346 nodeName:}" failed. No retries permitted until 2026-09-17 11:39:56.69402621 +0000 UTC m=+66.701473218 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls") pod "dns-default-8b6dg" (UID: "a0d623f4-748b-4d82-9a70-41671b79d346") : secret "dns-default-metrics-tls" not found Sep 17 11:39:40.794459 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.794428 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87h45\" (UniqueName: \"kubernetes.io/projected/344d4ef6-f336-46de-a0bd-0735aba220ec-kube-api-access-87h45\") pod \"insights-operator-5844b856db-skhl2\" (UID: \"344d4ef6-f336-46de-a0bd-0735aba220ec\") " pod="openshift-insights/insights-operator-5844b856db-skhl2" Sep 17 11:39:40.794593 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.794472 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/344d4ef6-f336-46de-a0bd-0735aba220ec-trusted-ca-bundle\") pod \"insights-operator-5844b856db-skhl2\" (UID: \"344d4ef6-f336-46de-a0bd-0735aba220ec\") " pod="openshift-insights/insights-operator-5844b856db-skhl2" Sep 17 11:39:40.794593 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.794515 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/344d4ef6-f336-46de-a0bd-0735aba220ec-serving-cert\") pod \"insights-operator-5844b856db-skhl2\" (UID: \"344d4ef6-f336-46de-a0bd-0735aba220ec\") " pod="openshift-insights/insights-operator-5844b856db-skhl2" Sep 17 11:39:40.794692 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.794672 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/344d4ef6-f336-46de-a0bd-0735aba220ec-tmp\") pod \"insights-operator-5844b856db-skhl2\" (UID: \"344d4ef6-f336-46de-a0bd-0735aba220ec\") " pod="openshift-insights/insights-operator-5844b856db-skhl2" Sep 17 11:39:40.794735 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.794722 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/344d4ef6-f336-46de-a0bd-0735aba220ec-service-ca-bundle\") pod \"insights-operator-5844b856db-skhl2\" (UID: \"344d4ef6-f336-46de-a0bd-0735aba220ec\") " pod="openshift-insights/insights-operator-5844b856db-skhl2" Sep 17 11:39:40.794785 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.794752 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/344d4ef6-f336-46de-a0bd-0735aba220ec-snapshots\") pod \"insights-operator-5844b856db-skhl2\" (UID: \"344d4ef6-f336-46de-a0bd-0735aba220ec\") " pod="openshift-insights/insights-operator-5844b856db-skhl2" Sep 17 11:39:40.795079 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.795056 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/344d4ef6-f336-46de-a0bd-0735aba220ec-tmp\") pod \"insights-operator-5844b856db-skhl2\" (UID: \"344d4ef6-f336-46de-a0bd-0735aba220ec\") " pod="openshift-insights/insights-operator-5844b856db-skhl2" Sep 17 11:39:40.795365 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.795344 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/344d4ef6-f336-46de-a0bd-0735aba220ec-snapshots\") pod \"insights-operator-5844b856db-skhl2\" (UID: \"344d4ef6-f336-46de-a0bd-0735aba220ec\") " pod="openshift-insights/insights-operator-5844b856db-skhl2" Sep 17 11:39:40.795435 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.795382 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/344d4ef6-f336-46de-a0bd-0735aba220ec-service-ca-bundle\") pod \"insights-operator-5844b856db-skhl2\" (UID: \"344d4ef6-f336-46de-a0bd-0735aba220ec\") " pod="openshift-insights/insights-operator-5844b856db-skhl2" Sep 17 11:39:40.795517 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.795500 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/344d4ef6-f336-46de-a0bd-0735aba220ec-trusted-ca-bundle\") pod \"insights-operator-5844b856db-skhl2\" (UID: \"344d4ef6-f336-46de-a0bd-0735aba220ec\") " pod="openshift-insights/insights-operator-5844b856db-skhl2" Sep 17 11:39:40.798026 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.798006 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/344d4ef6-f336-46de-a0bd-0735aba220ec-serving-cert\") pod \"insights-operator-5844b856db-skhl2\" (UID: \"344d4ef6-f336-46de-a0bd-0735aba220ec\") " pod="openshift-insights/insights-operator-5844b856db-skhl2" Sep 17 11:39:40.811979 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.811927 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87h45\" (UniqueName: \"kubernetes.io/projected/344d4ef6-f336-46de-a0bd-0735aba220ec-kube-api-access-87h45\") pod \"insights-operator-5844b856db-skhl2\" (UID: \"344d4ef6-f336-46de-a0bd-0735aba220ec\") " pod="openshift-insights/insights-operator-5844b856db-skhl2" Sep 17 11:39:40.867063 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:40.867038 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-skhl2" Sep 17 11:39:41.157269 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:41.157193 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-skhl2"] Sep 17 11:39:41.160380 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:39:41.160353 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod344d4ef6_f336_46de_a0bd_0735aba220ec.slice/crio-ed43800a77dbffeadae2c8aba65d90d8c008cd2ba43e322719c84193a2a6e0b6 WatchSource:0}: Error finding container ed43800a77dbffeadae2c8aba65d90d8c008cd2ba43e322719c84193a2a6e0b6: Status 404 returned error can't find the container with id ed43800a77dbffeadae2c8aba65d90d8c008cd2ba43e322719c84193a2a6e0b6 Sep 17 11:39:41.645700 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:41.645658 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-skhl2" event={"ID":"344d4ef6-f336-46de-a0bd-0735aba220ec","Type":"ContainerStarted","Data":"ed43800a77dbffeadae2c8aba65d90d8c008cd2ba43e322719c84193a2a6e0b6"} Sep 17 11:39:43.652102 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:43.652061 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-skhl2" event={"ID":"344d4ef6-f336-46de-a0bd-0735aba220ec","Type":"ContainerStarted","Data":"8b652ec362be1e89c3ce36c394c5939933256b697a3c5cb6fa0f39c9e09ba30d"} Sep 17 11:39:43.676605 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:43.676418 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-skhl2" podStartSLOduration=1.714388426 podStartE2EDuration="3.676401061s" totalImagesPullingTime="1.962012635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:39:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:39:43.675427554 +0000 UTC m=+53.682874584" watchObservedRunningTime="2026-09-17 11:39:43.676401061 +0000 UTC m=+53.683848093" Sep 17 11:39:44.176446 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:44.176412 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qksp8"] Sep 17 11:39:44.179495 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:44.179473 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:44.182893 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:44.182873 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 17 11:39:44.189949 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:44.189927 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 17 11:39:44.193370 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:44.193352 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qklrf\"" Sep 17 11:39:44.207362 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:44.207338 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qksp8"] Sep 17 11:39:44.322240 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:44.322211 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f8114ad-d2ca-48a8-b9af-349864c248c3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:44.322240 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:44.322243 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4lqh\" (UniqueName: \"kubernetes.io/projected/2f8114ad-d2ca-48a8-b9af-349864c248c3-kube-api-access-k4lqh\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:44.322449 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:44.322297 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f8114ad-d2ca-48a8-b9af-349864c248c3-data-volume\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:44.322449 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:44.322332 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2f8114ad-d2ca-48a8-b9af-349864c248c3-crio-socket\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:44.322449 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:44.322361 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f8114ad-d2ca-48a8-b9af-349864c248c3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:44.422872 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:44.422829 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f8114ad-d2ca-48a8-b9af-349864c248c3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:44.422872 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:44.422873 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4lqh\" (UniqueName: \"kubernetes.io/projected/2f8114ad-d2ca-48a8-b9af-349864c248c3-kube-api-access-k4lqh\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:44.423274 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:44.422967 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 11:39:44.423274 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:44.423020 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f8114ad-d2ca-48a8-b9af-349864c248c3-data-volume\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:44.423274 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:44.423038 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f8114ad-d2ca-48a8-b9af-349864c248c3-kube-rbac-proxy-cm podName:2f8114ad-d2ca-48a8-b9af-349864c248c3 nodeName:}" failed. No retries permitted until 2026-09-17 11:39:44.923021661 +0000 UTC m=+54.930468670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2f8114ad-d2ca-48a8-b9af-349864c248c3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qksp8" (UID: "2f8114ad-d2ca-48a8-b9af-349864c248c3") : configmap "kube-rbac-proxy" not found Sep 17 11:39:44.423274 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:44.423076 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2f8114ad-d2ca-48a8-b9af-349864c248c3-crio-socket\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:44.423274 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:44.423139 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f8114ad-d2ca-48a8-b9af-349864c248c3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:44.423274 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:44.423238 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 11:39:44.423274 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:44.423274 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f8114ad-d2ca-48a8-b9af-349864c248c3-insights-runtime-extractor-tls podName:2f8114ad-d2ca-48a8-b9af-349864c248c3 nodeName:}" failed. No retries permitted until 2026-09-17 11:39:44.923262705 +0000 UTC m=+54.930709714 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2f8114ad-d2ca-48a8-b9af-349864c248c3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qksp8" (UID: "2f8114ad-d2ca-48a8-b9af-349864c248c3") : secret "insights-runtime-extractor-tls" not found Sep 17 11:39:44.423585 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:44.423300 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2f8114ad-d2ca-48a8-b9af-349864c248c3-crio-socket\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:44.423585 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:44.423382 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f8114ad-d2ca-48a8-b9af-349864c248c3-data-volume\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:44.433469 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:44.433407 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4lqh\" (UniqueName: \"kubernetes.io/projected/2f8114ad-d2ca-48a8-b9af-349864c248c3-kube-api-access-k4lqh\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:44.926443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:44.926415 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f8114ad-d2ca-48a8-b9af-349864c248c3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:44.926823 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:44.926483 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f8114ad-d2ca-48a8-b9af-349864c248c3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:44.926823 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:44.926564 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 11:39:44.926823 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:44.926579 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 11:39:44.926823 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:44.926632 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f8114ad-d2ca-48a8-b9af-349864c248c3-insights-runtime-extractor-tls podName:2f8114ad-d2ca-48a8-b9af-349864c248c3 nodeName:}" failed. No retries permitted until 2026-09-17 11:39:45.926617953 +0000 UTC m=+55.934064960 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2f8114ad-d2ca-48a8-b9af-349864c248c3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qksp8" (UID: "2f8114ad-d2ca-48a8-b9af-349864c248c3") : secret "insights-runtime-extractor-tls" not found Sep 17 11:39:44.926823 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:44.926644 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f8114ad-d2ca-48a8-b9af-349864c248c3-kube-rbac-proxy-cm podName:2f8114ad-d2ca-48a8-b9af-349864c248c3 nodeName:}" failed. No retries permitted until 2026-09-17 11:39:45.926637398 +0000 UTC m=+55.934084406 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2f8114ad-d2ca-48a8-b9af-349864c248c3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qksp8" (UID: "2f8114ad-d2ca-48a8-b9af-349864c248c3") : configmap "kube-rbac-proxy" not found Sep 17 11:39:45.932547 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:45.932509 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f8114ad-d2ca-48a8-b9af-349864c248c3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:45.932985 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:45.932577 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f8114ad-d2ca-48a8-b9af-349864c248c3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:45.932985 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:45.932657 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 11:39:45.932985 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:45.932663 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 11:39:45.932985 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:45.932732 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f8114ad-d2ca-48a8-b9af-349864c248c3-insights-runtime-extractor-tls podName:2f8114ad-d2ca-48a8-b9af-349864c248c3 nodeName:}" failed. No retries permitted until 2026-09-17 11:39:47.932714734 +0000 UTC m=+57.940161747 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2f8114ad-d2ca-48a8-b9af-349864c248c3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qksp8" (UID: "2f8114ad-d2ca-48a8-b9af-349864c248c3") : secret "insights-runtime-extractor-tls" not found Sep 17 11:39:45.932985 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:45.932747 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f8114ad-d2ca-48a8-b9af-349864c248c3-kube-rbac-proxy-cm podName:2f8114ad-d2ca-48a8-b9af-349864c248c3 nodeName:}" failed. No retries permitted until 2026-09-17 11:39:47.932741155 +0000 UTC m=+57.940188168 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2f8114ad-d2ca-48a8-b9af-349864c248c3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qksp8" (UID: "2f8114ad-d2ca-48a8-b9af-349864c248c3") : configmap "kube-rbac-proxy" not found Sep 17 11:39:46.314181 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:46.314153 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wk6sh_e1152cde-f077-4e81-a902-d0cd724f0495/dns-node-resolver/0.log" Sep 17 11:39:46.912300 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:46.912275 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g6kpv_67153d8b-662e-4d11-b932-2ec9f96be172/node-ca/0.log" Sep 17 11:39:47.712433 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:47.712400 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nqcnx_d4657dd9-0b00-4a86-af8e-10392c1704f9/serve-healthcheck-canary/0.log" Sep 17 11:39:47.949002 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:47.948973 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f8114ad-d2ca-48a8-b9af-349864c248c3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:47.949182 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:47.949022 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f8114ad-d2ca-48a8-b9af-349864c248c3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:47.949182 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:47.949112 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 11:39:47.949182 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:47.949136 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 11:39:47.949182 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:47.949183 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f8114ad-d2ca-48a8-b9af-349864c248c3-kube-rbac-proxy-cm podName:2f8114ad-d2ca-48a8-b9af-349864c248c3 nodeName:}" failed. No retries permitted until 2026-09-17 11:39:51.949170518 +0000 UTC m=+61.956617525 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2f8114ad-d2ca-48a8-b9af-349864c248c3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qksp8" (UID: "2f8114ad-d2ca-48a8-b9af-349864c248c3") : configmap "kube-rbac-proxy" not found Sep 17 11:39:47.949342 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:47.949203 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f8114ad-d2ca-48a8-b9af-349864c248c3-insights-runtime-extractor-tls podName:2f8114ad-d2ca-48a8-b9af-349864c248c3 nodeName:}" failed. No retries permitted until 2026-09-17 11:39:51.949189731 +0000 UTC m=+61.956636745 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2f8114ad-d2ca-48a8-b9af-349864c248c3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qksp8" (UID: "2f8114ad-d2ca-48a8-b9af-349864c248c3") : secret "insights-runtime-extractor-tls" not found Sep 17 11:39:48.437836 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.437805 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-btxh2"] Sep 17 11:39:48.441033 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.441017 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-btxh2" Sep 17 11:39:48.445048 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.445025 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 17 11:39:48.445423 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.445405 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 17 11:39:48.445476 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.445446 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 17 11:39:48.446714 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.446698 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-fgpvq\"" Sep 17 11:39:48.467130 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.467089 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-btxh2"] Sep 17 11:39:48.546344 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.546307 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hkhqg"] Sep 17 11:39:48.548687 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.548673 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hkhqg" Sep 17 11:39:48.554075 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.554043 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sft5p\" (UniqueName: \"kubernetes.io/projected/21fcc133-9735-45b0-98c8-ba788b3b783a-kube-api-access-sft5p\") pod \"cluster-samples-operator-86499c89c4-btxh2\" (UID: \"21fcc133-9735-45b0-98c8-ba788b3b783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-btxh2" Sep 17 11:39:48.554209 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.554088 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/21fcc133-9735-45b0-98c8-ba788b3b783a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-btxh2\" (UID: \"21fcc133-9735-45b0-98c8-ba788b3b783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-btxh2" Sep 17 11:39:48.558186 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.558168 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 17 11:39:48.558273 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.558255 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 17 11:39:48.558316 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.558303 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-cd8tb\"" Sep 17 11:39:48.579281 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.579259 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hkhqg"] Sep 17 11:39:48.655181 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.655149 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz4p2\" (UniqueName: \"kubernetes.io/projected/17c82812-30fc-48a1-b945-d0708daa7629-kube-api-access-xz4p2\") pod \"volume-data-source-validator-86695b7cc7-hkhqg\" (UID: \"17c82812-30fc-48a1-b945-d0708daa7629\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hkhqg" Sep 17 11:39:48.655332 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.655197 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sft5p\" (UniqueName: \"kubernetes.io/projected/21fcc133-9735-45b0-98c8-ba788b3b783a-kube-api-access-sft5p\") pod \"cluster-samples-operator-86499c89c4-btxh2\" (UID: \"21fcc133-9735-45b0-98c8-ba788b3b783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-btxh2" Sep 17 11:39:48.655332 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.655260 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/21fcc133-9735-45b0-98c8-ba788b3b783a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-btxh2\" (UID: \"21fcc133-9735-45b0-98c8-ba788b3b783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-btxh2" Sep 17 11:39:48.655449 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:48.655351 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 11:39:48.655449 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:48.655405 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21fcc133-9735-45b0-98c8-ba788b3b783a-samples-operator-tls podName:21fcc133-9735-45b0-98c8-ba788b3b783a nodeName:}" failed. No retries permitted until 2026-09-17 11:39:49.155389495 +0000 UTC m=+59.162836503 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/21fcc133-9735-45b0-98c8-ba788b3b783a-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-btxh2" (UID: "21fcc133-9735-45b0-98c8-ba788b3b783a") : secret "samples-operator-tls" not found Sep 17 11:39:48.679587 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.679558 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sft5p\" (UniqueName: \"kubernetes.io/projected/21fcc133-9735-45b0-98c8-ba788b3b783a-kube-api-access-sft5p\") pod \"cluster-samples-operator-86499c89c4-btxh2\" (UID: \"21fcc133-9735-45b0-98c8-ba788b3b783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-btxh2" Sep 17 11:39:48.755829 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.755803 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz4p2\" (UniqueName: \"kubernetes.io/projected/17c82812-30fc-48a1-b945-d0708daa7629-kube-api-access-xz4p2\") pod \"volume-data-source-validator-86695b7cc7-hkhqg\" (UID: \"17c82812-30fc-48a1-b945-d0708daa7629\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hkhqg" Sep 17 11:39:48.765228 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.765200 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz4p2\" (UniqueName: \"kubernetes.io/projected/17c82812-30fc-48a1-b945-d0708daa7629-kube-api-access-xz4p2\") pod \"volume-data-source-validator-86695b7cc7-hkhqg\" (UID: \"17c82812-30fc-48a1-b945-d0708daa7629\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hkhqg" Sep 17 11:39:48.856491 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:48.856464 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hkhqg" Sep 17 11:39:49.132859 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.132781 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hkhqg"] Sep 17 11:39:49.136694 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:39:49.136664 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17c82812_30fc_48a1_b945_d0708daa7629.slice/crio-8998fb4b90ab6c9fd42c7599a410076f9c929b159a393142eed314c90e467739 WatchSource:0}: Error finding container 8998fb4b90ab6c9fd42c7599a410076f9c929b159a393142eed314c90e467739: Status 404 returned error can't find the container with id 8998fb4b90ab6c9fd42c7599a410076f9c929b159a393142eed314c90e467739 Sep 17 11:39:49.159832 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.159810 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/21fcc133-9735-45b0-98c8-ba788b3b783a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-btxh2\" (UID: \"21fcc133-9735-45b0-98c8-ba788b3b783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-btxh2" Sep 17 11:39:49.159996 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:49.159977 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 11:39:49.160150 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:49.160064 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21fcc133-9735-45b0-98c8-ba788b3b783a-samples-operator-tls podName:21fcc133-9735-45b0-98c8-ba788b3b783a nodeName:}" failed. No retries permitted until 2026-09-17 11:39:50.160043075 +0000 UTC m=+60.167490096 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/21fcc133-9735-45b0-98c8-ba788b3b783a-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-btxh2" (UID: "21fcc133-9735-45b0-98c8-ba788b3b783a") : secret "samples-operator-tls" not found Sep 17 11:39:49.458442 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.458354 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-gn9sh"] Sep 17 11:39:49.461982 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.461960 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" Sep 17 11:39:49.464856 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.464836 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 17 11:39:49.464965 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.464856 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 17 11:39:49.464965 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.464856 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 17 11:39:49.464965 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.464845 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-dd97n\"" Sep 17 11:39:49.465510 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.465492 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 17 11:39:49.470105 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.470086 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 17 11:39:49.475806 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.475782 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-gn9sh"] Sep 17 11:39:49.562757 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.562717 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/124b0387-2e6e-4f73-b71a-2ead313e3701-serving-cert\") pod \"console-operator-696884fb9c-gn9sh\" (UID: \"124b0387-2e6e-4f73-b71a-2ead313e3701\") " pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" Sep 17 11:39:49.562937 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.562830 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/124b0387-2e6e-4f73-b71a-2ead313e3701-trusted-ca\") pod \"console-operator-696884fb9c-gn9sh\" (UID: \"124b0387-2e6e-4f73-b71a-2ead313e3701\") " pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" Sep 17 11:39:49.562937 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.562907 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/124b0387-2e6e-4f73-b71a-2ead313e3701-config\") pod \"console-operator-696884fb9c-gn9sh\" (UID: \"124b0387-2e6e-4f73-b71a-2ead313e3701\") " pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" Sep 17 11:39:49.563051 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.562992 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk6sz\" (UniqueName: \"kubernetes.io/projected/124b0387-2e6e-4f73-b71a-2ead313e3701-kube-api-access-hk6sz\") pod \"console-operator-696884fb9c-gn9sh\" (UID: \"124b0387-2e6e-4f73-b71a-2ead313e3701\") " pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" Sep 17 11:39:49.592433 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.592405 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nn75c" Sep 17 11:39:49.664333 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.664305 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk6sz\" (UniqueName: \"kubernetes.io/projected/124b0387-2e6e-4f73-b71a-2ead313e3701-kube-api-access-hk6sz\") pod \"console-operator-696884fb9c-gn9sh\" (UID: \"124b0387-2e6e-4f73-b71a-2ead313e3701\") " pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" Sep 17 11:39:49.664333 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.664336 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/124b0387-2e6e-4f73-b71a-2ead313e3701-serving-cert\") pod \"console-operator-696884fb9c-gn9sh\" (UID: \"124b0387-2e6e-4f73-b71a-2ead313e3701\") " pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" Sep 17 11:39:49.664510 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.664409 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/124b0387-2e6e-4f73-b71a-2ead313e3701-trusted-ca\") pod \"console-operator-696884fb9c-gn9sh\" (UID: \"124b0387-2e6e-4f73-b71a-2ead313e3701\") " pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" Sep 17 11:39:49.664510 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.664495 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/124b0387-2e6e-4f73-b71a-2ead313e3701-config\") pod \"console-operator-696884fb9c-gn9sh\" (UID: \"124b0387-2e6e-4f73-b71a-2ead313e3701\") " pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" Sep 17 11:39:49.665250 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.665179 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/124b0387-2e6e-4f73-b71a-2ead313e3701-config\") pod \"console-operator-696884fb9c-gn9sh\" (UID: \"124b0387-2e6e-4f73-b71a-2ead313e3701\") " pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" Sep 17 11:39:49.665454 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.665435 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/124b0387-2e6e-4f73-b71a-2ead313e3701-trusted-ca\") pod \"console-operator-696884fb9c-gn9sh\" (UID: \"124b0387-2e6e-4f73-b71a-2ead313e3701\") " pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" Sep 17 11:39:49.666980 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.666955 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/124b0387-2e6e-4f73-b71a-2ead313e3701-serving-cert\") pod \"console-operator-696884fb9c-gn9sh\" (UID: \"124b0387-2e6e-4f73-b71a-2ead313e3701\") " pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" Sep 17 11:39:49.668239 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.668216 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hkhqg" event={"ID":"17c82812-30fc-48a1-b945-d0708daa7629","Type":"ContainerStarted","Data":"8998fb4b90ab6c9fd42c7599a410076f9c929b159a393142eed314c90e467739"} Sep 17 11:39:49.674357 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.674340 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk6sz\" (UniqueName: \"kubernetes.io/projected/124b0387-2e6e-4f73-b71a-2ead313e3701-kube-api-access-hk6sz\") pod \"console-operator-696884fb9c-gn9sh\" (UID: \"124b0387-2e6e-4f73-b71a-2ead313e3701\") " pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" Sep 17 11:39:49.770779 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:49.770759 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" Sep 17 11:39:50.059346 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:50.059263 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-gn9sh"] Sep 17 11:39:50.062770 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:39:50.062739 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod124b0387_2e6e_4f73_b71a_2ead313e3701.slice/crio-0c78532a43b51e47e01e1872022fd424ddbc878dd52b8accb3e8128331163fea WatchSource:0}: Error finding container 0c78532a43b51e47e01e1872022fd424ddbc878dd52b8accb3e8128331163fea: Status 404 returned error can't find the container with id 0c78532a43b51e47e01e1872022fd424ddbc878dd52b8accb3e8128331163fea Sep 17 11:39:50.169587 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:50.169559 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/21fcc133-9735-45b0-98c8-ba788b3b783a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-btxh2\" (UID: \"21fcc133-9735-45b0-98c8-ba788b3b783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-btxh2" Sep 17 11:39:50.169697 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:50.169667 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 11:39:50.169752 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:50.169745 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21fcc133-9735-45b0-98c8-ba788b3b783a-samples-operator-tls podName:21fcc133-9735-45b0-98c8-ba788b3b783a nodeName:}" failed. No retries permitted until 2026-09-17 11:39:52.169730038 +0000 UTC m=+62.177177047 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/21fcc133-9735-45b0-98c8-ba788b3b783a-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-btxh2" (UID: "21fcc133-9735-45b0-98c8-ba788b3b783a") : secret "samples-operator-tls" not found Sep 17 11:39:50.432681 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:50.432606 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qz2jt"] Sep 17 11:39:50.435635 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:50.435613 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qz2jt" Sep 17 11:39:50.438936 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:50.438915 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 17 11:39:50.439050 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:50.438946 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 17 11:39:50.439143 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:50.439094 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-rhs6g\"" Sep 17 11:39:50.439209 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:50.439149 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 17 11:39:50.439339 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:50.439314 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 17 11:39:50.448106 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:50.448085 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qz2jt"] Sep 17 11:39:50.572826 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:50.572788 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbb8t\" (UniqueName: \"kubernetes.io/projected/facf9640-adc2-4f43-a3c0-5d0b0521ba7f-kube-api-access-rbb8t\") pod \"kube-storage-version-migrator-operator-6c8f84f897-qz2jt\" (UID: \"facf9640-adc2-4f43-a3c0-5d0b0521ba7f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qz2jt" Sep 17 11:39:50.573000 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:50.572836 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/facf9640-adc2-4f43-a3c0-5d0b0521ba7f-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-qz2jt\" (UID: \"facf9640-adc2-4f43-a3c0-5d0b0521ba7f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qz2jt" Sep 17 11:39:50.573000 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:50.572940 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/facf9640-adc2-4f43-a3c0-5d0b0521ba7f-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-qz2jt\" (UID: \"facf9640-adc2-4f43-a3c0-5d0b0521ba7f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qz2jt" Sep 17 11:39:50.673539 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:50.673475 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/facf9640-adc2-4f43-a3c0-5d0b0521ba7f-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-qz2jt\" (UID: \"facf9640-adc2-4f43-a3c0-5d0b0521ba7f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qz2jt" Sep 17 11:39:50.673733 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:50.673618 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbb8t\" (UniqueName: \"kubernetes.io/projected/facf9640-adc2-4f43-a3c0-5d0b0521ba7f-kube-api-access-rbb8t\") pod \"kube-storage-version-migrator-operator-6c8f84f897-qz2jt\" (UID: \"facf9640-adc2-4f43-a3c0-5d0b0521ba7f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qz2jt" Sep 17 11:39:50.673733 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:50.673666 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/facf9640-adc2-4f43-a3c0-5d0b0521ba7f-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-qz2jt\" (UID: \"facf9640-adc2-4f43-a3c0-5d0b0521ba7f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qz2jt" Sep 17 11:39:50.674374 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:50.674343 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/facf9640-adc2-4f43-a3c0-5d0b0521ba7f-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-qz2jt\" (UID: \"facf9640-adc2-4f43-a3c0-5d0b0521ba7f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qz2jt" Sep 17 11:39:50.677104 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:50.677067 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" event={"ID":"124b0387-2e6e-4f73-b71a-2ead313e3701","Type":"ContainerStarted","Data":"0c78532a43b51e47e01e1872022fd424ddbc878dd52b8accb3e8128331163fea"} Sep 17 11:39:50.677523 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:50.677499 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/facf9640-adc2-4f43-a3c0-5d0b0521ba7f-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-qz2jt\" (UID: \"facf9640-adc2-4f43-a3c0-5d0b0521ba7f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qz2jt" Sep 17 11:39:50.685182 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:50.685133 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbb8t\" (UniqueName: \"kubernetes.io/projected/facf9640-adc2-4f43-a3c0-5d0b0521ba7f-kube-api-access-rbb8t\") pod \"kube-storage-version-migrator-operator-6c8f84f897-qz2jt\" (UID: \"facf9640-adc2-4f43-a3c0-5d0b0521ba7f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qz2jt" Sep 17 11:39:50.746349 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:50.746314 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qz2jt" Sep 17 11:39:51.090092 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:51.089674 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qz2jt"] Sep 17 11:39:51.092470 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:39:51.092432 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfacf9640_adc2_4f43_a3c0_5d0b0521ba7f.slice/crio-82e655647f8b0bed6faf0b69cf0c50bd816a70650bced9ba145dde9625c01c93 WatchSource:0}: Error finding container 82e655647f8b0bed6faf0b69cf0c50bd816a70650bced9ba145dde9625c01c93: Status 404 returned error can't find the container with id 82e655647f8b0bed6faf0b69cf0c50bd816a70650bced9ba145dde9625c01c93 Sep 17 11:39:51.681670 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:51.681631 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hkhqg" event={"ID":"17c82812-30fc-48a1-b945-d0708daa7629","Type":"ContainerStarted","Data":"c0ade971d5bf78ae30b60ed448085bfbdeddf97df7f3b80d6ba36acd71c19172"} Sep 17 11:39:51.683190 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:51.683161 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qz2jt" event={"ID":"facf9640-adc2-4f43-a3c0-5d0b0521ba7f","Type":"ContainerStarted","Data":"82e655647f8b0bed6faf0b69cf0c50bd816a70650bced9ba145dde9625c01c93"} Sep 17 11:39:51.704541 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:51.704484 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hkhqg" podStartSLOduration=1.9741133419999999 podStartE2EDuration="3.704467502s" totalImagesPullingTime="1.73035416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:39:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:39:51.703279008 +0000 UTC m=+61.710726038" watchObservedRunningTime="2026-09-17 11:39:51.704467502 +0000 UTC m=+61.711914587" Sep 17 11:39:51.984868 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:51.984772 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f8114ad-d2ca-48a8-b9af-349864c248c3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:51.985041 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:51.984876 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f8114ad-d2ca-48a8-b9af-349864c248c3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:39:51.985041 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:51.984948 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 11:39:51.985041 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:51.984991 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 11:39:51.985041 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:51.985041 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f8114ad-d2ca-48a8-b9af-349864c248c3-insights-runtime-extractor-tls podName:2f8114ad-d2ca-48a8-b9af-349864c248c3 nodeName:}" failed. No retries permitted until 2026-09-17 11:39:59.985018572 +0000 UTC m=+69.992465595 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2f8114ad-d2ca-48a8-b9af-349864c248c3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qksp8" (UID: "2f8114ad-d2ca-48a8-b9af-349864c248c3") : secret "insights-runtime-extractor-tls" not found Sep 17 11:39:51.985236 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:51.985060 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f8114ad-d2ca-48a8-b9af-349864c248c3-kube-rbac-proxy-cm podName:2f8114ad-d2ca-48a8-b9af-349864c248c3 nodeName:}" failed. No retries permitted until 2026-09-17 11:39:59.985051075 +0000 UTC m=+69.992498083 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2f8114ad-d2ca-48a8-b9af-349864c248c3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qksp8" (UID: "2f8114ad-d2ca-48a8-b9af-349864c248c3") : configmap "kube-rbac-proxy" not found Sep 17 11:39:52.186185 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:52.186142 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/21fcc133-9735-45b0-98c8-ba788b3b783a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-btxh2\" (UID: \"21fcc133-9735-45b0-98c8-ba788b3b783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-btxh2" Sep 17 11:39:52.186585 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:52.186309 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 11:39:52.186585 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:52.186387 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21fcc133-9735-45b0-98c8-ba788b3b783a-samples-operator-tls podName:21fcc133-9735-45b0-98c8-ba788b3b783a nodeName:}" failed. No retries permitted until 2026-09-17 11:39:56.18637037 +0000 UTC m=+66.193817382 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/21fcc133-9735-45b0-98c8-ba788b3b783a-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-btxh2" (UID: "21fcc133-9735-45b0-98c8-ba788b3b783a") : secret "samples-operator-tls" not found Sep 17 11:39:52.688944 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:52.688918 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/0.log" Sep 17 11:39:52.689108 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:52.688964 2581 generic.go:356] "Generic (PLEG): container finished" podID="124b0387-2e6e-4f73-b71a-2ead313e3701" containerID="9bab22a6d1869928c452ce52ab2d9c1eb520821cc7d12da290ee41920c511ace" exitCode=255 Sep 17 11:39:52.689108 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:52.689060 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" event={"ID":"124b0387-2e6e-4f73-b71a-2ead313e3701","Type":"ContainerDied","Data":"9bab22a6d1869928c452ce52ab2d9c1eb520821cc7d12da290ee41920c511ace"} Sep 17 11:39:52.689320 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:52.689300 2581 scope.go:117] "RemoveContainer" containerID="9bab22a6d1869928c452ce52ab2d9c1eb520821cc7d12da290ee41920c511ace" Sep 17 11:39:53.693691 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:53.693647 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qz2jt" event={"ID":"facf9640-adc2-4f43-a3c0-5d0b0521ba7f","Type":"ContainerStarted","Data":"de000a3426f4402076ca612685c48d6d3189224ae2eae9d92a5d30a77dd908b8"} Sep 17 11:39:53.695413 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:53.695395 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/1.log" Sep 17 11:39:53.695812 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:53.695796 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/0.log" Sep 17 11:39:53.695889 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:53.695828 2581 generic.go:356] "Generic (PLEG): container finished" podID="124b0387-2e6e-4f73-b71a-2ead313e3701" containerID="dd64a4f811f56214460b925af4a8fbe2069520fe63dcb3d1333a9b19bc9f6545" exitCode=255 Sep 17 11:39:53.695889 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:53.695853 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" event={"ID":"124b0387-2e6e-4f73-b71a-2ead313e3701","Type":"ContainerDied","Data":"dd64a4f811f56214460b925af4a8fbe2069520fe63dcb3d1333a9b19bc9f6545"} Sep 17 11:39:53.695967 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:53.695902 2581 scope.go:117] "RemoveContainer" containerID="9bab22a6d1869928c452ce52ab2d9c1eb520821cc7d12da290ee41920c511ace" Sep 17 11:39:53.696139 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:53.696102 2581 scope.go:117] "RemoveContainer" containerID="dd64a4f811f56214460b925af4a8fbe2069520fe63dcb3d1333a9b19bc9f6545" Sep 17 11:39:53.696313 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:53.696289 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-gn9sh_openshift-console-operator(124b0387-2e6e-4f73-b71a-2ead313e3701)\"" pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" podUID="124b0387-2e6e-4f73-b71a-2ead313e3701" Sep 17 11:39:53.765924 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:53.765867 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qz2jt" podStartSLOduration=1.6340790840000001 podStartE2EDuration="3.76584272s" totalImagesPullingTime="2.131763636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:39:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:39:53.726619537 +0000 UTC m=+63.734066566" watchObservedRunningTime="2026-09-17 11:39:53.76584272 +0000 UTC m=+63.773289751" Sep 17 11:39:54.699940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:54.699909 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/1.log" Sep 17 11:39:54.700375 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:54.700357 2581 scope.go:117] "RemoveContainer" containerID="dd64a4f811f56214460b925af4a8fbe2069520fe63dcb3d1333a9b19bc9f6545" Sep 17 11:39:54.700533 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:54.700510 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-gn9sh_openshift-console-operator(124b0387-2e6e-4f73-b71a-2ead313e3701)\"" pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" podUID="124b0387-2e6e-4f73-b71a-2ead313e3701" Sep 17 11:39:55.531136 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:55.531099 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-bxwdl"] Sep 17 11:39:55.534421 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:55.534405 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bxwdl" Sep 17 11:39:55.538474 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:55.538452 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-g8jrq\"" Sep 17 11:39:55.549684 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:55.549659 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-bxwdl"] Sep 17 11:39:55.606686 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:55.606654 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ljbr\" (UniqueName: \"kubernetes.io/projected/0d51f0af-c4a9-4e64-86c5-07bb72fcf19b-kube-api-access-5ljbr\") pod \"network-check-source-6f8b768455-bxwdl\" (UID: \"0d51f0af-c4a9-4e64-86c5-07bb72fcf19b\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-bxwdl" Sep 17 11:39:55.707739 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:55.707707 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ljbr\" (UniqueName: \"kubernetes.io/projected/0d51f0af-c4a9-4e64-86c5-07bb72fcf19b-kube-api-access-5ljbr\") pod \"network-check-source-6f8b768455-bxwdl\" (UID: \"0d51f0af-c4a9-4e64-86c5-07bb72fcf19b\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-bxwdl" Sep 17 11:39:55.729467 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:55.729437 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 17 11:39:55.739717 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:55.739688 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 17 11:39:55.749329 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:55.749306 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ljbr\" (UniqueName: \"kubernetes.io/projected/0d51f0af-c4a9-4e64-86c5-07bb72fcf19b-kube-api-access-5ljbr\") pod \"network-check-source-6f8b768455-bxwdl\" (UID: \"0d51f0af-c4a9-4e64-86c5-07bb72fcf19b\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-bxwdl" Sep 17 11:39:55.843502 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:55.843422 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bxwdl" Sep 17 11:39:56.111184 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:56.111085 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs\") pod \"network-metrics-daemon-f5svv\" (UID: \"757a11fc-3750-4ef9-94f6-6c6da0d5c95c\") " pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:39:56.114193 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:56.114172 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 17 11:39:56.121859 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:56.121838 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 17 11:39:56.121948 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:56.121901 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs podName:757a11fc-3750-4ef9-94f6-6c6da0d5c95c nodeName:}" failed. No retries permitted until 2026-09-17 11:41:00.121884006 +0000 UTC m=+130.129331014 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs") pod "network-metrics-daemon-f5svv" (UID: "757a11fc-3750-4ef9-94f6-6c6da0d5c95c") : secret "metrics-daemon-secret" not found Sep 17 11:39:56.124975 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:56.124953 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-bxwdl"] Sep 17 11:39:56.128202 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:39:56.128178 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d51f0af_c4a9_4e64_86c5_07bb72fcf19b.slice/crio-604d298e61134c2369a7ae19eede2a4eee224676333e9e1479f195eec08f8655 WatchSource:0}: Error finding container 604d298e61134c2369a7ae19eede2a4eee224676333e9e1479f195eec08f8655: Status 404 returned error can't find the container with id 604d298e61134c2369a7ae19eede2a4eee224676333e9e1479f195eec08f8655 Sep 17 11:39:56.211887 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:56.211849 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret\") pod \"global-pull-secret-syncer-6jmh4\" (UID: \"7a49d705-8c21-411b-8988-260b0158a9be\") " pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:39:56.211887 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:56.211890 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxqzg\" (UniqueName: \"kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg\") pod \"network-check-target-pk9p9\" (UID: \"78be981b-e482-4dcf-a146-ea8549a3a847\") " pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:39:56.212112 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:56.211934 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/21fcc133-9735-45b0-98c8-ba788b3b783a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-btxh2\" (UID: \"21fcc133-9735-45b0-98c8-ba788b3b783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-btxh2" Sep 17 11:39:56.212112 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:56.212057 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 11:39:56.212211 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:56.212148 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21fcc133-9735-45b0-98c8-ba788b3b783a-samples-operator-tls podName:21fcc133-9735-45b0-98c8-ba788b3b783a nodeName:}" failed. No retries permitted until 2026-09-17 11:40:04.212104898 +0000 UTC m=+74.219551922 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/21fcc133-9735-45b0-98c8-ba788b3b783a-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-btxh2" (UID: "21fcc133-9735-45b0-98c8-ba788b3b783a") : secret "samples-operator-tls" not found Sep 17 11:39:56.214669 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:56.214637 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxqzg\" (UniqueName: \"kubernetes.io/projected/78be981b-e482-4dcf-a146-ea8549a3a847-kube-api-access-cxqzg\") pod \"network-check-target-pk9p9\" (UID: \"78be981b-e482-4dcf-a146-ea8549a3a847\") " pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:39:56.215546 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:56.215531 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 11:39:56.225156 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:56.225112 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7a49d705-8c21-411b-8988-260b0158a9be-original-pull-secret\") pod \"global-pull-secret-syncer-6jmh4\" (UID: \"7a49d705-8c21-411b-8988-260b0158a9be\") " pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:39:56.338446 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:56.338393 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jmh4" Sep 17 11:39:56.437276 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:56.437251 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nrbmw\"" Sep 17 11:39:56.445330 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:56.445314 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:39:56.608168 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:56.608139 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6jmh4"] Sep 17 11:39:56.610792 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:39:56.610765 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a49d705_8c21_411b_8988_260b0158a9be.slice/crio-4bac5ffd34812cb97b90cce0775dcf29ca8dc3409a8f9c6e6163bc1c478166ba WatchSource:0}: Error finding container 4bac5ffd34812cb97b90cce0775dcf29ca8dc3409a8f9c6e6163bc1c478166ba: Status 404 returned error can't find the container with id 4bac5ffd34812cb97b90cce0775dcf29ca8dc3409a8f9c6e6163bc1c478166ba Sep 17 11:39:56.706840 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:56.706764 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bxwdl" event={"ID":"0d51f0af-c4a9-4e64-86c5-07bb72fcf19b","Type":"ContainerStarted","Data":"604d298e61134c2369a7ae19eede2a4eee224676333e9e1479f195eec08f8655"} Sep 17 11:39:56.707841 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:56.707818 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6jmh4" event={"ID":"7a49d705-8c21-411b-8988-260b0158a9be","Type":"ContainerStarted","Data":"4bac5ffd34812cb97b90cce0775dcf29ca8dc3409a8f9c6e6163bc1c478166ba"} Sep 17 11:39:56.715360 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:56.715343 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls\") pod \"dns-default-8b6dg\" (UID: \"a0d623f4-748b-4d82-9a70-41671b79d346\") " pod="openshift-dns/dns-default-8b6dg" Sep 17 11:39:56.715505 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:56.715492 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 11:39:56.715557 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:56.715548 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls podName:a0d623f4-748b-4d82-9a70-41671b79d346 nodeName:}" failed. No retries permitted until 2026-09-17 11:40:28.715533294 +0000 UTC m=+98.722980303 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls") pod "dns-default-8b6dg" (UID: "a0d623f4-748b-4d82-9a70-41671b79d346") : secret "dns-default-metrics-tls" not found Sep 17 11:39:56.718698 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:56.718622 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pk9p9"] Sep 17 11:39:56.721020 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:39:56.721000 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78be981b_e482_4dcf_a146_ea8549a3a847.slice/crio-9a9ccce454da129739554f30a1029455cca5c74bfd2a8a9d9fd2885805825c2f WatchSource:0}: Error finding container 9a9ccce454da129739554f30a1029455cca5c74bfd2a8a9d9fd2885805825c2f: Status 404 returned error can't find the container with id 9a9ccce454da129739554f30a1029455cca5c74bfd2a8a9d9fd2885805825c2f Sep 17 11:39:57.713249 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:57.713195 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pk9p9" event={"ID":"78be981b-e482-4dcf-a146-ea8549a3a847","Type":"ContainerStarted","Data":"9a9ccce454da129739554f30a1029455cca5c74bfd2a8a9d9fd2885805825c2f"} Sep 17 11:39:59.771748 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:59.771709 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" Sep 17 11:39:59.772196 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:59.771815 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" Sep 17 11:39:59.772255 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:39:59.772224 2581 scope.go:117] "RemoveContainer" containerID="dd64a4f811f56214460b925af4a8fbe2069520fe63dcb3d1333a9b19bc9f6545" Sep 17 11:39:59.772430 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:39:59.772406 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-gn9sh_openshift-console-operator(124b0387-2e6e-4f73-b71a-2ead313e3701)\"" pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" podUID="124b0387-2e6e-4f73-b71a-2ead313e3701" Sep 17 11:40:00.045059 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:00.044976 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f8114ad-d2ca-48a8-b9af-349864c248c3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:40:00.045059 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:00.045052 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f8114ad-d2ca-48a8-b9af-349864c248c3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:40:00.045284 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:40:00.045162 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 11:40:00.045284 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:40:00.045239 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f8114ad-d2ca-48a8-b9af-349864c248c3-insights-runtime-extractor-tls podName:2f8114ad-d2ca-48a8-b9af-349864c248c3 nodeName:}" failed. No retries permitted until 2026-09-17 11:40:16.045222 +0000 UTC m=+86.052669008 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2f8114ad-d2ca-48a8-b9af-349864c248c3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qksp8" (UID: "2f8114ad-d2ca-48a8-b9af-349864c248c3") : secret "insights-runtime-extractor-tls" not found Sep 17 11:40:00.045732 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:00.045705 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f8114ad-d2ca-48a8-b9af-349864c248c3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:40:01.727486 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:01.727450 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bxwdl" event={"ID":"0d51f0af-c4a9-4e64-86c5-07bb72fcf19b","Type":"ContainerStarted","Data":"177266074c6ed2ead641a84f85401ab410f9c8dcd2925476e3c9780ccce45019"} Sep 17 11:40:01.729180 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:01.729155 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pk9p9" event={"ID":"78be981b-e482-4dcf-a146-ea8549a3a847","Type":"ContainerStarted","Data":"675ae10114085d63aab8173643291b50ec27c272299bdee420b52d3c2047c000"} Sep 17 11:40:01.729329 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:01.729218 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:40:01.730736 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:01.730717 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6jmh4" event={"ID":"7a49d705-8c21-411b-8988-260b0158a9be","Type":"ContainerStarted","Data":"f1d5c155f18c603c08c0ae40b770ae71d7b3aeaa701503e2516f341a3ad5cc39"} Sep 17 11:40:01.755715 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:01.755673 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bxwdl" podStartSLOduration=1.794548386 podStartE2EDuration="6.755658275s" totalImagesPullingTime="4.961109889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:39:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:40:01.753471347 +0000 UTC m=+71.760918376" watchObservedRunningTime="2026-09-17 11:40:01.755658275 +0000 UTC m=+71.763105305" Sep 17 11:40:01.780798 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:01.780749 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6jmh4" podStartSLOduration=67.297434182 podStartE2EDuration="1m11.780734001s" totalImagesPullingTime="4.483299819s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 11:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:40:01.778307285 +0000 UTC m=+71.785754315" watchObservedRunningTime="2026-09-17 11:40:01.780734001 +0000 UTC m=+71.788181031" Sep 17 11:40:01.802038 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:01.801981 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pk9p9" podStartSLOduration=67.418844376 podStartE2EDuration="1m11.801960277s" totalImagesPullingTime="4.383115901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:40:01.801043961 +0000 UTC m=+71.808490992" watchObservedRunningTime="2026-09-17 11:40:01.801960277 +0000 UTC m=+71.809407305" Sep 17 11:40:04.279361 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:04.279319 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/21fcc133-9735-45b0-98c8-ba788b3b783a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-btxh2\" (UID: \"21fcc133-9735-45b0-98c8-ba788b3b783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-btxh2" Sep 17 11:40:04.281963 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:04.281939 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/21fcc133-9735-45b0-98c8-ba788b3b783a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-btxh2\" (UID: \"21fcc133-9735-45b0-98c8-ba788b3b783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-btxh2" Sep 17 11:40:04.353028 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:04.352994 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-fgpvq\"" Sep 17 11:40:04.360537 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:04.360518 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-btxh2" Sep 17 11:40:04.638592 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:04.638517 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-btxh2"] Sep 17 11:40:04.746057 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:04.746023 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-btxh2" event={"ID":"21fcc133-9735-45b0-98c8-ba788b3b783a","Type":"ContainerStarted","Data":"0dd52a7c177159f00cbc092f3c218c087cbea6dfbba3da86ec42ae5279525582"} Sep 17 11:40:06.755489 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:06.755460 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-btxh2" event={"ID":"21fcc133-9735-45b0-98c8-ba788b3b783a","Type":"ContainerStarted","Data":"9e27b606d079fa4f3b1396862fe716a0638dae8dea7c09e7e051c9b0f80f1ec9"} Sep 17 11:40:06.755928 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:06.755511 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-btxh2" event={"ID":"21fcc133-9735-45b0-98c8-ba788b3b783a","Type":"ContainerStarted","Data":"798f8b78bfd0bc0f2fec867b9d806e9e07e5be4228685c8c3ea282e7cfae90f0"} Sep 17 11:40:06.793386 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:06.793338 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-btxh2" podStartSLOduration=17.023884627 podStartE2EDuration="18.793323169s" totalImagesPullingTime="1.769438542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:39:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:40:06.79197055 +0000 UTC m=+76.799417579" watchObservedRunningTime="2026-09-17 11:40:06.793323169 +0000 UTC m=+76.800770198" Sep 17 11:40:10.424739 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:10.424712 2581 scope.go:117] "RemoveContainer" containerID="dd64a4f811f56214460b925af4a8fbe2069520fe63dcb3d1333a9b19bc9f6545" Sep 17 11:40:10.545020 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:40:10.544998 2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod124b0387_2e6e_4f73_b71a_2ead313e3701.slice/crio-conmon-6600ed4c783ab9f65d3e517b89fd03b8f7694d5e46a6301b4e7831f526d02ead.scope\": RecentStats: unable to find data in memory cache]" Sep 17 11:40:10.768639 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:10.768612 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/2.log" Sep 17 11:40:10.769043 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:10.769029 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/1.log" Sep 17 11:40:10.769093 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:10.769060 2581 generic.go:356] "Generic (PLEG): container finished" podID="124b0387-2e6e-4f73-b71a-2ead313e3701" containerID="6600ed4c783ab9f65d3e517b89fd03b8f7694d5e46a6301b4e7831f526d02ead" exitCode=255 Sep 17 11:40:10.769162 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:10.769147 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" event={"ID":"124b0387-2e6e-4f73-b71a-2ead313e3701","Type":"ContainerDied","Data":"6600ed4c783ab9f65d3e517b89fd03b8f7694d5e46a6301b4e7831f526d02ead"} Sep 17 11:40:10.769207 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:10.769197 2581 scope.go:117] "RemoveContainer" containerID="dd64a4f811f56214460b925af4a8fbe2069520fe63dcb3d1333a9b19bc9f6545" Sep 17 11:40:10.769515 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:10.769496 2581 scope.go:117] "RemoveContainer" containerID="6600ed4c783ab9f65d3e517b89fd03b8f7694d5e46a6301b4e7831f526d02ead" Sep 17 11:40:10.769738 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:40:10.769684 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-gn9sh_openshift-console-operator(124b0387-2e6e-4f73-b71a-2ead313e3701)\"" pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" podUID="124b0387-2e6e-4f73-b71a-2ead313e3701" Sep 17 11:40:11.778254 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:11.778150 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/2.log" Sep 17 11:40:16.071348 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:16.071308 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f8114ad-d2ca-48a8-b9af-349864c248c3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:40:16.073713 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:16.073691 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f8114ad-d2ca-48a8-b9af-349864c248c3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qksp8\" (UID: \"2f8114ad-d2ca-48a8-b9af-349864c248c3\") " pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:40:16.290993 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:16.290958 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qklrf\"" Sep 17 11:40:16.299181 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:16.299161 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qksp8" Sep 17 11:40:16.576934 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:16.576907 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qksp8"] Sep 17 11:40:16.580375 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:40:16.580343 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f8114ad_d2ca_48a8_b9af_349864c248c3.slice/crio-296f62834b6493ea728abefa4c6a1aed8445f013812c0987cc930935b835d347 WatchSource:0}: Error finding container 296f62834b6493ea728abefa4c6a1aed8445f013812c0987cc930935b835d347: Status 404 returned error can't find the container with id 296f62834b6493ea728abefa4c6a1aed8445f013812c0987cc930935b835d347 Sep 17 11:40:16.795563 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:16.795523 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qksp8" event={"ID":"2f8114ad-d2ca-48a8-b9af-349864c248c3","Type":"ContainerStarted","Data":"4a0c504997522ba9b25b03120b7628eaf5cb38fe49eb77b48dfd4fa2860a4272"} Sep 17 11:40:16.795726 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:16.795593 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qksp8" event={"ID":"2f8114ad-d2ca-48a8-b9af-349864c248c3","Type":"ContainerStarted","Data":"296f62834b6493ea728abefa4c6a1aed8445f013812c0987cc930935b835d347"} Sep 17 11:40:17.802300 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:17.802261 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qksp8" event={"ID":"2f8114ad-d2ca-48a8-b9af-349864c248c3","Type":"ContainerStarted","Data":"71952f57dba1797663ca7bafcad52363fa29642539d583bf98af0a1feea01959"} Sep 17 11:40:18.808439 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:18.808404 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qksp8" event={"ID":"2f8114ad-d2ca-48a8-b9af-349864c248c3","Type":"ContainerStarted","Data":"2948be12c9e8b9d3222baf7f86508456db63ccc540118f224644380e89eaba46"} Sep 17 11:40:18.829478 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:18.829430 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qksp8" podStartSLOduration=32.870562183 podStartE2EDuration="34.829413535s" totalImagesPullingTime="1.958851352s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 11:39:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:40:18.828057418 +0000 UTC m=+88.835504448" watchObservedRunningTime="2026-09-17 11:40:18.829413535 +0000 UTC m=+88.836860565" Sep 17 11:40:19.771438 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:19.771404 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" Sep 17 11:40:19.771599 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:19.771462 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" Sep 17 11:40:19.771772 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:19.771756 2581 scope.go:117] "RemoveContainer" containerID="6600ed4c783ab9f65d3e517b89fd03b8f7694d5e46a6301b4e7831f526d02ead" Sep 17 11:40:19.771919 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:40:19.771902 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-gn9sh_openshift-console-operator(124b0387-2e6e-4f73-b71a-2ead313e3701)\"" pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" podUID="124b0387-2e6e-4f73-b71a-2ead313e3701" Sep 17 11:40:20.671266 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.671228 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75dd4cb98-l5cb7"] Sep 17 11:40:20.674632 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.674610 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75dd4cb98-l5cb7" Sep 17 11:40:20.678755 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.678729 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 17 11:40:20.678874 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.678772 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-tg8p6\"" Sep 17 11:40:20.678874 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.678863 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 17 11:40:20.680947 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.680923 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 17 11:40:20.681401 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.681170 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 17 11:40:20.693729 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.693701 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75dd4cb98-l5cb7"] Sep 17 11:40:20.715501 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.715471 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-659fbbd545-hznqv"] Sep 17 11:40:20.718427 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.718409 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-659fbbd545-hznqv" Sep 17 11:40:20.721197 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.721178 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 17 11:40:20.747538 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.747514 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-659fbbd545-hznqv"] Sep 17 11:40:20.808265 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.808238 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9608f58c-67d7-44df-86f6-2b6fe501d1ba-tmp\") pod \"klusterlet-addon-workmgr-659fbbd545-hznqv\" (UID: \"9608f58c-67d7-44df-86f6-2b6fe501d1ba\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-659fbbd545-hznqv" Sep 17 11:40:20.808392 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.808299 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45nkk\" (UniqueName: \"kubernetes.io/projected/9608f58c-67d7-44df-86f6-2b6fe501d1ba-kube-api-access-45nkk\") pod \"klusterlet-addon-workmgr-659fbbd545-hznqv\" (UID: \"9608f58c-67d7-44df-86f6-2b6fe501d1ba\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-659fbbd545-hznqv" Sep 17 11:40:20.808392 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.808326 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/75c865d6-9091-40cc-934c-e7e60ecd571e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-75dd4cb98-l5cb7\" (UID: \"75c865d6-9091-40cc-934c-e7e60ecd571e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75dd4cb98-l5cb7" Sep 17 11:40:20.808392 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.808349 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5nld\" (UniqueName: \"kubernetes.io/projected/75c865d6-9091-40cc-934c-e7e60ecd571e-kube-api-access-d5nld\") pod \"managed-serviceaccount-addon-agent-75dd4cb98-l5cb7\" (UID: \"75c865d6-9091-40cc-934c-e7e60ecd571e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75dd4cb98-l5cb7" Sep 17 11:40:20.808392 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.808385 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9608f58c-67d7-44df-86f6-2b6fe501d1ba-klusterlet-config\") pod \"klusterlet-addon-workmgr-659fbbd545-hznqv\" (UID: \"9608f58c-67d7-44df-86f6-2b6fe501d1ba\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-659fbbd545-hznqv" Sep 17 11:40:20.909383 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.909347 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9608f58c-67d7-44df-86f6-2b6fe501d1ba-klusterlet-config\") pod \"klusterlet-addon-workmgr-659fbbd545-hznqv\" (UID: \"9608f58c-67d7-44df-86f6-2b6fe501d1ba\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-659fbbd545-hznqv" Sep 17 11:40:20.909383 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.909389 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9608f58c-67d7-44df-86f6-2b6fe501d1ba-tmp\") pod \"klusterlet-addon-workmgr-659fbbd545-hznqv\" (UID: \"9608f58c-67d7-44df-86f6-2b6fe501d1ba\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-659fbbd545-hznqv" Sep 17 11:40:20.909614 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.909446 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45nkk\" (UniqueName: \"kubernetes.io/projected/9608f58c-67d7-44df-86f6-2b6fe501d1ba-kube-api-access-45nkk\") pod \"klusterlet-addon-workmgr-659fbbd545-hznqv\" (UID: \"9608f58c-67d7-44df-86f6-2b6fe501d1ba\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-659fbbd545-hznqv" Sep 17 11:40:20.909614 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.909463 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/75c865d6-9091-40cc-934c-e7e60ecd571e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-75dd4cb98-l5cb7\" (UID: \"75c865d6-9091-40cc-934c-e7e60ecd571e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75dd4cb98-l5cb7" Sep 17 11:40:20.909614 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.909486 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5nld\" (UniqueName: \"kubernetes.io/projected/75c865d6-9091-40cc-934c-e7e60ecd571e-kube-api-access-d5nld\") pod \"managed-serviceaccount-addon-agent-75dd4cb98-l5cb7\" (UID: \"75c865d6-9091-40cc-934c-e7e60ecd571e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75dd4cb98-l5cb7" Sep 17 11:40:20.909919 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.909887 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9608f58c-67d7-44df-86f6-2b6fe501d1ba-tmp\") pod \"klusterlet-addon-workmgr-659fbbd545-hznqv\" (UID: \"9608f58c-67d7-44df-86f6-2b6fe501d1ba\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-659fbbd545-hznqv" Sep 17 11:40:20.912154 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.912109 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9608f58c-67d7-44df-86f6-2b6fe501d1ba-klusterlet-config\") pod \"klusterlet-addon-workmgr-659fbbd545-hznqv\" (UID: \"9608f58c-67d7-44df-86f6-2b6fe501d1ba\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-659fbbd545-hznqv" Sep 17 11:40:20.912274 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.912252 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/75c865d6-9091-40cc-934c-e7e60ecd571e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-75dd4cb98-l5cb7\" (UID: \"75c865d6-9091-40cc-934c-e7e60ecd571e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75dd4cb98-l5cb7" Sep 17 11:40:20.922446 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.922385 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5nld\" (UniqueName: \"kubernetes.io/projected/75c865d6-9091-40cc-934c-e7e60ecd571e-kube-api-access-d5nld\") pod \"managed-serviceaccount-addon-agent-75dd4cb98-l5cb7\" (UID: \"75c865d6-9091-40cc-934c-e7e60ecd571e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75dd4cb98-l5cb7" Sep 17 11:40:20.922446 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.922434 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45nkk\" (UniqueName: \"kubernetes.io/projected/9608f58c-67d7-44df-86f6-2b6fe501d1ba-kube-api-access-45nkk\") pod \"klusterlet-addon-workmgr-659fbbd545-hznqv\" (UID: \"9608f58c-67d7-44df-86f6-2b6fe501d1ba\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-659fbbd545-hznqv" Sep 17 11:40:20.996174 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:20.996148 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75dd4cb98-l5cb7" Sep 17 11:40:21.027407 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:21.027384 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-659fbbd545-hznqv" Sep 17 11:40:21.287626 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:21.287600 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75dd4cb98-l5cb7"] Sep 17 11:40:21.289270 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:40:21.289247 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75c865d6_9091_40cc_934c_e7e60ecd571e.slice/crio-7f340c1d582abc75bff27f649f444592ab13e8616569e62ddda903e2c5257042 WatchSource:0}: Error finding container 7f340c1d582abc75bff27f649f444592ab13e8616569e62ddda903e2c5257042: Status 404 returned error can't find the container with id 7f340c1d582abc75bff27f649f444592ab13e8616569e62ddda903e2c5257042 Sep 17 11:40:21.309174 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:21.309152 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-659fbbd545-hznqv"] Sep 17 11:40:21.312231 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:40:21.312201 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9608f58c_67d7_44df_86f6_2b6fe501d1ba.slice/crio-88af04b9ee597cbdbf438a04f3bfc068f877331aa866db4fea67d0a8b55a2516 WatchSource:0}: Error finding container 88af04b9ee597cbdbf438a04f3bfc068f877331aa866db4fea67d0a8b55a2516: Status 404 returned error can't find the container with id 88af04b9ee597cbdbf438a04f3bfc068f877331aa866db4fea67d0a8b55a2516 Sep 17 11:40:21.818790 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:21.818754 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75dd4cb98-l5cb7" event={"ID":"75c865d6-9091-40cc-934c-e7e60ecd571e","Type":"ContainerStarted","Data":"7f340c1d582abc75bff27f649f444592ab13e8616569e62ddda903e2c5257042"} Sep 17 11:40:21.819821 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:21.819799 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-659fbbd545-hznqv" event={"ID":"9608f58c-67d7-44df-86f6-2b6fe501d1ba","Type":"ContainerStarted","Data":"88af04b9ee597cbdbf438a04f3bfc068f877331aa866db4fea67d0a8b55a2516"} Sep 17 11:40:23.834918 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:23.834879 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75dd4cb98-l5cb7" event={"ID":"75c865d6-9091-40cc-934c-e7e60ecd571e","Type":"ContainerStarted","Data":"ce5eeb2f49ef6735ad9e2ae512971234ccc1dceb06f65f4ef979bb6a8e9ca54a"} Sep 17 11:40:23.854136 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:23.854081 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75dd4cb98-l5cb7" podStartSLOduration=1.4044960770000001 podStartE2EDuration="3.854066306s" totalImagesPullingTime="2.449570229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:40:23.852966975 +0000 UTC m=+93.860414005" watchObservedRunningTime="2026-09-17 11:40:23.854066306 +0000 UTC m=+93.861513359" Sep 17 11:40:28.773856 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:28.773822 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls\") pod \"dns-default-8b6dg\" (UID: \"a0d623f4-748b-4d82-9a70-41671b79d346\") " pod="openshift-dns/dns-default-8b6dg" Sep 17 11:40:28.776236 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:28.776217 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0d623f4-748b-4d82-9a70-41671b79d346-metrics-tls\") pod \"dns-default-8b6dg\" (UID: \"a0d623f4-748b-4d82-9a70-41671b79d346\") " pod="openshift-dns/dns-default-8b6dg" Sep 17 11:40:28.809095 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:28.809072 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hs2cv\"" Sep 17 11:40:28.817319 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:28.817299 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8b6dg" Sep 17 11:40:29.097210 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.097112 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8b6dg"] Sep 17 11:40:29.100217 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:40:29.100188 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0d623f4_748b_4d82_9a70_41671b79d346.slice/crio-068891e771bff17f27388e128db1ff354fe8e221d680b40a157f9d4fa6679556 WatchSource:0}: Error finding container 068891e771bff17f27388e128db1ff354fe8e221d680b40a157f9d4fa6679556: Status 404 returned error can't find the container with id 068891e771bff17f27388e128db1ff354fe8e221d680b40a157f9d4fa6679556 Sep 17 11:40:29.856363 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.856317 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8b6dg" event={"ID":"a0d623f4-748b-4d82-9a70-41671b79d346","Type":"ContainerStarted","Data":"068891e771bff17f27388e128db1ff354fe8e221d680b40a157f9d4fa6679556"} Sep 17 11:40:29.860693 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.860663 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8"] Sep 17 11:40:29.864967 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.864946 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-pmps2"] Sep 17 11:40:29.865092 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.865073 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" Sep 17 11:40:29.868051 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.868027 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 17 11:40:29.869047 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.869028 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:29.869441 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.869415 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 17 11:40:29.870566 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.870548 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-q9zv2\"" Sep 17 11:40:29.871954 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.871733 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 17 11:40:29.871954 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.871951 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 17 11:40:29.872088 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.871977 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 17 11:40:29.872088 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.871987 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 17 11:40:29.872088 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.872056 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 17 11:40:29.874414 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.874393 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bphnw\"" Sep 17 11:40:29.875885 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.875865 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 17 11:40:29.876340 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.876323 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 17 11:40:29.890110 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.890087 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8"] Sep 17 11:40:29.983725 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.983675 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-node-exporter-wtmp\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:29.983885 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.983733 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-root\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:29.983885 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.983773 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/10929b4d-9cbb-45fd-92ee-427667b142c6-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-2ptb8\" (UID: \"10929b4d-9cbb-45fd-92ee-427667b142c6\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" Sep 17 11:40:29.983885 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.983803 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/10929b4d-9cbb-45fd-92ee-427667b142c6-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-2ptb8\" (UID: \"10929b4d-9cbb-45fd-92ee-427667b142c6\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" Sep 17 11:40:29.983885 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.983832 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/10929b4d-9cbb-45fd-92ee-427667b142c6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-2ptb8\" (UID: \"10929b4d-9cbb-45fd-92ee-427667b142c6\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" Sep 17 11:40:29.983885 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.983865 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10929b4d-9cbb-45fd-92ee-427667b142c6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-2ptb8\" (UID: \"10929b4d-9cbb-45fd-92ee-427667b142c6\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" Sep 17 11:40:29.984160 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.983906 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-node-exporter-accelerators-collector-config\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:29.984160 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.983958 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-node-exporter-tls\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:29.984160 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.983998 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p8km\" (UniqueName: \"kubernetes.io/projected/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-kube-api-access-5p8km\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:29.984160 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.984062 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-node-exporter-textfile\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:29.984160 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.984086 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10929b4d-9cbb-45fd-92ee-427667b142c6-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-2ptb8\" (UID: \"10929b4d-9cbb-45fd-92ee-427667b142c6\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" Sep 17 11:40:29.984160 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.984146 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:29.984392 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.984184 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-sys\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:29.984392 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.984209 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-metrics-client-ca\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:29.984392 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:29.984302 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgjzn\" (UniqueName: \"kubernetes.io/projected/10929b4d-9cbb-45fd-92ee-427667b142c6-kube-api-access-tgjzn\") pod \"kube-state-metrics-59fbdd77d-2ptb8\" (UID: \"10929b4d-9cbb-45fd-92ee-427667b142c6\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" Sep 17 11:40:30.085134 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.085095 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-node-exporter-wtmp\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:30.085330 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.085152 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-root\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:30.085330 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.085183 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/10929b4d-9cbb-45fd-92ee-427667b142c6-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-2ptb8\" (UID: \"10929b4d-9cbb-45fd-92ee-427667b142c6\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" Sep 17 11:40:30.085330 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.085209 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/10929b4d-9cbb-45fd-92ee-427667b142c6-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-2ptb8\" (UID: \"10929b4d-9cbb-45fd-92ee-427667b142c6\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" Sep 17 11:40:30.085330 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.085237 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/10929b4d-9cbb-45fd-92ee-427667b142c6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-2ptb8\" (UID: \"10929b4d-9cbb-45fd-92ee-427667b142c6\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" Sep 17 11:40:30.085330 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.085268 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10929b4d-9cbb-45fd-92ee-427667b142c6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-2ptb8\" (UID: \"10929b4d-9cbb-45fd-92ee-427667b142c6\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" Sep 17 11:40:30.085330 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.085303 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-node-exporter-wtmp\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:30.085618 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:40:30.085334 2581 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Sep 17 11:40:30.085618 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:40:30.085409 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10929b4d-9cbb-45fd-92ee-427667b142c6-kube-state-metrics-tls podName:10929b4d-9cbb-45fd-92ee-427667b142c6 nodeName:}" failed. No retries permitted until 2026-09-17 11:40:30.585389533 +0000 UTC m=+100.592836562 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/10929b4d-9cbb-45fd-92ee-427667b142c6-kube-state-metrics-tls") pod "kube-state-metrics-59fbdd77d-2ptb8" (UID: "10929b4d-9cbb-45fd-92ee-427667b142c6") : secret "kube-state-metrics-tls" not found Sep 17 11:40:30.085618 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.085269 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-root\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:30.085618 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.085308 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-node-exporter-accelerators-collector-config\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:30.085618 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.085469 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-node-exporter-tls\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:30.085618 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.085494 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p8km\" (UniqueName: \"kubernetes.io/projected/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-kube-api-access-5p8km\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:30.085618 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.085542 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-node-exporter-textfile\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:30.085618 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.085567 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10929b4d-9cbb-45fd-92ee-427667b142c6-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-2ptb8\" (UID: \"10929b4d-9cbb-45fd-92ee-427667b142c6\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" Sep 17 11:40:30.085618 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.085599 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:30.086054 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.085639 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-sys\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:30.086054 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.085665 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-metrics-client-ca\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:30.086054 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.085706 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgjzn\" (UniqueName: \"kubernetes.io/projected/10929b4d-9cbb-45fd-92ee-427667b142c6-kube-api-access-tgjzn\") pod \"kube-state-metrics-59fbdd77d-2ptb8\" (UID: \"10929b4d-9cbb-45fd-92ee-427667b142c6\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" Sep 17 11:40:30.086054 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.085731 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/10929b4d-9cbb-45fd-92ee-427667b142c6-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-2ptb8\" (UID: \"10929b4d-9cbb-45fd-92ee-427667b142c6\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" Sep 17 11:40:30.086054 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.085876 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-node-exporter-accelerators-collector-config\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:30.086357 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.086059 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-sys\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:30.086357 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.086161 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/10929b4d-9cbb-45fd-92ee-427667b142c6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-2ptb8\" (UID: \"10929b4d-9cbb-45fd-92ee-427667b142c6\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" Sep 17 11:40:30.086357 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.086061 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-node-exporter-textfile\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:30.086611 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.086586 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10929b4d-9cbb-45fd-92ee-427667b142c6-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-2ptb8\" (UID: \"10929b4d-9cbb-45fd-92ee-427667b142c6\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" Sep 17 11:40:30.087062 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.087035 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-metrics-client-ca\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:30.088773 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.088745 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-node-exporter-tls\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:30.088890 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.088873 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10929b4d-9cbb-45fd-92ee-427667b142c6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-2ptb8\" (UID: \"10929b4d-9cbb-45fd-92ee-427667b142c6\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" Sep 17 11:40:30.089206 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.089185 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:30.097704 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.097683 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p8km\" (UniqueName: \"kubernetes.io/projected/ddf4132f-a985-4fa4-96dc-1a2d4e0d810e-kube-api-access-5p8km\") pod \"node-exporter-pmps2\" (UID: \"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e\") " pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:30.097996 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.097968 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgjzn\" (UniqueName: \"kubernetes.io/projected/10929b4d-9cbb-45fd-92ee-427667b142c6-kube-api-access-tgjzn\") pod \"kube-state-metrics-59fbdd77d-2ptb8\" (UID: \"10929b4d-9cbb-45fd-92ee-427667b142c6\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" Sep 17 11:40:30.185950 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.185865 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pmps2" Sep 17 11:40:30.199849 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:40:30.199819 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddf4132f_a985_4fa4_96dc_1a2d4e0d810e.slice/crio-0ff6674abde21b4c3a54600c773124bd8342409b22e2cd21739b77cc7c263231 WatchSource:0}: Error finding container 0ff6674abde21b4c3a54600c773124bd8342409b22e2cd21739b77cc7c263231: Status 404 returned error can't find the container with id 0ff6674abde21b4c3a54600c773124bd8342409b22e2cd21739b77cc7c263231 Sep 17 11:40:30.425159 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.424748 2581 scope.go:117] "RemoveContainer" containerID="6600ed4c783ab9f65d3e517b89fd03b8f7694d5e46a6301b4e7831f526d02ead" Sep 17 11:40:30.425159 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:40:30.424962 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-gn9sh_openshift-console-operator(124b0387-2e6e-4f73-b71a-2ead313e3701)\"" pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" podUID="124b0387-2e6e-4f73-b71a-2ead313e3701" Sep 17 11:40:30.590149 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.590103 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/10929b4d-9cbb-45fd-92ee-427667b142c6-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-2ptb8\" (UID: \"10929b4d-9cbb-45fd-92ee-427667b142c6\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" Sep 17 11:40:30.592944 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.592921 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/10929b4d-9cbb-45fd-92ee-427667b142c6-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-2ptb8\" (UID: \"10929b4d-9cbb-45fd-92ee-427667b142c6\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" Sep 17 11:40:30.778250 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.778227 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" Sep 17 11:40:30.862381 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:30.862355 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pmps2" event={"ID":"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e","Type":"ContainerStarted","Data":"0ff6674abde21b4c3a54600c773124bd8342409b22e2cd21739b77cc7c263231"} Sep 17 11:40:31.074170 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.074138 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8"] Sep 17 11:40:31.079959 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:40:31.079918 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10929b4d_9cbb_45fd_92ee_427667b142c6.slice/crio-ab1216b9987867f7be6aad6d6a0118138d3120a19ee3ff882da8e7e3ffb1c761 WatchSource:0}: Error finding container ab1216b9987867f7be6aad6d6a0118138d3120a19ee3ff882da8e7e3ffb1c761: Status 404 returned error can't find the container with id ab1216b9987867f7be6aad6d6a0118138d3120a19ee3ff882da8e7e3ffb1c761 Sep 17 11:40:31.780095 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.780037 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6b877f875b-r8bm6"] Sep 17 11:40:31.797701 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.797658 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:31.798458 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.798255 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6b877f875b-r8bm6"] Sep 17 11:40:31.800462 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.800438 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 17 11:40:31.802103 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.801162 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 17 11:40:31.802103 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.801416 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 17 11:40:31.802103 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.801681 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-hs252ioa72ve\"" Sep 17 11:40:31.802103 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.801875 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-vvwkm\"" Sep 17 11:40:31.802103 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.801910 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 17 11:40:31.802103 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.802064 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 17 11:40:31.870835 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.870746 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8b6dg" event={"ID":"a0d623f4-748b-4d82-9a70-41671b79d346","Type":"ContainerStarted","Data":"6622cc178a897902067ff378db7d95e381e84f282a7e8e4e03ecfd98a4cfc7ec"} Sep 17 11:40:31.870835 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.870814 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8b6dg" event={"ID":"a0d623f4-748b-4d82-9a70-41671b79d346","Type":"ContainerStarted","Data":"be6d273bddc2e27f6fcf856a6f430f4e336bb63f706e1b4c35ece0bd0c1bcefc"} Sep 17 11:40:31.871350 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.871038 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8b6dg" Sep 17 11:40:31.873210 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.873138 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pmps2" event={"ID":"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e","Type":"ContainerStarted","Data":"dc95841c705ba2f48c1f91c89614c2fa90af67bb7f443b19c4d1cf4adcd87c62"} Sep 17 11:40:31.876668 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.876605 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" event={"ID":"10929b4d-9cbb-45fd-92ee-427667b142c6","Type":"ContainerStarted","Data":"ab1216b9987867f7be6aad6d6a0118138d3120a19ee3ff882da8e7e3ffb1c761"} Sep 17 11:40:31.898860 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.898803 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8b6dg" podStartSLOduration=66.224444685 podStartE2EDuration="1m7.898784147s" totalImagesPullingTime="1.674339462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:39:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:40:31.897742576 +0000 UTC m=+101.905189610" watchObservedRunningTime="2026-09-17 11:40:31.898784147 +0000 UTC m=+101.906231178" Sep 17 11:40:31.901903 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.901869 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b3fa9a70-d6af-4872-a2af-49b83baff353-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:31.902029 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.901908 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b3fa9a70-d6af-4872-a2af-49b83baff353-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:31.902029 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.901942 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b3fa9a70-d6af-4872-a2af-49b83baff353-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:31.902029 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.901977 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b3fa9a70-d6af-4872-a2af-49b83baff353-secret-grpc-tls\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:31.902029 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.902005 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zffqg\" (UniqueName: \"kubernetes.io/projected/b3fa9a70-d6af-4872-a2af-49b83baff353-kube-api-access-zffqg\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:31.902256 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.902046 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b3fa9a70-d6af-4872-a2af-49b83baff353-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:31.902256 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.902072 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b3fa9a70-d6af-4872-a2af-49b83baff353-secret-thanos-querier-tls\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:31.902256 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:31.902139 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3fa9a70-d6af-4872-a2af-49b83baff353-metrics-client-ca\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:32.003038 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:32.002994 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b3fa9a70-d6af-4872-a2af-49b83baff353-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:32.003235 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:32.003050 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b3fa9a70-d6af-4872-a2af-49b83baff353-secret-thanos-querier-tls\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:32.003235 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:32.003097 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3fa9a70-d6af-4872-a2af-49b83baff353-metrics-client-ca\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:32.003359 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:32.003236 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b3fa9a70-d6af-4872-a2af-49b83baff353-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:32.003359 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:32.003258 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b3fa9a70-d6af-4872-a2af-49b83baff353-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:32.003359 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:32.003299 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b3fa9a70-d6af-4872-a2af-49b83baff353-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:32.003359 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:32.003332 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b3fa9a70-d6af-4872-a2af-49b83baff353-secret-grpc-tls\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:32.003545 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:32.003361 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zffqg\" (UniqueName: \"kubernetes.io/projected/b3fa9a70-d6af-4872-a2af-49b83baff353-kube-api-access-zffqg\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:32.005690 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:32.005273 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3fa9a70-d6af-4872-a2af-49b83baff353-metrics-client-ca\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:32.008971 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:32.008945 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b3fa9a70-d6af-4872-a2af-49b83baff353-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:32.009180 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:32.009107 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b3fa9a70-d6af-4872-a2af-49b83baff353-secret-grpc-tls\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:32.010169 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:32.009695 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b3fa9a70-d6af-4872-a2af-49b83baff353-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:32.011443 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:32.011398 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b3fa9a70-d6af-4872-a2af-49b83baff353-secret-thanos-querier-tls\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:32.011778 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:32.011732 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b3fa9a70-d6af-4872-a2af-49b83baff353-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:32.011912 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:32.011894 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b3fa9a70-d6af-4872-a2af-49b83baff353-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:32.013256 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:32.013214 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zffqg\" (UniqueName: \"kubernetes.io/projected/b3fa9a70-d6af-4872-a2af-49b83baff353-kube-api-access-zffqg\") pod \"thanos-querier-6b877f875b-r8bm6\" (UID: \"b3fa9a70-d6af-4872-a2af-49b83baff353\") " pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:32.125174 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:32.125080 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:32.741197 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:32.741169 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pk9p9" Sep 17 11:40:32.882731 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:32.882640 2581 generic.go:356] "Generic (PLEG): container finished" podID="ddf4132f-a985-4fa4-96dc-1a2d4e0d810e" containerID="dc95841c705ba2f48c1f91c89614c2fa90af67bb7f443b19c4d1cf4adcd87c62" exitCode=0 Sep 17 11:40:32.883226 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:32.882755 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pmps2" event={"ID":"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e","Type":"ContainerDied","Data":"dc95841c705ba2f48c1f91c89614c2fa90af67bb7f443b19c4d1cf4adcd87c62"} Sep 17 11:40:34.448793 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:34.448759 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-nxcqh"] Sep 17 11:40:34.466766 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:34.466740 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nxcqh" Sep 17 11:40:34.469538 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:34.469517 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-q646h\"" Sep 17 11:40:34.469669 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:34.469569 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 17 11:40:34.472529 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:34.472499 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-nxcqh"] Sep 17 11:40:34.627854 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:34.627823 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3828af14-839e-4afd-908b-ebeb82098f99-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-nxcqh\" (UID: \"3828af14-839e-4afd-908b-ebeb82098f99\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nxcqh" Sep 17 11:40:34.628023 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:34.627872 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4p4f\" (UniqueName: \"kubernetes.io/projected/3828af14-839e-4afd-908b-ebeb82098f99-kube-api-access-g4p4f\") pod \"monitoring-plugin-5d85778b9d-nxcqh\" (UID: \"3828af14-839e-4afd-908b-ebeb82098f99\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nxcqh" Sep 17 11:40:34.730342 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:34.729549 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3828af14-839e-4afd-908b-ebeb82098f99-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-nxcqh\" (UID: \"3828af14-839e-4afd-908b-ebeb82098f99\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nxcqh" Sep 17 11:40:34.730342 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:34.729605 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4p4f\" (UniqueName: \"kubernetes.io/projected/3828af14-839e-4afd-908b-ebeb82098f99-kube-api-access-g4p4f\") pod \"monitoring-plugin-5d85778b9d-nxcqh\" (UID: \"3828af14-839e-4afd-908b-ebeb82098f99\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nxcqh" Sep 17 11:40:34.730342 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:40:34.730027 2581 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 17 11:40:34.730342 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:40:34.730091 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3828af14-839e-4afd-908b-ebeb82098f99-monitoring-plugin-cert podName:3828af14-839e-4afd-908b-ebeb82098f99 nodeName:}" failed. No retries permitted until 2026-09-17 11:40:35.230072553 +0000 UTC m=+105.237519581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/3828af14-839e-4afd-908b-ebeb82098f99-monitoring-plugin-cert") pod "monitoring-plugin-5d85778b9d-nxcqh" (UID: "3828af14-839e-4afd-908b-ebeb82098f99") : secret "monitoring-plugin-cert" not found Sep 17 11:40:34.746753 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:34.746573 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4p4f\" (UniqueName: \"kubernetes.io/projected/3828af14-839e-4afd-908b-ebeb82098f99-kube-api-access-g4p4f\") pod \"monitoring-plugin-5d85778b9d-nxcqh\" (UID: \"3828af14-839e-4afd-908b-ebeb82098f99\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nxcqh" Sep 17 11:40:34.900059 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:34.899926 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pmps2" event={"ID":"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e","Type":"ContainerStarted","Data":"66d7ef5d2665d6765dbcf18ae652c50bffe1b130a6ec23a52f996e43ac6113eb"} Sep 17 11:40:34.905095 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:34.902968 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-659fbbd545-hznqv" event={"ID":"9608f58c-67d7-44df-86f6-2b6fe501d1ba","Type":"ContainerStarted","Data":"883ade1a55dd85cf26e08a3e128ef0f09d03de755d9f7793b1017ceae6074145"} Sep 17 11:40:34.905095 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:34.904225 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-659fbbd545-hznqv" Sep 17 11:40:34.906438 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:34.906421 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-659fbbd545-hznqv" Sep 17 11:40:34.908282 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:34.908257 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" event={"ID":"10929b4d-9cbb-45fd-92ee-427667b142c6","Type":"ContainerStarted","Data":"fa02cb55f25f0dc11b4f5caf16e28f19dd841fd47418cb37fc7a2a432fbc82f1"} Sep 17 11:40:34.934337 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:34.934289 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-659fbbd545-hznqv" podStartSLOduration=1.550600016 podStartE2EDuration="14.934271459s" totalImagesPullingTime="13.383671443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:40:34.932754223 +0000 UTC m=+104.940201278" watchObservedRunningTime="2026-09-17 11:40:34.934271459 +0000 UTC m=+104.941718488" Sep 17 11:40:34.985785 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:34.985760 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6b877f875b-r8bm6"] Sep 17 11:40:34.987790 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:40:34.987758 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3fa9a70_d6af_4872_a2af_49b83baff353.slice/crio-31da48aab4597f2db81d159495b450a0fa98902918e63d8fca6c4cf686255345 WatchSource:0}: Error finding container 31da48aab4597f2db81d159495b450a0fa98902918e63d8fca6c4cf686255345: Status 404 returned error can't find the container with id 31da48aab4597f2db81d159495b450a0fa98902918e63d8fca6c4cf686255345 Sep 17 11:40:35.216142 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.216079 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-687c687b86-k2dpc"] Sep 17 11:40:35.233993 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.233920 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3828af14-839e-4afd-908b-ebeb82098f99-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-nxcqh\" (UID: \"3828af14-839e-4afd-908b-ebeb82098f99\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nxcqh" Sep 17 11:40:35.234138 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.234014 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.236716 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.236692 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3828af14-839e-4afd-908b-ebeb82098f99-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-nxcqh\" (UID: \"3828af14-839e-4afd-908b-ebeb82098f99\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nxcqh" Sep 17 11:40:35.244016 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.243995 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 17 11:40:35.250474 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.250442 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 17 11:40:35.250569 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.250497 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 17 11:40:35.250790 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.250774 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 17 11:40:35.250967 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.250948 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-qxzlp\"" Sep 17 11:40:35.251030 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.250948 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 17 11:40:35.259948 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.259927 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-687c687b86-k2dpc"] Sep 17 11:40:35.262629 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.262607 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 17 11:40:35.334486 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.334443 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/73902f0f-5a1d-4102-b852-7795cae6647a-federate-client-tls\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.334486 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.334489 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73902f0f-5a1d-4102-b852-7795cae6647a-metrics-client-ca\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.334713 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.334506 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73902f0f-5a1d-4102-b852-7795cae6647a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.334713 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.334525 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/73902f0f-5a1d-4102-b852-7795cae6647a-secret-telemeter-client\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.334713 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.334631 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htbd5\" (UniqueName: \"kubernetes.io/projected/73902f0f-5a1d-4102-b852-7795cae6647a-kube-api-access-htbd5\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.334713 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.334670 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73902f0f-5a1d-4102-b852-7795cae6647a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.334713 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.334707 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/73902f0f-5a1d-4102-b852-7795cae6647a-telemeter-client-tls\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.334898 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.334744 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73902f0f-5a1d-4102-b852-7795cae6647a-serving-certs-ca-bundle\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.376275 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.376250 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nxcqh" Sep 17 11:40:35.435801 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.435764 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/73902f0f-5a1d-4102-b852-7795cae6647a-federate-client-tls\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.436086 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.436066 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73902f0f-5a1d-4102-b852-7795cae6647a-metrics-client-ca\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.436332 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.436314 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73902f0f-5a1d-4102-b852-7795cae6647a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.436444 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.436430 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/73902f0f-5a1d-4102-b852-7795cae6647a-secret-telemeter-client\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.436594 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.436578 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htbd5\" (UniqueName: \"kubernetes.io/projected/73902f0f-5a1d-4102-b852-7795cae6647a-kube-api-access-htbd5\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.436721 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.436707 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73902f0f-5a1d-4102-b852-7795cae6647a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.436838 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.436826 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/73902f0f-5a1d-4102-b852-7795cae6647a-telemeter-client-tls\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.436977 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.436961 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73902f0f-5a1d-4102-b852-7795cae6647a-serving-certs-ca-bundle\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.437383 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.437069 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73902f0f-5a1d-4102-b852-7795cae6647a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.437383 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.436881 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73902f0f-5a1d-4102-b852-7795cae6647a-metrics-client-ca\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.437801 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.437776 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73902f0f-5a1d-4102-b852-7795cae6647a-serving-certs-ca-bundle\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.439181 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.439154 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/73902f0f-5a1d-4102-b852-7795cae6647a-federate-client-tls\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.439705 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.439685 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/73902f0f-5a1d-4102-b852-7795cae6647a-telemeter-client-tls\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.440406 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.440382 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73902f0f-5a1d-4102-b852-7795cae6647a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.440516 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.440448 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/73902f0f-5a1d-4102-b852-7795cae6647a-secret-telemeter-client\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.450210 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.450048 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htbd5\" (UniqueName: \"kubernetes.io/projected/73902f0f-5a1d-4102-b852-7795cae6647a-kube-api-access-htbd5\") pod \"telemeter-client-687c687b86-k2dpc\" (UID: \"73902f0f-5a1d-4102-b852-7795cae6647a\") " pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.543770 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.543737 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" Sep 17 11:40:35.700084 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.699866 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-nxcqh"] Sep 17 11:40:35.702488 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:40:35.702458 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3828af14_839e_4afd_908b_ebeb82098f99.slice/crio-06336626de4c21db8859faeeb3e47e854b17aad0cde061d5e3679c4bddaece14 WatchSource:0}: Error finding container 06336626de4c21db8859faeeb3e47e854b17aad0cde061d5e3679c4bddaece14: Status 404 returned error can't find the container with id 06336626de4c21db8859faeeb3e47e854b17aad0cde061d5e3679c4bddaece14 Sep 17 11:40:35.892546 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.892467 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-687c687b86-k2dpc"] Sep 17 11:40:35.896703 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:40:35.896670 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73902f0f_5a1d_4102_b852_7795cae6647a.slice/crio-2d6de90aac23b7d1a4e0dfcc80152069a08ec638d1637c0dd4fe064680536c06 WatchSource:0}: Error finding container 2d6de90aac23b7d1a4e0dfcc80152069a08ec638d1637c0dd4fe064680536c06: Status 404 returned error can't find the container with id 2d6de90aac23b7d1a4e0dfcc80152069a08ec638d1637c0dd4fe064680536c06 Sep 17 11:40:35.916383 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.916349 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pmps2" event={"ID":"ddf4132f-a985-4fa4-96dc-1a2d4e0d810e","Type":"ContainerStarted","Data":"09e9ecc1dc1c4dc2c3f5b49a93b2457dd7eac1fb1b6858aac0a6683d09e779ec"} Sep 17 11:40:35.917874 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.917823 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nxcqh" event={"ID":"3828af14-839e-4afd-908b-ebeb82098f99","Type":"ContainerStarted","Data":"06336626de4c21db8859faeeb3e47e854b17aad0cde061d5e3679c4bddaece14"} Sep 17 11:40:35.919309 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.919286 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" event={"ID":"b3fa9a70-d6af-4872-a2af-49b83baff353","Type":"ContainerStarted","Data":"31da48aab4597f2db81d159495b450a0fa98902918e63d8fca6c4cf686255345"} Sep 17 11:40:35.922231 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.922208 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" event={"ID":"10929b4d-9cbb-45fd-92ee-427667b142c6","Type":"ContainerStarted","Data":"e4c32866236b7181e066a5c8cd474ee0b11bb3f69eeb54aa8fd18f5096208227"} Sep 17 11:40:35.922321 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.922260 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" event={"ID":"10929b4d-9cbb-45fd-92ee-427667b142c6","Type":"ContainerStarted","Data":"01716fd49f3a1c347fce400e2b1dad025182b0e992a1978fa3c97c4655333e70"} Sep 17 11:40:35.924167 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.924096 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" event={"ID":"73902f0f-5a1d-4102-b852-7795cae6647a","Type":"ContainerStarted","Data":"2d6de90aac23b7d1a4e0dfcc80152069a08ec638d1637c0dd4fe064680536c06"} Sep 17 11:40:35.959283 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.959232 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-pmps2" podStartSLOduration=5.482491773 podStartE2EDuration="6.95921133s" totalImagesPullingTime="1.384301371s" totalInitContainerRuntime="92.418186ms" isStatefulPod=true podCreationTimestamp="2026-09-17 11:40:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:40:35.954630183 +0000 UTC m=+105.962077212" watchObservedRunningTime="2026-09-17 11:40:35.95921133 +0000 UTC m=+105.966658361" Sep 17 11:40:35.994389 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:35.994324 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2ptb8" podStartSLOduration=3.394080905 podStartE2EDuration="6.994305261s" totalImagesPullingTime="3.600224356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:40:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:40:35.992333975 +0000 UTC m=+105.999781005" watchObservedRunningTime="2026-09-17 11:40:35.994305261 +0000 UTC m=+106.001752291" Sep 17 11:40:38.937703 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:38.937663 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" event={"ID":"73902f0f-5a1d-4102-b852-7795cae6647a","Type":"ContainerStarted","Data":"4df185ae5605f0e410b7963799284bc5a64aa9346a76fb69f22475215208c1a0"} Sep 17 11:40:38.939181 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:38.939157 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nxcqh" event={"ID":"3828af14-839e-4afd-908b-ebeb82098f99","Type":"ContainerStarted","Data":"1f86644ac65afed5498fe183e27988bd5e6b78865db05b0ff0a62a06c23eee08"} Sep 17 11:40:38.939348 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:38.939329 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nxcqh" Sep 17 11:40:38.941694 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:38.941670 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" event={"ID":"b3fa9a70-d6af-4872-a2af-49b83baff353","Type":"ContainerStarted","Data":"4db3ac0fc355da5c65c5e29fe959b23c49afd5fa33901308ecc336d011f5e791"} Sep 17 11:40:38.941769 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:38.941724 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" event={"ID":"b3fa9a70-d6af-4872-a2af-49b83baff353","Type":"ContainerStarted","Data":"1a32b58aa4d0336641a63c3d18b84591d6d6ef3ebefdffb64198a6ec3ff48e23"} Sep 17 11:40:38.941769 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:38.941753 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" event={"ID":"b3fa9a70-d6af-4872-a2af-49b83baff353","Type":"ContainerStarted","Data":"f4647e0c31f8541d6a9447c5c2bacf6a25278f8e5d97a2a92494351243e60fee"} Sep 17 11:40:38.944359 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:38.944342 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nxcqh" Sep 17 11:40:38.987175 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:38.987111 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nxcqh" podStartSLOduration=2.302438354 podStartE2EDuration="4.987096447s" totalImagesPullingTime="2.684658093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:40:38.965172313 +0000 UTC m=+108.972619343" watchObservedRunningTime="2026-09-17 11:40:38.987096447 +0000 UTC m=+108.994543522" Sep 17 11:40:39.949462 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:39.949366 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" event={"ID":"73902f0f-5a1d-4102-b852-7795cae6647a","Type":"ContainerStarted","Data":"a71475711a0b66d1852e0f93834194eef88bf7f1dbd5244b439f09ffb4d95e2c"} Sep 17 11:40:39.949876 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:39.949484 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" event={"ID":"73902f0f-5a1d-4102-b852-7795cae6647a","Type":"ContainerStarted","Data":"3c85e3598ace2eb4099d00144fd42ff49e38aed1afc3f518ed5e8752f7495d57"} Sep 17 11:40:39.978249 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:39.978200 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-687c687b86-k2dpc" podStartSLOduration=1.3092075570000001 podStartE2EDuration="4.978179695s" totalImagesPullingTime="3.668972138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:40:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:40:39.976871286 +0000 UTC m=+109.984318315" watchObservedRunningTime="2026-09-17 11:40:39.978179695 +0000 UTC m=+109.985626728" Sep 17 11:40:40.956276 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:40.956230 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" event={"ID":"b3fa9a70-d6af-4872-a2af-49b83baff353","Type":"ContainerStarted","Data":"ba3c8f7f47bd120b6e9c530bdd00e96c8ce1cb4264d55095a0d038f0b805d098"} Sep 17 11:40:40.956699 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:40.956320 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" event={"ID":"b3fa9a70-d6af-4872-a2af-49b83baff353","Type":"ContainerStarted","Data":"3c0c6a017f070ae0cd6bfcb21fe77250bb7e4c2c5f585e5e2b58a73031e43212"} Sep 17 11:40:40.956699 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:40.956364 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" event={"ID":"b3fa9a70-d6af-4872-a2af-49b83baff353","Type":"ContainerStarted","Data":"5d44a5a077a7f325b2f3f5885401172103c2e90db9e9569c870a84e6f9659ef5"} Sep 17 11:40:40.956785 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:40.956740 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:40.981677 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:40.981537 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" podStartSLOduration=5.159098063 podStartE2EDuration="9.981518767s" totalImagesPullingTime="4.822420704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:40:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:40:40.979891732 +0000 UTC m=+110.987338761" watchObservedRunningTime="2026-09-17 11:40:40.981518767 +0000 UTC m=+110.988965798" Sep 17 11:40:41.885672 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:41.885636 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8b6dg" Sep 17 11:40:44.423969 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:44.423936 2581 scope.go:117] "RemoveContainer" containerID="6600ed4c783ab9f65d3e517b89fd03b8f7694d5e46a6301b4e7831f526d02ead" Sep 17 11:40:44.973961 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:44.973931 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/2.log" Sep 17 11:40:44.974172 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:44.973983 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" event={"ID":"124b0387-2e6e-4f73-b71a-2ead313e3701","Type":"ContainerStarted","Data":"e7c5f27c388d18f09432ad8506893de5f13ac42f9a239c0023a06738369f3c01"} Sep 17 11:40:44.974317 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:44.974295 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" Sep 17 11:40:44.979819 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:44.979792 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" Sep 17 11:40:45.016993 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:45.016935 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-gn9sh" podStartSLOduration=53.579961239 podStartE2EDuration="56.016917331s" totalImagesPullingTime="2.436956092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:39:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:40:45.015570102 +0000 UTC m=+115.023017132" watchObservedRunningTime="2026-09-17 11:40:45.016917331 +0000 UTC m=+115.024364360" Sep 17 11:40:45.163216 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:45.163183 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-jbvnp"] Sep 17 11:40:45.166484 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:45.166460 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-jbvnp" Sep 17 11:40:45.170078 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:45.170053 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 17 11:40:45.170229 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:45.170106 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-pwk5f\"" Sep 17 11:40:45.170444 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:45.170430 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 17 11:40:45.180639 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:45.180614 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-jbvnp"] Sep 17 11:40:45.329212 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:45.329177 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcxb5\" (UniqueName: \"kubernetes.io/projected/9ef7feb3-5cc2-4447-9581-29156afc6ca7-kube-api-access-gcxb5\") pod \"downloads-76f99584f4-jbvnp\" (UID: \"9ef7feb3-5cc2-4447-9581-29156afc6ca7\") " pod="openshift-console/downloads-76f99584f4-jbvnp" Sep 17 11:40:45.430446 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:45.430418 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcxb5\" (UniqueName: \"kubernetes.io/projected/9ef7feb3-5cc2-4447-9581-29156afc6ca7-kube-api-access-gcxb5\") pod \"downloads-76f99584f4-jbvnp\" (UID: \"9ef7feb3-5cc2-4447-9581-29156afc6ca7\") " pod="openshift-console/downloads-76f99584f4-jbvnp" Sep 17 11:40:45.439638 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:45.439602 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcxb5\" (UniqueName: \"kubernetes.io/projected/9ef7feb3-5cc2-4447-9581-29156afc6ca7-kube-api-access-gcxb5\") pod \"downloads-76f99584f4-jbvnp\" (UID: \"9ef7feb3-5cc2-4447-9581-29156afc6ca7\") " pod="openshift-console/downloads-76f99584f4-jbvnp" Sep 17 11:40:45.476681 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:45.476651 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-jbvnp" Sep 17 11:40:45.753617 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:45.753589 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-jbvnp"] Sep 17 11:40:45.756658 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:40:45.756632 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ef7feb3_5cc2_4447_9581_29156afc6ca7.slice/crio-99d16538b1c9d43fe0cb6c41116cbcb650828ebf6665d5db849f17489391f8c1 WatchSource:0}: Error finding container 99d16538b1c9d43fe0cb6c41116cbcb650828ebf6665d5db849f17489391f8c1: Status 404 returned error can't find the container with id 99d16538b1c9d43fe0cb6c41116cbcb650828ebf6665d5db849f17489391f8c1 Sep 17 11:40:45.979604 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:45.979568 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-jbvnp" event={"ID":"9ef7feb3-5cc2-4447-9581-29156afc6ca7","Type":"ContainerStarted","Data":"99d16538b1c9d43fe0cb6c41116cbcb650828ebf6665d5db849f17489391f8c1"} Sep 17 11:40:46.969590 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:46.969561 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6b877f875b-r8bm6" Sep 17 11:40:56.088165 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.088110 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b7bd5fc8d-twfzd"] Sep 17 11:40:56.092624 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.092598 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:40:56.098622 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.098185 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-c9t95\"" Sep 17 11:40:56.098622 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.098239 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 17 11:40:56.098622 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.098391 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 17 11:40:56.098622 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.098450 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 17 11:40:56.098622 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.098598 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 17 11:40:56.098977 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.098598 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 17 11:40:56.109226 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.109200 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b7bd5fc8d-twfzd"] Sep 17 11:40:56.129022 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.128996 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b68da1c-44fc-4055-b5cc-ca6755ad1780-console-config\") pod \"console-6b7bd5fc8d-twfzd\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:40:56.129194 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.129026 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt2fs\" (UniqueName: \"kubernetes.io/projected/5b68da1c-44fc-4055-b5cc-ca6755ad1780-kube-api-access-bt2fs\") pod \"console-6b7bd5fc8d-twfzd\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:40:56.129194 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.129160 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b68da1c-44fc-4055-b5cc-ca6755ad1780-console-serving-cert\") pod \"console-6b7bd5fc8d-twfzd\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:40:56.129194 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.129189 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b68da1c-44fc-4055-b5cc-ca6755ad1780-service-ca\") pod \"console-6b7bd5fc8d-twfzd\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:40:56.129349 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.129206 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b68da1c-44fc-4055-b5cc-ca6755ad1780-console-oauth-config\") pod \"console-6b7bd5fc8d-twfzd\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:40:56.129349 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.129231 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b68da1c-44fc-4055-b5cc-ca6755ad1780-oauth-serving-cert\") pod \"console-6b7bd5fc8d-twfzd\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:40:56.230244 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.230199 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b68da1c-44fc-4055-b5cc-ca6755ad1780-console-serving-cert\") pod \"console-6b7bd5fc8d-twfzd\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:40:56.230437 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.230257 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b68da1c-44fc-4055-b5cc-ca6755ad1780-service-ca\") pod \"console-6b7bd5fc8d-twfzd\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:40:56.230437 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.230283 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b68da1c-44fc-4055-b5cc-ca6755ad1780-console-oauth-config\") pod \"console-6b7bd5fc8d-twfzd\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:40:56.230437 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.230299 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b68da1c-44fc-4055-b5cc-ca6755ad1780-oauth-serving-cert\") pod \"console-6b7bd5fc8d-twfzd\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:40:56.230437 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.230331 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b68da1c-44fc-4055-b5cc-ca6755ad1780-console-config\") pod \"console-6b7bd5fc8d-twfzd\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:40:56.230437 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.230354 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt2fs\" (UniqueName: \"kubernetes.io/projected/5b68da1c-44fc-4055-b5cc-ca6755ad1780-kube-api-access-bt2fs\") pod \"console-6b7bd5fc8d-twfzd\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:40:56.231405 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.231355 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b68da1c-44fc-4055-b5cc-ca6755ad1780-console-config\") pod \"console-6b7bd5fc8d-twfzd\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:40:56.231523 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.231434 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b68da1c-44fc-4055-b5cc-ca6755ad1780-oauth-serving-cert\") pod \"console-6b7bd5fc8d-twfzd\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:40:56.231651 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.231623 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b68da1c-44fc-4055-b5cc-ca6755ad1780-service-ca\") pod \"console-6b7bd5fc8d-twfzd\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:40:56.233289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.233267 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b68da1c-44fc-4055-b5cc-ca6755ad1780-console-serving-cert\") pod \"console-6b7bd5fc8d-twfzd\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:40:56.233378 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.233302 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b68da1c-44fc-4055-b5cc-ca6755ad1780-console-oauth-config\") pod \"console-6b7bd5fc8d-twfzd\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:40:56.240927 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.240907 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt2fs\" (UniqueName: \"kubernetes.io/projected/5b68da1c-44fc-4055-b5cc-ca6755ad1780-kube-api-access-bt2fs\") pod \"console-6b7bd5fc8d-twfzd\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:40:56.404400 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:40:56.404297 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:41:00.167206 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:00.167170 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs\") pod \"network-metrics-daemon-f5svv\" (UID: \"757a11fc-3750-4ef9-94f6-6c6da0d5c95c\") " pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:41:00.169882 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:00.169859 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757a11fc-3750-4ef9-94f6-6c6da0d5c95c-metrics-certs\") pod \"network-metrics-daemon-f5svv\" (UID: \"757a11fc-3750-4ef9-94f6-6c6da0d5c95c\") " pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:41:00.236484 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:00.236455 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l8fq5\"" Sep 17 11:41:00.244665 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:00.244639 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5svv" Sep 17 11:41:02.596743 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:02.596632 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f5svv"] Sep 17 11:41:02.600984 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:02.600920 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b7bd5fc8d-twfzd"] Sep 17 11:41:02.603762 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:41:02.603734 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b68da1c_44fc_4055_b5cc_ca6755ad1780.slice/crio-277bff86249e128997266e253650fe7e6353582b6f43f09b6594e0d2c4ac7ae2 WatchSource:0}: Error finding container 277bff86249e128997266e253650fe7e6353582b6f43f09b6594e0d2c4ac7ae2: Status 404 returned error can't find the container with id 277bff86249e128997266e253650fe7e6353582b6f43f09b6594e0d2c4ac7ae2 Sep 17 11:41:03.063051 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:03.063004 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f5svv" event={"ID":"757a11fc-3750-4ef9-94f6-6c6da0d5c95c","Type":"ContainerStarted","Data":"9e63213f33d614f99843ed2432da8aee7687fdbfc1cc454f14142970c6223572"} Sep 17 11:41:03.065093 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:03.065056 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-jbvnp" event={"ID":"9ef7feb3-5cc2-4447-9581-29156afc6ca7","Type":"ContainerStarted","Data":"7cf8d7c6ef75173489c1cd703f1444bcc4cf715bce83ac295351c2ee269492c1"} Sep 17 11:41:03.065455 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:03.065432 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-jbvnp" Sep 17 11:41:03.066737 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:03.066712 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7bd5fc8d-twfzd" event={"ID":"5b68da1c-44fc-4055-b5cc-ca6755ad1780","Type":"ContainerStarted","Data":"277bff86249e128997266e253650fe7e6353582b6f43f09b6594e0d2c4ac7ae2"} Sep 17 11:41:03.084840 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:03.084789 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-jbvnp" podStartSLOduration=1.402539775 podStartE2EDuration="18.084769045s" totalImagesPullingTime="16.68222927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:40:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:41:03.084077064 +0000 UTC m=+133.091524093" watchObservedRunningTime="2026-09-17 11:41:03.084769045 +0000 UTC m=+133.092216077" Sep 17 11:41:03.086029 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:03.086003 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-jbvnp" Sep 17 11:41:05.091594 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.091545 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f5svv" event={"ID":"757a11fc-3750-4ef9-94f6-6c6da0d5c95c","Type":"ContainerStarted","Data":"59f84f6645a5154fa7d2bd44c49907e75ca69711efcbc6e503aa1359f9eeeed6"} Sep 17 11:41:05.463736 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.463647 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c9f96bcb8-psrgk"] Sep 17 11:41:05.487163 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.487103 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c9f96bcb8-psrgk"] Sep 17 11:41:05.487388 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.487242 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.496593 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.496570 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 17 11:41:05.623539 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.623503 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-console-oauth-config\") pod \"console-6c9f96bcb8-psrgk\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.623810 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.623570 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-console-serving-cert\") pod \"console-6c9f96bcb8-psrgk\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.623810 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.623600 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-console-config\") pod \"console-6c9f96bcb8-psrgk\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.623810 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.623646 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-trusted-ca-bundle\") pod \"console-6c9f96bcb8-psrgk\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.623810 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.623688 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-service-ca\") pod \"console-6c9f96bcb8-psrgk\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.623810 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.623725 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-oauth-serving-cert\") pod \"console-6c9f96bcb8-psrgk\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.623810 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.623758 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf4sn\" (UniqueName: \"kubernetes.io/projected/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-kube-api-access-gf4sn\") pod \"console-6c9f96bcb8-psrgk\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.725147 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.725046 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-console-oauth-config\") pod \"console-6c9f96bcb8-psrgk\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.726378 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.725376 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-console-serving-cert\") pod \"console-6c9f96bcb8-psrgk\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.726378 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.725444 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-console-config\") pod \"console-6c9f96bcb8-psrgk\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.726378 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.725596 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-trusted-ca-bundle\") pod \"console-6c9f96bcb8-psrgk\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.726378 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.725651 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-service-ca\") pod \"console-6c9f96bcb8-psrgk\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.726378 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.725690 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-oauth-serving-cert\") pod \"console-6c9f96bcb8-psrgk\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.726930 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.726504 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf4sn\" (UniqueName: \"kubernetes.io/projected/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-kube-api-access-gf4sn\") pod \"console-6c9f96bcb8-psrgk\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.727176 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.727147 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-oauth-serving-cert\") pod \"console-6c9f96bcb8-psrgk\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.727555 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.727532 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-service-ca\") pod \"console-6c9f96bcb8-psrgk\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.727985 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.727630 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-console-config\") pod \"console-6c9f96bcb8-psrgk\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.727985 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.727769 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-trusted-ca-bundle\") pod \"console-6c9f96bcb8-psrgk\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.735434 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.735391 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf4sn\" (UniqueName: \"kubernetes.io/projected/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-kube-api-access-gf4sn\") pod \"console-6c9f96bcb8-psrgk\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.744662 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.744615 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-console-serving-cert\") pod \"console-6c9f96bcb8-psrgk\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.747275 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.747252 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-console-oauth-config\") pod \"console-6c9f96bcb8-psrgk\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:05.803081 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:05.802853 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:06.430650 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:06.430570 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c9f96bcb8-psrgk"] Sep 17 11:41:06.623982 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:41:06.623943 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6946d2fa_bd8c_4bd4_a77e_9d2a252c91b4.slice/crio-c99721295b0b5eb1ffc5ffa922ce64788cf5cf93772b156ba35bcb86fb56d706 WatchSource:0}: Error finding container c99721295b0b5eb1ffc5ffa922ce64788cf5cf93772b156ba35bcb86fb56d706: Status 404 returned error can't find the container with id c99721295b0b5eb1ffc5ffa922ce64788cf5cf93772b156ba35bcb86fb56d706 Sep 17 11:41:07.104395 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:07.104339 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f5svv" event={"ID":"757a11fc-3750-4ef9-94f6-6c6da0d5c95c","Type":"ContainerStarted","Data":"244af05d4644e2814ed39a9ca2c8f65e5d4faa94f748abfa7e94d627db8717c7"} Sep 17 11:41:07.106740 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:07.106698 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7bd5fc8d-twfzd" event={"ID":"5b68da1c-44fc-4055-b5cc-ca6755ad1780","Type":"ContainerStarted","Data":"af2d8eb2db5f67948cb6ea952b38debdea1a36a8234dc5b265590e6495bd1890"} Sep 17 11:41:07.108847 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:07.108820 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c9f96bcb8-psrgk" event={"ID":"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4","Type":"ContainerStarted","Data":"55621d08c5b51d56539c22eb9a03c8f73dce7c2eeae6d1698325fbd499db5392"} Sep 17 11:41:07.108964 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:07.108860 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c9f96bcb8-psrgk" event={"ID":"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4","Type":"ContainerStarted","Data":"c99721295b0b5eb1ffc5ffa922ce64788cf5cf93772b156ba35bcb86fb56d706"} Sep 17 11:41:07.124533 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:07.124488 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-f5svv" podStartSLOduration=135.089884025 podStartE2EDuration="2m17.124473568s" totalImagesPullingTime="2.034589543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:41:07.122873263 +0000 UTC m=+137.130320294" watchObservedRunningTime="2026-09-17 11:41:07.124473568 +0000 UTC m=+137.131920620" Sep 17 11:41:07.143259 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:07.143201 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b7bd5fc8d-twfzd" podStartSLOduration=7.08692045 podStartE2EDuration="11.143186482s" totalImagesPullingTime="4.056266032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:41:07.142210836 +0000 UTC m=+137.149657879" watchObservedRunningTime="2026-09-17 11:41:07.143186482 +0000 UTC m=+137.150633512" Sep 17 11:41:14.142271 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:14.142237 2581 generic.go:356] "Generic (PLEG): container finished" podID="344d4ef6-f336-46de-a0bd-0735aba220ec" containerID="8b652ec362be1e89c3ce36c394c5939933256b697a3c5cb6fa0f39c9e09ba30d" exitCode=0 Sep 17 11:41:14.142684 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:14.142305 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-skhl2" event={"ID":"344d4ef6-f336-46de-a0bd-0735aba220ec","Type":"ContainerDied","Data":"8b652ec362be1e89c3ce36c394c5939933256b697a3c5cb6fa0f39c9e09ba30d"} Sep 17 11:41:14.142684 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:14.142652 2581 scope.go:117] "RemoveContainer" containerID="8b652ec362be1e89c3ce36c394c5939933256b697a3c5cb6fa0f39c9e09ba30d" Sep 17 11:41:14.165608 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:14.165565 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c9f96bcb8-psrgk" podStartSLOduration=9.165550498 podStartE2EDuration="9.165550498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:41:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:41:07.167835792 +0000 UTC m=+137.175282823" watchObservedRunningTime="2026-09-17 11:41:14.165550498 +0000 UTC m=+144.172997528" Sep 17 11:41:15.148400 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:15.148362 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-skhl2" event={"ID":"344d4ef6-f336-46de-a0bd-0735aba220ec","Type":"ContainerStarted","Data":"fed9f2d238d780aa4106ca38374c527e1edce55b278526e727b7f149afe68545"} Sep 17 11:41:15.803997 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:15.803962 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:15.804199 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:15.804026 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:15.809039 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:15.809016 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:16.156513 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:16.156437 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:41:16.211097 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:16.211065 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b7bd5fc8d-twfzd"] Sep 17 11:41:16.404850 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:16.404816 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:41:16.487276 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:16.487189 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-2ptb8_10929b4d-9cbb-45fd-92ee-427667b142c6/kube-state-metrics/0.log" Sep 17 11:41:16.690312 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:16.690279 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-2ptb8_10929b4d-9cbb-45fd-92ee-427667b142c6/kube-rbac-proxy-main/0.log" Sep 17 11:41:16.886170 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:16.886136 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-2ptb8_10929b4d-9cbb-45fd-92ee-427667b142c6/kube-rbac-proxy-self/0.log" Sep 17 11:41:17.287480 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:17.287455 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-nxcqh_3828af14-839e-4afd-908b-ebeb82098f99/monitoring-plugin/0.log" Sep 17 11:41:18.686668 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:18.686640 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pmps2_ddf4132f-a985-4fa4-96dc-1a2d4e0d810e/init-textfile/0.log" Sep 17 11:41:18.886629 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:18.886599 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pmps2_ddf4132f-a985-4fa4-96dc-1a2d4e0d810e/node-exporter/0.log" Sep 17 11:41:19.085523 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:19.085490 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pmps2_ddf4132f-a985-4fa4-96dc-1a2d4e0d810e/kube-rbac-proxy/0.log" Sep 17 11:41:19.166489 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:19.166464 2581 generic.go:356] "Generic (PLEG): container finished" podID="facf9640-adc2-4f43-a3c0-5d0b0521ba7f" containerID="de000a3426f4402076ca612685c48d6d3189224ae2eae9d92a5d30a77dd908b8" exitCode=0 Sep 17 11:41:19.166638 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:19.166545 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qz2jt" event={"ID":"facf9640-adc2-4f43-a3c0-5d0b0521ba7f","Type":"ContainerDied","Data":"de000a3426f4402076ca612685c48d6d3189224ae2eae9d92a5d30a77dd908b8"} Sep 17 11:41:19.166890 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:19.166873 2581 scope.go:117] "RemoveContainer" containerID="de000a3426f4402076ca612685c48d6d3189224ae2eae9d92a5d30a77dd908b8" Sep 17 11:41:20.173357 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:20.173325 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qz2jt" event={"ID":"facf9640-adc2-4f43-a3c0-5d0b0521ba7f","Type":"ContainerStarted","Data":"800f451fc0c75ecd0e1a99f1b5f9de9383b66dba5425845beb3fc99cb438275d"} Sep 17 11:41:21.885544 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:21.885500 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-687c687b86-k2dpc_73902f0f-5a1d-4102-b852-7795cae6647a/telemeter-client/0.log" Sep 17 11:41:22.085437 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:22.085398 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-687c687b86-k2dpc_73902f0f-5a1d-4102-b852-7795cae6647a/reload/0.log" Sep 17 11:41:22.286289 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:22.286262 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-687c687b86-k2dpc_73902f0f-5a1d-4102-b852-7795cae6647a/kube-rbac-proxy/0.log" Sep 17 11:41:22.485792 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:22.485764 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b877f875b-r8bm6_b3fa9a70-d6af-4872-a2af-49b83baff353/thanos-query/0.log" Sep 17 11:41:22.697150 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:22.697054 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b877f875b-r8bm6_b3fa9a70-d6af-4872-a2af-49b83baff353/kube-rbac-proxy-web/0.log" Sep 17 11:41:22.886190 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:22.886158 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b877f875b-r8bm6_b3fa9a70-d6af-4872-a2af-49b83baff353/kube-rbac-proxy/0.log" Sep 17 11:41:23.086784 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:23.086757 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b877f875b-r8bm6_b3fa9a70-d6af-4872-a2af-49b83baff353/prom-label-proxy/0.log" Sep 17 11:41:23.286350 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:23.286324 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b877f875b-r8bm6_b3fa9a70-d6af-4872-a2af-49b83baff353/kube-rbac-proxy-rules/0.log" Sep 17 11:41:23.487067 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:23.486975 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b877f875b-r8bm6_b3fa9a70-d6af-4872-a2af-49b83baff353/kube-rbac-proxy-metrics/0.log" Sep 17 11:41:23.886878 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:23.886854 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/2.log" Sep 17 11:41:24.099539 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:24.099507 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/3.log" Sep 17 11:41:24.288710 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:24.288671 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6b7bd5fc8d-twfzd_5b68da1c-44fc-4055-b5cc-ca6755ad1780/console/0.log" Sep 17 11:41:24.486758 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:24.486718 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6c9f96bcb8-psrgk_6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4/console/0.log" Sep 17 11:41:24.689020 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:24.688935 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-jbvnp_9ef7feb3-5cc2-4447-9581-29156afc6ca7/download-server/0.log" Sep 17 11:41:41.235140 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:41.235041 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b7bd5fc8d-twfzd" podUID="5b68da1c-44fc-4055-b5cc-ca6755ad1780" containerName="console" containerID="cri-o://af2d8eb2db5f67948cb6ea952b38debdea1a36a8234dc5b265590e6495bd1890" gracePeriod=15 Sep 17 11:41:41.503553 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:41.503529 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:41:41.566651 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:41.566625 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b68da1c-44fc-4055-b5cc-ca6755ad1780-oauth-serving-cert\") pod \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " Sep 17 11:41:41.566816 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:41.566674 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b68da1c-44fc-4055-b5cc-ca6755ad1780-console-config\") pod \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " Sep 17 11:41:41.566816 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:41.566736 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bt2fs\" (UniqueName: \"kubernetes.io/projected/5b68da1c-44fc-4055-b5cc-ca6755ad1780-kube-api-access-bt2fs\") pod \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " Sep 17 11:41:41.566816 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:41.566774 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b68da1c-44fc-4055-b5cc-ca6755ad1780-console-oauth-config\") pod \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " Sep 17 11:41:41.566991 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:41.566896 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b68da1c-44fc-4055-b5cc-ca6755ad1780-console-serving-cert\") pod \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " Sep 17 11:41:41.566991 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:41.566970 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b68da1c-44fc-4055-b5cc-ca6755ad1780-service-ca\") pod \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\" (UID: \"5b68da1c-44fc-4055-b5cc-ca6755ad1780\") " Sep 17 11:41:41.567097 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:41.567033 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b68da1c-44fc-4055-b5cc-ca6755ad1780-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5b68da1c-44fc-4055-b5cc-ca6755ad1780" (UID: "5b68da1c-44fc-4055-b5cc-ca6755ad1780"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 11:41:41.567190 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:41.567152 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b68da1c-44fc-4055-b5cc-ca6755ad1780-console-config" (OuterVolumeSpecName: "console-config") pod "5b68da1c-44fc-4055-b5cc-ca6755ad1780" (UID: "5b68da1c-44fc-4055-b5cc-ca6755ad1780"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 11:41:41.567344 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:41.567328 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b68da1c-44fc-4055-b5cc-ca6755ad1780-console-config\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:41:41.567391 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:41.567346 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b68da1c-44fc-4055-b5cc-ca6755ad1780-oauth-serving-cert\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:41:41.567391 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:41.567352 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b68da1c-44fc-4055-b5cc-ca6755ad1780-service-ca" (OuterVolumeSpecName: "service-ca") pod "5b68da1c-44fc-4055-b5cc-ca6755ad1780" (UID: "5b68da1c-44fc-4055-b5cc-ca6755ad1780"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 11:41:41.569362 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:41.569341 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b68da1c-44fc-4055-b5cc-ca6755ad1780-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5b68da1c-44fc-4055-b5cc-ca6755ad1780" (UID: "5b68da1c-44fc-4055-b5cc-ca6755ad1780"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 11:41:41.569608 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:41.569580 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b68da1c-44fc-4055-b5cc-ca6755ad1780-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5b68da1c-44fc-4055-b5cc-ca6755ad1780" (UID: "5b68da1c-44fc-4055-b5cc-ca6755ad1780"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 11:41:41.569608 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:41.569588 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b68da1c-44fc-4055-b5cc-ca6755ad1780-kube-api-access-bt2fs" (OuterVolumeSpecName: "kube-api-access-bt2fs") pod "5b68da1c-44fc-4055-b5cc-ca6755ad1780" (UID: "5b68da1c-44fc-4055-b5cc-ca6755ad1780"). InnerVolumeSpecName "kube-api-access-bt2fs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 11:41:41.667929 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:41.667852 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bt2fs\" (UniqueName: \"kubernetes.io/projected/5b68da1c-44fc-4055-b5cc-ca6755ad1780-kube-api-access-bt2fs\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:41:41.667929 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:41.667879 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b68da1c-44fc-4055-b5cc-ca6755ad1780-console-oauth-config\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:41:41.667929 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:41.667888 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b68da1c-44fc-4055-b5cc-ca6755ad1780-console-serving-cert\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:41:41.667929 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:41.667897 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b68da1c-44fc-4055-b5cc-ca6755ad1780-service-ca\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:41:42.263707 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:42.263671 2581 generic.go:356] "Generic (PLEG): container finished" podID="5b68da1c-44fc-4055-b5cc-ca6755ad1780" containerID="af2d8eb2db5f67948cb6ea952b38debdea1a36a8234dc5b265590e6495bd1890" exitCode=0 Sep 17 11:41:42.264159 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:42.263729 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7bd5fc8d-twfzd" Sep 17 11:41:42.264159 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:42.263760 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7bd5fc8d-twfzd" event={"ID":"5b68da1c-44fc-4055-b5cc-ca6755ad1780","Type":"ContainerDied","Data":"af2d8eb2db5f67948cb6ea952b38debdea1a36a8234dc5b265590e6495bd1890"} Sep 17 11:41:42.264159 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:42.263814 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7bd5fc8d-twfzd" event={"ID":"5b68da1c-44fc-4055-b5cc-ca6755ad1780","Type":"ContainerDied","Data":"277bff86249e128997266e253650fe7e6353582b6f43f09b6594e0d2c4ac7ae2"} Sep 17 11:41:42.264159 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:42.263838 2581 scope.go:117] "RemoveContainer" containerID="af2d8eb2db5f67948cb6ea952b38debdea1a36a8234dc5b265590e6495bd1890" Sep 17 11:41:42.272763 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:42.272745 2581 scope.go:117] "RemoveContainer" containerID="af2d8eb2db5f67948cb6ea952b38debdea1a36a8234dc5b265590e6495bd1890" Sep 17 11:41:42.273038 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:41:42.273021 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af2d8eb2db5f67948cb6ea952b38debdea1a36a8234dc5b265590e6495bd1890\": container with ID starting with af2d8eb2db5f67948cb6ea952b38debdea1a36a8234dc5b265590e6495bd1890 not found: ID does not exist" containerID="af2d8eb2db5f67948cb6ea952b38debdea1a36a8234dc5b265590e6495bd1890" Sep 17 11:41:42.273087 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:42.273047 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af2d8eb2db5f67948cb6ea952b38debdea1a36a8234dc5b265590e6495bd1890"} err="failed to get container status \"af2d8eb2db5f67948cb6ea952b38debdea1a36a8234dc5b265590e6495bd1890\": rpc error: code = NotFound desc = could not find container \"af2d8eb2db5f67948cb6ea952b38debdea1a36a8234dc5b265590e6495bd1890\": container with ID starting with af2d8eb2db5f67948cb6ea952b38debdea1a36a8234dc5b265590e6495bd1890 not found: ID does not exist" Sep 17 11:41:42.284700 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:42.284678 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b7bd5fc8d-twfzd"] Sep 17 11:41:42.287459 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:42.287434 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b7bd5fc8d-twfzd"] Sep 17 11:41:42.428026 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:42.427993 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b68da1c-44fc-4055-b5cc-ca6755ad1780" path="/var/lib/kubelet/pods/5b68da1c-44fc-4055-b5cc-ca6755ad1780/volumes" Sep 17 11:41:48.954101 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:48.954070 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64b6cf8966-lw8rv"] Sep 17 11:41:48.954589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:48.954441 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b68da1c-44fc-4055-b5cc-ca6755ad1780" containerName="console" Sep 17 11:41:48.954589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:48.954454 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b68da1c-44fc-4055-b5cc-ca6755ad1780" containerName="console" Sep 17 11:41:48.954589 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:48.954520 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b68da1c-44fc-4055-b5cc-ca6755ad1780" containerName="console" Sep 17 11:41:48.976198 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:48.976176 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:48.982078 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:48.982053 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64b6cf8966-lw8rv"] Sep 17 11:41:49.133389 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.133353 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-oauth-serving-cert\") pod \"console-64b6cf8966-lw8rv\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.133389 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.133393 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-service-ca\") pod \"console-64b6cf8966-lw8rv\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.133587 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.133423 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-console-config\") pod \"console-64b6cf8966-lw8rv\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.133587 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.133470 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bcf9f9e5-a441-4365-8357-e3d99722bd6c-console-serving-cert\") pod \"console-64b6cf8966-lw8rv\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.133587 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.133487 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-trusted-ca-bundle\") pod \"console-64b6cf8966-lw8rv\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.133587 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.133528 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bshn\" (UniqueName: \"kubernetes.io/projected/bcf9f9e5-a441-4365-8357-e3d99722bd6c-kube-api-access-2bshn\") pod \"console-64b6cf8966-lw8rv\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.133587 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.133569 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bcf9f9e5-a441-4365-8357-e3d99722bd6c-console-oauth-config\") pod \"console-64b6cf8966-lw8rv\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.234221 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.234139 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-console-config\") pod \"console-64b6cf8966-lw8rv\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.234221 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.234190 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bcf9f9e5-a441-4365-8357-e3d99722bd6c-console-serving-cert\") pod \"console-64b6cf8966-lw8rv\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.234221 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.234216 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-trusted-ca-bundle\") pod \"console-64b6cf8966-lw8rv\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.234487 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.234248 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bshn\" (UniqueName: \"kubernetes.io/projected/bcf9f9e5-a441-4365-8357-e3d99722bd6c-kube-api-access-2bshn\") pod \"console-64b6cf8966-lw8rv\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.234487 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.234276 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bcf9f9e5-a441-4365-8357-e3d99722bd6c-console-oauth-config\") pod \"console-64b6cf8966-lw8rv\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.234487 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.234350 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-oauth-serving-cert\") pod \"console-64b6cf8966-lw8rv\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.234487 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.234379 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-service-ca\") pod \"console-64b6cf8966-lw8rv\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.235104 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.234868 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-console-config\") pod \"console-64b6cf8966-lw8rv\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.235104 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.235062 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-trusted-ca-bundle\") pod \"console-64b6cf8966-lw8rv\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.235325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.235062 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-oauth-serving-cert\") pod \"console-64b6cf8966-lw8rv\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.235385 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.235318 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-service-ca\") pod \"console-64b6cf8966-lw8rv\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.237013 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.236988 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bcf9f9e5-a441-4365-8357-e3d99722bd6c-console-serving-cert\") pod \"console-64b6cf8966-lw8rv\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.237098 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.236988 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bcf9f9e5-a441-4365-8357-e3d99722bd6c-console-oauth-config\") pod \"console-64b6cf8966-lw8rv\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.244247 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.244224 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bshn\" (UniqueName: \"kubernetes.io/projected/bcf9f9e5-a441-4365-8357-e3d99722bd6c-kube-api-access-2bshn\") pod \"console-64b6cf8966-lw8rv\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.286260 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.286230 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:49.611025 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:49.611000 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64b6cf8966-lw8rv"] Sep 17 11:41:49.612319 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:41:49.612289 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcf9f9e5_a441_4365_8357_e3d99722bd6c.slice/crio-05b95a543c40f7bd7d0e7ba8bd5067707032e3e1baf54f7633bdc84a6da02769 WatchSource:0}: Error finding container 05b95a543c40f7bd7d0e7ba8bd5067707032e3e1baf54f7633bdc84a6da02769: Status 404 returned error can't find the container with id 05b95a543c40f7bd7d0e7ba8bd5067707032e3e1baf54f7633bdc84a6da02769 Sep 17 11:41:50.306609 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:50.306571 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64b6cf8966-lw8rv" event={"ID":"bcf9f9e5-a441-4365-8357-e3d99722bd6c","Type":"ContainerStarted","Data":"e63aafe1ed4d47d13bceac33a729fc908d53dbba05cb6a897470b06272be68a2"} Sep 17 11:41:50.306982 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:50.306620 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64b6cf8966-lw8rv" event={"ID":"bcf9f9e5-a441-4365-8357-e3d99722bd6c","Type":"ContainerStarted","Data":"05b95a543c40f7bd7d0e7ba8bd5067707032e3e1baf54f7633bdc84a6da02769"} Sep 17 11:41:50.332800 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:50.332758 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64b6cf8966-lw8rv" podStartSLOduration=2.332741785 podStartE2EDuration="2.332741785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:41:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:41:50.331806467 +0000 UTC m=+180.339253497" watchObservedRunningTime="2026-09-17 11:41:50.332741785 +0000 UTC m=+180.340188815" Sep 17 11:41:52.192129 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.192082 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 17 11:41:52.221218 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.221188 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 17 11:41:52.221368 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.221287 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.224375 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.224349 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 17 11:41:52.224375 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.224362 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 17 11:41:52.224586 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.224415 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 17 11:41:52.224586 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.224455 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 17 11:41:52.224735 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.224707 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 17 11:41:52.224860 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.224750 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 17 11:41:52.224860 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.224780 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 17 11:41:52.224969 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.224871 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qklq7\"" Sep 17 11:41:52.225024 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.224985 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 17 11:41:52.230512 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.230493 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 17 11:41:52.359451 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.359423 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6c914b05-45d8-4851-bdb6-bff91edf3780-config-volume\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.359451 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.359454 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6c914b05-45d8-4851-bdb6-bff91edf3780-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.359645 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.359472 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6c914b05-45d8-4851-bdb6-bff91edf3780-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.359645 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.359495 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6c914b05-45d8-4851-bdb6-bff91edf3780-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.359645 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.359551 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6c914b05-45d8-4851-bdb6-bff91edf3780-config-out\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.359645 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.359597 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6c914b05-45d8-4851-bdb6-bff91edf3780-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.359645 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.359618 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c914b05-45d8-4851-bdb6-bff91edf3780-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.359645 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.359640 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6c914b05-45d8-4851-bdb6-bff91edf3780-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.359823 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.359655 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c914b05-45d8-4851-bdb6-bff91edf3780-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.359823 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.359674 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6c914b05-45d8-4851-bdb6-bff91edf3780-web-config\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.359823 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.359724 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k44pq\" (UniqueName: \"kubernetes.io/projected/6c914b05-45d8-4851-bdb6-bff91edf3780-kube-api-access-k44pq\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.359823 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.359753 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c914b05-45d8-4851-bdb6-bff91edf3780-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.359823 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.359786 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c914b05-45d8-4851-bdb6-bff91edf3780-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.460273 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.460198 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6c914b05-45d8-4851-bdb6-bff91edf3780-web-config\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.460273 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.460244 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k44pq\" (UniqueName: \"kubernetes.io/projected/6c914b05-45d8-4851-bdb6-bff91edf3780-kube-api-access-k44pq\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.460273 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.460270 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c914b05-45d8-4851-bdb6-bff91edf3780-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.460528 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.460300 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c914b05-45d8-4851-bdb6-bff91edf3780-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.460528 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.460334 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6c914b05-45d8-4851-bdb6-bff91edf3780-config-volume\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.460528 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.460348 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6c914b05-45d8-4851-bdb6-bff91edf3780-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.460528 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.460371 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6c914b05-45d8-4851-bdb6-bff91edf3780-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.460528 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.460408 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6c914b05-45d8-4851-bdb6-bff91edf3780-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.460528 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.460440 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6c914b05-45d8-4851-bdb6-bff91edf3780-config-out\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.460528 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.460470 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6c914b05-45d8-4851-bdb6-bff91edf3780-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.460528 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.460493 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c914b05-45d8-4851-bdb6-bff91edf3780-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.460528 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.460514 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6c914b05-45d8-4851-bdb6-bff91edf3780-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.460528 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.460529 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c914b05-45d8-4851-bdb6-bff91edf3780-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.460991 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.460879 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6c914b05-45d8-4851-bdb6-bff91edf3780-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.461825 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.461798 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c914b05-45d8-4851-bdb6-bff91edf3780-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.462682 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.462659 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c914b05-45d8-4851-bdb6-bff91edf3780-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.463624 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.463599 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6c914b05-45d8-4851-bdb6-bff91edf3780-web-config\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.463733 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.463632 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c914b05-45d8-4851-bdb6-bff91edf3780-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.463973 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.463929 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c914b05-45d8-4851-bdb6-bff91edf3780-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.464068 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.464020 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6c914b05-45d8-4851-bdb6-bff91edf3780-config-volume\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.464168 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.464111 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6c914b05-45d8-4851-bdb6-bff91edf3780-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.464231 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.464169 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6c914b05-45d8-4851-bdb6-bff91edf3780-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.464287 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.464244 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6c914b05-45d8-4851-bdb6-bff91edf3780-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.465263 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.465240 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6c914b05-45d8-4851-bdb6-bff91edf3780-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.465509 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.465489 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6c914b05-45d8-4851-bdb6-bff91edf3780-config-out\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.469227 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.469208 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k44pq\" (UniqueName: \"kubernetes.io/projected/6c914b05-45d8-4851-bdb6-bff91edf3780-kube-api-access-k44pq\") pod \"alertmanager-main-0\" (UID: \"6c914b05-45d8-4851-bdb6-bff91edf3780\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.532025 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.531997 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 17 11:41:52.853248 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:52.853215 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 17 11:41:52.855618 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:41:52.855591 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c914b05_45d8_4851_bdb6_bff91edf3780.slice/crio-d0e92ffea5df43eead4fc07620a7859054477faaef3a809eadd31aa6a04c7485 WatchSource:0}: Error finding container d0e92ffea5df43eead4fc07620a7859054477faaef3a809eadd31aa6a04c7485: Status 404 returned error can't find the container with id d0e92ffea5df43eead4fc07620a7859054477faaef3a809eadd31aa6a04c7485 Sep 17 11:41:53.320956 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:53.320924 2581 generic.go:356] "Generic (PLEG): container finished" podID="6c914b05-45d8-4851-bdb6-bff91edf3780" containerID="11eae0353a385722eb51983ba2597fae96af1895c758aeb2fd490eed503043bb" exitCode=0 Sep 17 11:41:53.321407 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:53.320995 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c914b05-45d8-4851-bdb6-bff91edf3780","Type":"ContainerDied","Data":"11eae0353a385722eb51983ba2597fae96af1895c758aeb2fd490eed503043bb"} Sep 17 11:41:53.321407 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:53.321072 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c914b05-45d8-4851-bdb6-bff91edf3780","Type":"ContainerStarted","Data":"d0e92ffea5df43eead4fc07620a7859054477faaef3a809eadd31aa6a04c7485"} Sep 17 11:41:55.333297 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:55.333262 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c914b05-45d8-4851-bdb6-bff91edf3780","Type":"ContainerStarted","Data":"224f2c1beebef2e4dc92338cb96793d6558e58baf161a99da17ab6a56a12eefc"} Sep 17 11:41:55.333704 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:55.333333 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c914b05-45d8-4851-bdb6-bff91edf3780","Type":"ContainerStarted","Data":"581d59e05630573f3b961d38fa1ccc61352e690d01f6559d59e722f676691058"} Sep 17 11:41:55.333704 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:55.333358 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c914b05-45d8-4851-bdb6-bff91edf3780","Type":"ContainerStarted","Data":"d350be733bade1041f59cb684a45dd9cc83dd4b26c71abc19078a555348bd24b"} Sep 17 11:41:55.333704 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:55.333388 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c914b05-45d8-4851-bdb6-bff91edf3780","Type":"ContainerStarted","Data":"2816238aadc1c4175a13ecdc79bd57337919007dc1c8c874393efead784708a7"} Sep 17 11:41:55.333704 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:55.333412 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c914b05-45d8-4851-bdb6-bff91edf3780","Type":"ContainerStarted","Data":"3ae684e72396874b88ff1ce8c631b8fdd4d851d84d93dc2eeeec6aa1cc7b09ea"} Sep 17 11:41:55.333704 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:55.333444 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c914b05-45d8-4851-bdb6-bff91edf3780","Type":"ContainerStarted","Data":"eaf31ea6ee45defa7d51654ea9bd2a895a253cec8136e52bb2bc25984a680bbe"} Sep 17 11:41:55.362103 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:55.362057 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.9988762 podStartE2EDuration="3.362032237s" totalImagesPullingTime="1.348150786s" totalInitContainerRuntime="15.005251ms" isStatefulPod=false podCreationTimestamp="2026-09-17 11:41:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:41:55.36132846 +0000 UTC m=+185.368775489" watchObservedRunningTime="2026-09-17 11:41:55.362032237 +0000 UTC m=+185.369479266" Sep 17 11:41:59.286392 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:59.286350 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:59.286858 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:59.286412 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:59.291314 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:59.291289 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:59.355222 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:59.355188 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:41:59.407089 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:41:59.407060 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c9f96bcb8-psrgk"] Sep 17 11:42:24.427642 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.427604 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c9f96bcb8-psrgk" podUID="6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4" containerName="console" containerID="cri-o://55621d08c5b51d56539c22eb9a03c8f73dce7c2eeae6d1698325fbd499db5392" gracePeriod=15 Sep 17 11:42:24.676151 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.676109 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:42:24.728906 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.728825 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gf4sn\" (UniqueName: \"kubernetes.io/projected/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-kube-api-access-gf4sn\") pod \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " Sep 17 11:42:24.728906 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.728882 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-console-config\") pod \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " Sep 17 11:42:24.729132 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.728926 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-trusted-ca-bundle\") pod \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " Sep 17 11:42:24.729132 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.728955 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-service-ca\") pod \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " Sep 17 11:42:24.729132 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.729010 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-console-serving-cert\") pod \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " Sep 17 11:42:24.729132 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.729033 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-console-oauth-config\") pod \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " Sep 17 11:42:24.729132 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.729059 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-oauth-serving-cert\") pod \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\" (UID: \"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4\") " Sep 17 11:42:24.729495 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.729359 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-console-config" (OuterVolumeSpecName: "console-config") pod "6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4" (UID: "6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 11:42:24.729548 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.729465 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4" (UID: "6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 11:42:24.729823 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.729794 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-service-ca" (OuterVolumeSpecName: "service-ca") pod "6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4" (UID: "6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 11:42:24.729980 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.729955 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4" (UID: "6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 11:42:24.731305 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.731276 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-kube-api-access-gf4sn" (OuterVolumeSpecName: "kube-api-access-gf4sn") pod "6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4" (UID: "6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4"). InnerVolumeSpecName "kube-api-access-gf4sn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 11:42:24.731419 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.731311 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4" (UID: "6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 11:42:24.731419 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.731324 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4" (UID: "6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 11:42:24.830197 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.830164 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gf4sn\" (UniqueName: \"kubernetes.io/projected/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-kube-api-access-gf4sn\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:42:24.830197 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.830195 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-console-config\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:42:24.830371 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.830208 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-trusted-ca-bundle\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:42:24.830371 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.830219 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-service-ca\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:42:24.830371 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.830231 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-console-serving-cert\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:42:24.830371 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.830241 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-console-oauth-config\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:42:24.830371 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:24.830253 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4-oauth-serving-cert\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:42:25.458816 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:25.458779 2581 generic.go:356] "Generic (PLEG): container finished" podID="6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4" containerID="55621d08c5b51d56539c22eb9a03c8f73dce7c2eeae6d1698325fbd499db5392" exitCode=0 Sep 17 11:42:25.459316 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:25.458823 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c9f96bcb8-psrgk" Sep 17 11:42:25.459316 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:25.458822 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c9f96bcb8-psrgk" event={"ID":"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4","Type":"ContainerDied","Data":"55621d08c5b51d56539c22eb9a03c8f73dce7c2eeae6d1698325fbd499db5392"} Sep 17 11:42:25.459316 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:25.458971 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c9f96bcb8-psrgk" event={"ID":"6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4","Type":"ContainerDied","Data":"c99721295b0b5eb1ffc5ffa922ce64788cf5cf93772b156ba35bcb86fb56d706"} Sep 17 11:42:25.459316 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:25.459008 2581 scope.go:117] "RemoveContainer" containerID="55621d08c5b51d56539c22eb9a03c8f73dce7c2eeae6d1698325fbd499db5392" Sep 17 11:42:25.468205 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:25.468190 2581 scope.go:117] "RemoveContainer" containerID="55621d08c5b51d56539c22eb9a03c8f73dce7c2eeae6d1698325fbd499db5392" Sep 17 11:42:25.468438 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:42:25.468422 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55621d08c5b51d56539c22eb9a03c8f73dce7c2eeae6d1698325fbd499db5392\": container with ID starting with 55621d08c5b51d56539c22eb9a03c8f73dce7c2eeae6d1698325fbd499db5392 not found: ID does not exist" containerID="55621d08c5b51d56539c22eb9a03c8f73dce7c2eeae6d1698325fbd499db5392" Sep 17 11:42:25.468483 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:25.468446 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55621d08c5b51d56539c22eb9a03c8f73dce7c2eeae6d1698325fbd499db5392"} err="failed to get container status \"55621d08c5b51d56539c22eb9a03c8f73dce7c2eeae6d1698325fbd499db5392\": rpc error: code = NotFound desc = could not find container \"55621d08c5b51d56539c22eb9a03c8f73dce7c2eeae6d1698325fbd499db5392\": container with ID starting with 55621d08c5b51d56539c22eb9a03c8f73dce7c2eeae6d1698325fbd499db5392 not found: ID does not exist" Sep 17 11:42:25.480154 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:25.480108 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c9f96bcb8-psrgk"] Sep 17 11:42:25.487183 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:25.487164 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c9f96bcb8-psrgk"] Sep 17 11:42:26.429470 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:42:26.429441 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4" path="/var/lib/kubelet/pods/6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4/volumes" Sep 17 11:43:50.331103 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:43:50.331069 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/2.log" Sep 17 11:43:50.336209 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:43:50.336181 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/2.log" Sep 17 11:43:50.342511 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:43:50.342486 2581 kubelet.go:1648] "Image garbage collection succeeded" Sep 17 11:46:01.181667 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.181633 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75bfb5fc9-gzdjt"] Sep 17 11:46:01.182184 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.181959 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4" containerName="console" Sep 17 11:46:01.182184 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.181971 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4" containerName="console" Sep 17 11:46:01.182184 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.182036 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6946d2fa-bd8c-4bd4-a77e-9d2a252c91b4" containerName="console" Sep 17 11:46:01.184978 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.184958 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.268032 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.267996 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cce32baa-7461-40f2-98cd-84c30f043a53-console-serving-cert\") pod \"console-75bfb5fc9-gzdjt\" (UID: \"cce32baa-7461-40f2-98cd-84c30f043a53\") " pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.268195 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.268044 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cce32baa-7461-40f2-98cd-84c30f043a53-service-ca\") pod \"console-75bfb5fc9-gzdjt\" (UID: \"cce32baa-7461-40f2-98cd-84c30f043a53\") " pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.268195 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.268069 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cce32baa-7461-40f2-98cd-84c30f043a53-trusted-ca-bundle\") pod \"console-75bfb5fc9-gzdjt\" (UID: \"cce32baa-7461-40f2-98cd-84c30f043a53\") " pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.268195 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.268092 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zxng\" (UniqueName: \"kubernetes.io/projected/cce32baa-7461-40f2-98cd-84c30f043a53-kube-api-access-2zxng\") pod \"console-75bfb5fc9-gzdjt\" (UID: \"cce32baa-7461-40f2-98cd-84c30f043a53\") " pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.268195 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.268135 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cce32baa-7461-40f2-98cd-84c30f043a53-console-config\") pod \"console-75bfb5fc9-gzdjt\" (UID: \"cce32baa-7461-40f2-98cd-84c30f043a53\") " pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.268195 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.268168 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cce32baa-7461-40f2-98cd-84c30f043a53-oauth-serving-cert\") pod \"console-75bfb5fc9-gzdjt\" (UID: \"cce32baa-7461-40f2-98cd-84c30f043a53\") " pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.268361 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.268268 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cce32baa-7461-40f2-98cd-84c30f043a53-console-oauth-config\") pod \"console-75bfb5fc9-gzdjt\" (UID: \"cce32baa-7461-40f2-98cd-84c30f043a53\") " pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.331037 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.331009 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75bfb5fc9-gzdjt"] Sep 17 11:46:01.368982 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.368951 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cce32baa-7461-40f2-98cd-84c30f043a53-service-ca\") pod \"console-75bfb5fc9-gzdjt\" (UID: \"cce32baa-7461-40f2-98cd-84c30f043a53\") " pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.368982 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.368984 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cce32baa-7461-40f2-98cd-84c30f043a53-trusted-ca-bundle\") pod \"console-75bfb5fc9-gzdjt\" (UID: \"cce32baa-7461-40f2-98cd-84c30f043a53\") " pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.369194 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.369001 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zxng\" (UniqueName: \"kubernetes.io/projected/cce32baa-7461-40f2-98cd-84c30f043a53-kube-api-access-2zxng\") pod \"console-75bfb5fc9-gzdjt\" (UID: \"cce32baa-7461-40f2-98cd-84c30f043a53\") " pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.369194 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.369041 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cce32baa-7461-40f2-98cd-84c30f043a53-console-config\") pod \"console-75bfb5fc9-gzdjt\" (UID: \"cce32baa-7461-40f2-98cd-84c30f043a53\") " pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.369273 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.369179 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cce32baa-7461-40f2-98cd-84c30f043a53-oauth-serving-cert\") pod \"console-75bfb5fc9-gzdjt\" (UID: \"cce32baa-7461-40f2-98cd-84c30f043a53\") " pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.369329 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.369289 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cce32baa-7461-40f2-98cd-84c30f043a53-console-oauth-config\") pod \"console-75bfb5fc9-gzdjt\" (UID: \"cce32baa-7461-40f2-98cd-84c30f043a53\") " pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.369382 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.369352 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cce32baa-7461-40f2-98cd-84c30f043a53-console-serving-cert\") pod \"console-75bfb5fc9-gzdjt\" (UID: \"cce32baa-7461-40f2-98cd-84c30f043a53\") " pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.369771 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.369746 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cce32baa-7461-40f2-98cd-84c30f043a53-service-ca\") pod \"console-75bfb5fc9-gzdjt\" (UID: \"cce32baa-7461-40f2-98cd-84c30f043a53\") " pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.369866 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.369812 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cce32baa-7461-40f2-98cd-84c30f043a53-console-config\") pod \"console-75bfb5fc9-gzdjt\" (UID: \"cce32baa-7461-40f2-98cd-84c30f043a53\") " pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.369866 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.369841 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cce32baa-7461-40f2-98cd-84c30f043a53-oauth-serving-cert\") pod \"console-75bfb5fc9-gzdjt\" (UID: \"cce32baa-7461-40f2-98cd-84c30f043a53\") " pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.369975 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.369888 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cce32baa-7461-40f2-98cd-84c30f043a53-trusted-ca-bundle\") pod \"console-75bfb5fc9-gzdjt\" (UID: \"cce32baa-7461-40f2-98cd-84c30f043a53\") " pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.371933 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.371904 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cce32baa-7461-40f2-98cd-84c30f043a53-console-serving-cert\") pod \"console-75bfb5fc9-gzdjt\" (UID: \"cce32baa-7461-40f2-98cd-84c30f043a53\") " pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.372027 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.371910 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cce32baa-7461-40f2-98cd-84c30f043a53-console-oauth-config\") pod \"console-75bfb5fc9-gzdjt\" (UID: \"cce32baa-7461-40f2-98cd-84c30f043a53\") " pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.380376 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.380358 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zxng\" (UniqueName: \"kubernetes.io/projected/cce32baa-7461-40f2-98cd-84c30f043a53-kube-api-access-2zxng\") pod \"console-75bfb5fc9-gzdjt\" (UID: \"cce32baa-7461-40f2-98cd-84c30f043a53\") " pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.495019 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.494943 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:01.789523 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:01.789500 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75bfb5fc9-gzdjt"] Sep 17 11:46:01.791411 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:46:01.791385 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcce32baa_7461_40f2_98cd_84c30f043a53.slice/crio-74e7a8a06b1128b6cab4880675c5f0932d5061754e9d33660a599f4f6fba2852 WatchSource:0}: Error finding container 74e7a8a06b1128b6cab4880675c5f0932d5061754e9d33660a599f4f6fba2852: Status 404 returned error can't find the container with id 74e7a8a06b1128b6cab4880675c5f0932d5061754e9d33660a599f4f6fba2852 Sep 17 11:46:02.344011 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:02.343980 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75bfb5fc9-gzdjt" event={"ID":"cce32baa-7461-40f2-98cd-84c30f043a53","Type":"ContainerStarted","Data":"0c1affe15779a86740e093f1e1c94c350c32dc3b65dfb0fbf5de155cdf8c4827"} Sep 17 11:46:02.344528 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:02.344027 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75bfb5fc9-gzdjt" event={"ID":"cce32baa-7461-40f2-98cd-84c30f043a53","Type":"ContainerStarted","Data":"74e7a8a06b1128b6cab4880675c5f0932d5061754e9d33660a599f4f6fba2852"} Sep 17 11:46:02.380264 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:02.380217 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75bfb5fc9-gzdjt" podStartSLOduration=1.380201631 podStartE2EDuration="1.380201631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:46:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:46:02.379424279 +0000 UTC m=+432.386871309" watchObservedRunningTime="2026-09-17 11:46:02.380201631 +0000 UTC m=+432.387648661" Sep 17 11:46:11.495393 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:11.495360 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:11.495813 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:11.495418 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:11.500442 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:11.500424 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:12.395081 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:12.395052 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75bfb5fc9-gzdjt" Sep 17 11:46:12.471173 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:12.471141 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64b6cf8966-lw8rv"] Sep 17 11:46:37.490534 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.490479 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-64b6cf8966-lw8rv" podUID="bcf9f9e5-a441-4365-8357-e3d99722bd6c" containerName="console" containerID="cri-o://e63aafe1ed4d47d13bceac33a729fc908d53dbba05cb6a897470b06272be68a2" gracePeriod=15 Sep 17 11:46:37.740707 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.740649 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:46:37.873163 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.873111 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-oauth-serving-cert\") pod \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " Sep 17 11:46:37.873163 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.873163 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-service-ca\") pod \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " Sep 17 11:46:37.873399 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.873192 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-trusted-ca-bundle\") pod \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " Sep 17 11:46:37.873399 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.873276 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bcf9f9e5-a441-4365-8357-e3d99722bd6c-console-oauth-config\") pod \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " Sep 17 11:46:37.873399 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.873301 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bcf9f9e5-a441-4365-8357-e3d99722bd6c-console-serving-cert\") pod \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " Sep 17 11:46:37.873399 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.873341 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-console-config\") pod \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " Sep 17 11:46:37.873399 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.873389 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bshn\" (UniqueName: \"kubernetes.io/projected/bcf9f9e5-a441-4365-8357-e3d99722bd6c-kube-api-access-2bshn\") pod \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\" (UID: \"bcf9f9e5-a441-4365-8357-e3d99722bd6c\") " Sep 17 11:46:37.873686 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.873647 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bcf9f9e5-a441-4365-8357-e3d99722bd6c" (UID: "bcf9f9e5-a441-4365-8357-e3d99722bd6c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 11:46:37.873759 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.873695 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-service-ca" (OuterVolumeSpecName: "service-ca") pod "bcf9f9e5-a441-4365-8357-e3d99722bd6c" (UID: "bcf9f9e5-a441-4365-8357-e3d99722bd6c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 11:46:37.873759 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.873718 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bcf9f9e5-a441-4365-8357-e3d99722bd6c" (UID: "bcf9f9e5-a441-4365-8357-e3d99722bd6c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 11:46:37.873837 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.873805 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-console-config" (OuterVolumeSpecName: "console-config") pod "bcf9f9e5-a441-4365-8357-e3d99722bd6c" (UID: "bcf9f9e5-a441-4365-8357-e3d99722bd6c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 11:46:37.875657 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.875628 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcf9f9e5-a441-4365-8357-e3d99722bd6c-kube-api-access-2bshn" (OuterVolumeSpecName: "kube-api-access-2bshn") pod "bcf9f9e5-a441-4365-8357-e3d99722bd6c" (UID: "bcf9f9e5-a441-4365-8357-e3d99722bd6c"). InnerVolumeSpecName "kube-api-access-2bshn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 11:46:37.875657 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.875630 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcf9f9e5-a441-4365-8357-e3d99722bd6c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bcf9f9e5-a441-4365-8357-e3d99722bd6c" (UID: "bcf9f9e5-a441-4365-8357-e3d99722bd6c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 11:46:37.875821 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.875680 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcf9f9e5-a441-4365-8357-e3d99722bd6c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bcf9f9e5-a441-4365-8357-e3d99722bd6c" (UID: "bcf9f9e5-a441-4365-8357-e3d99722bd6c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 11:46:37.974374 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.974339 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bshn\" (UniqueName: \"kubernetes.io/projected/bcf9f9e5-a441-4365-8357-e3d99722bd6c-kube-api-access-2bshn\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:46:37.974374 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.974367 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-oauth-serving-cert\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:46:37.974374 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.974376 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-service-ca\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:46:37.974607 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.974385 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-trusted-ca-bundle\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:46:37.974607 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.974395 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bcf9f9e5-a441-4365-8357-e3d99722bd6c-console-oauth-config\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:46:37.974607 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.974403 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bcf9f9e5-a441-4365-8357-e3d99722bd6c-console-serving-cert\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:46:37.974607 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:37.974412 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bcf9f9e5-a441-4365-8357-e3d99722bd6c-console-config\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:46:38.497902 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:38.497866 2581 generic.go:356] "Generic (PLEG): container finished" podID="bcf9f9e5-a441-4365-8357-e3d99722bd6c" containerID="e63aafe1ed4d47d13bceac33a729fc908d53dbba05cb6a897470b06272be68a2" exitCode=0 Sep 17 11:46:38.498352 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:38.497904 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64b6cf8966-lw8rv" event={"ID":"bcf9f9e5-a441-4365-8357-e3d99722bd6c","Type":"ContainerDied","Data":"e63aafe1ed4d47d13bceac33a729fc908d53dbba05cb6a897470b06272be68a2"} Sep 17 11:46:38.498352 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:38.497918 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64b6cf8966-lw8rv" Sep 17 11:46:38.498352 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:38.497962 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64b6cf8966-lw8rv" event={"ID":"bcf9f9e5-a441-4365-8357-e3d99722bd6c","Type":"ContainerDied","Data":"05b95a543c40f7bd7d0e7ba8bd5067707032e3e1baf54f7633bdc84a6da02769"} Sep 17 11:46:38.498352 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:38.497982 2581 scope.go:117] "RemoveContainer" containerID="e63aafe1ed4d47d13bceac33a729fc908d53dbba05cb6a897470b06272be68a2" Sep 17 11:46:38.506699 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:38.506682 2581 scope.go:117] "RemoveContainer" containerID="e63aafe1ed4d47d13bceac33a729fc908d53dbba05cb6a897470b06272be68a2" Sep 17 11:46:38.506978 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:46:38.506942 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e63aafe1ed4d47d13bceac33a729fc908d53dbba05cb6a897470b06272be68a2\": container with ID starting with e63aafe1ed4d47d13bceac33a729fc908d53dbba05cb6a897470b06272be68a2 not found: ID does not exist" containerID="e63aafe1ed4d47d13bceac33a729fc908d53dbba05cb6a897470b06272be68a2" Sep 17 11:46:38.506978 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:38.506968 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e63aafe1ed4d47d13bceac33a729fc908d53dbba05cb6a897470b06272be68a2"} err="failed to get container status \"e63aafe1ed4d47d13bceac33a729fc908d53dbba05cb6a897470b06272be68a2\": rpc error: code = NotFound desc = could not find container \"e63aafe1ed4d47d13bceac33a729fc908d53dbba05cb6a897470b06272be68a2\": container with ID starting with e63aafe1ed4d47d13bceac33a729fc908d53dbba05cb6a897470b06272be68a2 not found: ID does not exist" Sep 17 11:46:38.519548 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:38.519524 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64b6cf8966-lw8rv"] Sep 17 11:46:38.526526 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:38.526504 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64b6cf8966-lw8rv"] Sep 17 11:46:40.427920 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:46:40.427889 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcf9f9e5-a441-4365-8357-e3d99722bd6c" path="/var/lib/kubelet/pods/bcf9f9e5-a441-4365-8357-e3d99722bd6c/volumes" Sep 17 11:48:50.361954 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:48:50.361917 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/2.log" Sep 17 11:48:50.363054 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:48:50.363034 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/2.log" Sep 17 11:50:39.894030 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:50:39.893998 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-585pp"] Sep 17 11:50:39.894479 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:50:39.894450 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcf9f9e5-a441-4365-8357-e3d99722bd6c" containerName="console" Sep 17 11:50:39.894479 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:50:39.894464 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcf9f9e5-a441-4365-8357-e3d99722bd6c" containerName="console" Sep 17 11:50:39.894564 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:50:39.894544 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcf9f9e5-a441-4365-8357-e3d99722bd6c" containerName="console" Sep 17 11:50:39.897461 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:50:39.897445 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-585pp" Sep 17 11:50:39.900902 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:50:39.900879 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 17 11:50:39.901040 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:50:39.900922 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Sep 17 11:50:39.901721 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:50:39.901700 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-qxcvl\"" Sep 17 11:50:39.901883 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:50:39.901755 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 17 11:50:39.908996 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:50:39.908974 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-585pp"] Sep 17 11:50:39.984550 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:50:39.984519 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2grq\" (UniqueName: \"kubernetes.io/projected/1eab514c-b911-482c-8720-9cd054daaa56-kube-api-access-d2grq\") pod \"postgres-864846d4f5-585pp\" (UID: \"1eab514c-b911-482c-8720-9cd054daaa56\") " pod="test-ns-feast/postgres-864846d4f5-585pp" Sep 17 11:50:39.984694 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:50:39.984605 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/1eab514c-b911-482c-8720-9cd054daaa56-postgresdata\") pod \"postgres-864846d4f5-585pp\" (UID: \"1eab514c-b911-482c-8720-9cd054daaa56\") " pod="test-ns-feast/postgres-864846d4f5-585pp" Sep 17 11:50:40.085878 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:50:40.085840 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2grq\" (UniqueName: \"kubernetes.io/projected/1eab514c-b911-482c-8720-9cd054daaa56-kube-api-access-d2grq\") pod \"postgres-864846d4f5-585pp\" (UID: \"1eab514c-b911-482c-8720-9cd054daaa56\") " pod="test-ns-feast/postgres-864846d4f5-585pp" Sep 17 11:50:40.086057 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:50:40.085902 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/1eab514c-b911-482c-8720-9cd054daaa56-postgresdata\") pod \"postgres-864846d4f5-585pp\" (UID: \"1eab514c-b911-482c-8720-9cd054daaa56\") " pod="test-ns-feast/postgres-864846d4f5-585pp" Sep 17 11:50:40.086257 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:50:40.086240 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/1eab514c-b911-482c-8720-9cd054daaa56-postgresdata\") pod \"postgres-864846d4f5-585pp\" (UID: \"1eab514c-b911-482c-8720-9cd054daaa56\") " pod="test-ns-feast/postgres-864846d4f5-585pp" Sep 17 11:50:40.094472 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:50:40.094448 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2grq\" (UniqueName: \"kubernetes.io/projected/1eab514c-b911-482c-8720-9cd054daaa56-kube-api-access-d2grq\") pod \"postgres-864846d4f5-585pp\" (UID: \"1eab514c-b911-482c-8720-9cd054daaa56\") " pod="test-ns-feast/postgres-864846d4f5-585pp" Sep 17 11:50:40.206977 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:50:40.206911 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-585pp" Sep 17 11:50:40.476852 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:50:40.476770 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-585pp"] Sep 17 11:50:40.481845 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:50:40.481816 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1eab514c_b911_482c_8720_9cd054daaa56.slice/crio-cd9bd0945c2ddcb0a7760bac2ae4ab9bbc3860833d0a1403717e7239e6f6a542 WatchSource:0}: Error finding container cd9bd0945c2ddcb0a7760bac2ae4ab9bbc3860833d0a1403717e7239e6f6a542: Status 404 returned error can't find the container with id cd9bd0945c2ddcb0a7760bac2ae4ab9bbc3860833d0a1403717e7239e6f6a542 Sep 17 11:50:40.483710 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:50:40.483695 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 11:50:40.488940 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:50:40.488915 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-585pp" event={"ID":"1eab514c-b911-482c-8720-9cd054daaa56","Type":"ContainerStarted","Data":"cd9bd0945c2ddcb0a7760bac2ae4ab9bbc3860833d0a1403717e7239e6f6a542"} Sep 17 11:50:45.512942 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:50:45.512896 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-585pp" event={"ID":"1eab514c-b911-482c-8720-9cd054daaa56","Type":"ContainerStarted","Data":"470fb25c86c6c4b8d7ccdb8635891aa0d0b97cb535ae03eca0390c1b1b98abfc"} Sep 17 11:52:16.713352 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:52:16.713244 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-585pp" podStartSLOduration=93.023681547 podStartE2EDuration="1m37.713227675s" totalImagesPullingTime="4.689546128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:50:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:50:45.542263506 +0000 UTC m=+715.549710532" watchObservedRunningTime="2026-09-17 11:52:16.713227675 +0000 UTC m=+806.720674704" Sep 17 11:52:16.714005 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:52:16.713986 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-585pp"] Sep 17 11:52:16.714229 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:52:16.714204 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-585pp" podUID="1eab514c-b911-482c-8720-9cd054daaa56" containerName="postgres" containerID="cri-o://470fb25c86c6c4b8d7ccdb8635891aa0d0b97cb535ae03eca0390c1b1b98abfc" gracePeriod=30 Sep 17 11:52:16.890221 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:52:16.890187 2581 generic.go:356] "Generic (PLEG): container finished" podID="1eab514c-b911-482c-8720-9cd054daaa56" containerID="470fb25c86c6c4b8d7ccdb8635891aa0d0b97cb535ae03eca0390c1b1b98abfc" exitCode=0 Sep 17 11:52:16.890363 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:52:16.890252 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-585pp" event={"ID":"1eab514c-b911-482c-8720-9cd054daaa56","Type":"ContainerDied","Data":"470fb25c86c6c4b8d7ccdb8635891aa0d0b97cb535ae03eca0390c1b1b98abfc"} Sep 17 11:52:16.968306 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:52:16.968242 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-585pp" Sep 17 11:52:17.101083 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:52:17.101050 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2grq\" (UniqueName: \"kubernetes.io/projected/1eab514c-b911-482c-8720-9cd054daaa56-kube-api-access-d2grq\") pod \"1eab514c-b911-482c-8720-9cd054daaa56\" (UID: \"1eab514c-b911-482c-8720-9cd054daaa56\") " Sep 17 11:52:17.101325 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:52:17.101095 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/1eab514c-b911-482c-8720-9cd054daaa56-postgresdata\") pod \"1eab514c-b911-482c-8720-9cd054daaa56\" (UID: \"1eab514c-b911-482c-8720-9cd054daaa56\") " Sep 17 11:52:17.103956 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:52:17.103927 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1eab514c-b911-482c-8720-9cd054daaa56-kube-api-access-d2grq" (OuterVolumeSpecName: "kube-api-access-d2grq") pod "1eab514c-b911-482c-8720-9cd054daaa56" (UID: "1eab514c-b911-482c-8720-9cd054daaa56"). InnerVolumeSpecName "kube-api-access-d2grq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 11:52:17.129854 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:52:17.129825 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1eab514c-b911-482c-8720-9cd054daaa56-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "1eab514c-b911-482c-8720-9cd054daaa56" (UID: "1eab514c-b911-482c-8720-9cd054daaa56"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 11:52:17.202324 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:52:17.202291 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2grq\" (UniqueName: \"kubernetes.io/projected/1eab514c-b911-482c-8720-9cd054daaa56-kube-api-access-d2grq\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:52:17.202324 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:52:17.202320 2581 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/1eab514c-b911-482c-8720-9cd054daaa56-postgresdata\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:52:17.896388 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:52:17.896360 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-585pp" Sep 17 11:52:17.896819 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:52:17.896400 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-585pp" event={"ID":"1eab514c-b911-482c-8720-9cd054daaa56","Type":"ContainerDied","Data":"cd9bd0945c2ddcb0a7760bac2ae4ab9bbc3860833d0a1403717e7239e6f6a542"} Sep 17 11:52:17.896819 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:52:17.896450 2581 scope.go:117] "RemoveContainer" containerID="470fb25c86c6c4b8d7ccdb8635891aa0d0b97cb535ae03eca0390c1b1b98abfc" Sep 17 11:52:17.921358 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:52:17.921330 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-585pp"] Sep 17 11:52:17.924822 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:52:17.924799 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-585pp"] Sep 17 11:52:18.428025 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:52:18.427981 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1eab514c-b911-482c-8720-9cd054daaa56" path="/var/lib/kubelet/pods/1eab514c-b911-482c-8720-9cd054daaa56/volumes" Sep 17 11:53:50.389236 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:53:50.389162 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/2.log" Sep 17 11:53:50.389741 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:53:50.389355 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/2.log" Sep 17 11:55:47.865986 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:47.865953 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-6d5jq"] Sep 17 11:55:47.868538 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:47.866469 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1eab514c-b911-482c-8720-9cd054daaa56" containerName="postgres" Sep 17 11:55:47.868538 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:47.866486 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1eab514c-b911-482c-8720-9cd054daaa56" containerName="postgres" Sep 17 11:55:47.868538 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:47.866550 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1eab514c-b911-482c-8720-9cd054daaa56" containerName="postgres" Sep 17 11:55:47.869447 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:47.869424 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-6d5jq" Sep 17 11:55:47.872074 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:47.872052 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Sep 17 11:55:47.872226 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:47.872130 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Sep 17 11:55:47.872226 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:47.872204 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-xc9d8\"" Sep 17 11:55:47.883937 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:47.883909 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-6d5jq"] Sep 17 11:55:47.974216 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:47.974172 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn57b\" (UniqueName: \"kubernetes.io/projected/a65d8168-139c-4ddc-a7b4-c069e8b6396f-kube-api-access-sn57b\") pod \"redis-69bbd798b6-6d5jq\" (UID: \"a65d8168-139c-4ddc-a7b4-c069e8b6396f\") " pod="test-ns-feast-rest/redis-69bbd798b6-6d5jq" Sep 17 11:55:48.075074 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:48.075028 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn57b\" (UniqueName: \"kubernetes.io/projected/a65d8168-139c-4ddc-a7b4-c069e8b6396f-kube-api-access-sn57b\") pod \"redis-69bbd798b6-6d5jq\" (UID: \"a65d8168-139c-4ddc-a7b4-c069e8b6396f\") " pod="test-ns-feast-rest/redis-69bbd798b6-6d5jq" Sep 17 11:55:48.084033 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:48.084002 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn57b\" (UniqueName: \"kubernetes.io/projected/a65d8168-139c-4ddc-a7b4-c069e8b6396f-kube-api-access-sn57b\") pod \"redis-69bbd798b6-6d5jq\" (UID: \"a65d8168-139c-4ddc-a7b4-c069e8b6396f\") " pod="test-ns-feast-rest/redis-69bbd798b6-6d5jq" Sep 17 11:55:48.179241 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:48.179112 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-6d5jq" Sep 17 11:55:48.455194 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:48.455149 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-6d5jq"] Sep 17 11:55:48.457876 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:55:48.457851 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda65d8168_139c_4ddc_a7b4_c069e8b6396f.slice/crio-f130c1d9605cbdcbcf72a9c50e93a15b04c9d6d19af74ff38a85ebfa664e1276 WatchSource:0}: Error finding container f130c1d9605cbdcbcf72a9c50e93a15b04c9d6d19af74ff38a85ebfa664e1276: Status 404 returned error can't find the container with id f130c1d9605cbdcbcf72a9c50e93a15b04c9d6d19af74ff38a85ebfa664e1276 Sep 17 11:55:48.459669 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:48.459647 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 11:55:48.760521 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:48.760486 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-6d5jq" event={"ID":"a65d8168-139c-4ddc-a7b4-c069e8b6396f","Type":"ContainerStarted","Data":"f130c1d9605cbdcbcf72a9c50e93a15b04c9d6d19af74ff38a85ebfa664e1276"} Sep 17 11:55:50.771230 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:50.771189 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-6d5jq" event={"ID":"a65d8168-139c-4ddc-a7b4-c069e8b6396f","Type":"ContainerStarted","Data":"b62d09b45a0cea19b35db723bceb925f1b05ebf02a7c7e3429b1156f355d0a75"} Sep 17 11:55:50.791574 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:50.791525 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-6d5jq" podStartSLOduration=2.548577892 podStartE2EDuration="3.791509731s" totalImagesPullingTime="1.242931839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:55:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:55:50.790054333 +0000 UTC m=+1020.797501362" watchObservedRunningTime="2026-09-17 11:55:50.791509731 +0000 UTC m=+1020.798956761" Sep 17 11:55:53.306899 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:53.306858 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-fzc2s"] Sep 17 11:55:53.310946 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:53.310911 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-fzc2s" Sep 17 11:55:53.314176 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:53.314155 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Sep 17 11:55:53.320814 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:53.320790 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-fzc2s"] Sep 17 11:55:53.421642 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:53.421609 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kbx4\" (UniqueName: \"kubernetes.io/projected/17838de6-c210-42d3-b242-270940ba03a6-kube-api-access-5kbx4\") pod \"postgres-864846d4f5-fzc2s\" (UID: \"17838de6-c210-42d3-b242-270940ba03a6\") " pod="test-ns-feast-rest/postgres-864846d4f5-fzc2s" Sep 17 11:55:53.421806 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:53.421658 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/17838de6-c210-42d3-b242-270940ba03a6-postgresdata\") pod \"postgres-864846d4f5-fzc2s\" (UID: \"17838de6-c210-42d3-b242-270940ba03a6\") " pod="test-ns-feast-rest/postgres-864846d4f5-fzc2s" Sep 17 11:55:53.522957 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:53.522921 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/17838de6-c210-42d3-b242-270940ba03a6-postgresdata\") pod \"postgres-864846d4f5-fzc2s\" (UID: \"17838de6-c210-42d3-b242-270940ba03a6\") " pod="test-ns-feast-rest/postgres-864846d4f5-fzc2s" Sep 17 11:55:53.523142 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:53.523011 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kbx4\" (UniqueName: \"kubernetes.io/projected/17838de6-c210-42d3-b242-270940ba03a6-kube-api-access-5kbx4\") pod \"postgres-864846d4f5-fzc2s\" (UID: \"17838de6-c210-42d3-b242-270940ba03a6\") " pod="test-ns-feast-rest/postgres-864846d4f5-fzc2s" Sep 17 11:55:53.523342 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:53.523325 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/17838de6-c210-42d3-b242-270940ba03a6-postgresdata\") pod \"postgres-864846d4f5-fzc2s\" (UID: \"17838de6-c210-42d3-b242-270940ba03a6\") " pod="test-ns-feast-rest/postgres-864846d4f5-fzc2s" Sep 17 11:55:53.532179 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:53.532156 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kbx4\" (UniqueName: \"kubernetes.io/projected/17838de6-c210-42d3-b242-270940ba03a6-kube-api-access-5kbx4\") pod \"postgres-864846d4f5-fzc2s\" (UID: \"17838de6-c210-42d3-b242-270940ba03a6\") " pod="test-ns-feast-rest/postgres-864846d4f5-fzc2s" Sep 17 11:55:53.621758 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:53.621689 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-fzc2s" Sep 17 11:55:53.894162 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:53.894139 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-fzc2s"] Sep 17 11:55:53.896749 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:55:53.896719 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17838de6_c210_42d3_b242_270940ba03a6.slice/crio-b73e0fd11222f9d3f51df883156627050ac5b50a45d1a5b285761c25894eac3a WatchSource:0}: Error finding container b73e0fd11222f9d3f51df883156627050ac5b50a45d1a5b285761c25894eac3a: Status 404 returned error can't find the container with id b73e0fd11222f9d3f51df883156627050ac5b50a45d1a5b285761c25894eac3a Sep 17 11:55:54.789426 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:54.789387 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-fzc2s" event={"ID":"17838de6-c210-42d3-b242-270940ba03a6","Type":"ContainerStarted","Data":"111fd6003d8d15d91c20caec9cf272ef948f35b8fa3a448531c36d6341b4315a"} Sep 17 11:55:54.789802 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:54.789436 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-fzc2s" event={"ID":"17838de6-c210-42d3-b242-270940ba03a6","Type":"ContainerStarted","Data":"b73e0fd11222f9d3f51df883156627050ac5b50a45d1a5b285761c25894eac3a"} Sep 17 11:55:54.807667 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:55:54.807612 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-fzc2s" podStartSLOduration=1.807590781 podStartE2EDuration="1.807590781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:55:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:55:54.805399531 +0000 UTC m=+1024.812846561" watchObservedRunningTime="2026-09-17 11:55:54.807590781 +0000 UTC m=+1024.815037812" Sep 17 11:58:16.477904 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:16.477826 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-fzc2s"] Sep 17 11:58:16.478358 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:16.477997 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-fzc2s" podUID="17838de6-c210-42d3-b242-270940ba03a6" containerName="postgres" containerID="cri-o://111fd6003d8d15d91c20caec9cf272ef948f35b8fa3a448531c36d6341b4315a" gracePeriod=30 Sep 17 11:58:16.498243 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:16.498212 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-6d5jq"] Sep 17 11:58:16.498409 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:16.498386 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-6d5jq" podUID="a65d8168-139c-4ddc-a7b4-c069e8b6396f" containerName="redis" containerID="cri-o://b62d09b45a0cea19b35db723bceb925f1b05ebf02a7c7e3429b1156f355d0a75" gracePeriod=30 Sep 17 11:58:16.761771 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:16.761744 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-6d5jq" Sep 17 11:58:16.764524 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:16.764501 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-fzc2s" Sep 17 11:58:16.833727 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:16.833699 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kbx4\" (UniqueName: \"kubernetes.io/projected/17838de6-c210-42d3-b242-270940ba03a6-kube-api-access-5kbx4\") pod \"17838de6-c210-42d3-b242-270940ba03a6\" (UID: \"17838de6-c210-42d3-b242-270940ba03a6\") " Sep 17 11:58:16.833904 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:16.833806 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/17838de6-c210-42d3-b242-270940ba03a6-postgresdata\") pod \"17838de6-c210-42d3-b242-270940ba03a6\" (UID: \"17838de6-c210-42d3-b242-270940ba03a6\") " Sep 17 11:58:16.833904 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:16.833850 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sn57b\" (UniqueName: \"kubernetes.io/projected/a65d8168-139c-4ddc-a7b4-c069e8b6396f-kube-api-access-sn57b\") pod \"a65d8168-139c-4ddc-a7b4-c069e8b6396f\" (UID: \"a65d8168-139c-4ddc-a7b4-c069e8b6396f\") " Sep 17 11:58:16.836169 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:16.836139 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a65d8168-139c-4ddc-a7b4-c069e8b6396f-kube-api-access-sn57b" (OuterVolumeSpecName: "kube-api-access-sn57b") pod "a65d8168-139c-4ddc-a7b4-c069e8b6396f" (UID: "a65d8168-139c-4ddc-a7b4-c069e8b6396f"). InnerVolumeSpecName "kube-api-access-sn57b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 11:58:16.836334 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:16.836310 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17838de6-c210-42d3-b242-270940ba03a6-kube-api-access-5kbx4" (OuterVolumeSpecName: "kube-api-access-5kbx4") pod "17838de6-c210-42d3-b242-270940ba03a6" (UID: "17838de6-c210-42d3-b242-270940ba03a6"). InnerVolumeSpecName "kube-api-access-5kbx4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 11:58:16.865509 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:16.865472 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17838de6-c210-42d3-b242-270940ba03a6-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "17838de6-c210-42d3-b242-270940ba03a6" (UID: "17838de6-c210-42d3-b242-270940ba03a6"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 11:58:16.935451 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:16.935420 2581 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/17838de6-c210-42d3-b242-270940ba03a6-postgresdata\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:58:16.935451 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:16.935446 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sn57b\" (UniqueName: \"kubernetes.io/projected/a65d8168-139c-4ddc-a7b4-c069e8b6396f-kube-api-access-sn57b\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:58:16.935451 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:16.935455 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kbx4\" (UniqueName: \"kubernetes.io/projected/17838de6-c210-42d3-b242-270940ba03a6-kube-api-access-5kbx4\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 11:58:17.386024 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:17.385992 2581 generic.go:356] "Generic (PLEG): container finished" podID="17838de6-c210-42d3-b242-270940ba03a6" containerID="111fd6003d8d15d91c20caec9cf272ef948f35b8fa3a448531c36d6341b4315a" exitCode=0 Sep 17 11:58:17.386263 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:17.386035 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-fzc2s" Sep 17 11:58:17.386263 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:17.386075 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-fzc2s" event={"ID":"17838de6-c210-42d3-b242-270940ba03a6","Type":"ContainerDied","Data":"111fd6003d8d15d91c20caec9cf272ef948f35b8fa3a448531c36d6341b4315a"} Sep 17 11:58:17.386263 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:17.386147 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-fzc2s" event={"ID":"17838de6-c210-42d3-b242-270940ba03a6","Type":"ContainerDied","Data":"b73e0fd11222f9d3f51df883156627050ac5b50a45d1a5b285761c25894eac3a"} Sep 17 11:58:17.386263 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:17.386170 2581 scope.go:117] "RemoveContainer" containerID="111fd6003d8d15d91c20caec9cf272ef948f35b8fa3a448531c36d6341b4315a" Sep 17 11:58:17.387498 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:17.387438 2581 generic.go:356] "Generic (PLEG): container finished" podID="a65d8168-139c-4ddc-a7b4-c069e8b6396f" containerID="b62d09b45a0cea19b35db723bceb925f1b05ebf02a7c7e3429b1156f355d0a75" exitCode=0 Sep 17 11:58:17.387498 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:17.387481 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-6d5jq" Sep 17 11:58:17.387498 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:17.387491 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-6d5jq" event={"ID":"a65d8168-139c-4ddc-a7b4-c069e8b6396f","Type":"ContainerDied","Data":"b62d09b45a0cea19b35db723bceb925f1b05ebf02a7c7e3429b1156f355d0a75"} Sep 17 11:58:17.387675 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:17.387515 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-6d5jq" event={"ID":"a65d8168-139c-4ddc-a7b4-c069e8b6396f","Type":"ContainerDied","Data":"f130c1d9605cbdcbcf72a9c50e93a15b04c9d6d19af74ff38a85ebfa664e1276"} Sep 17 11:58:17.395851 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:17.395833 2581 scope.go:117] "RemoveContainer" containerID="111fd6003d8d15d91c20caec9cf272ef948f35b8fa3a448531c36d6341b4315a" Sep 17 11:58:17.396197 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:58:17.396174 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"111fd6003d8d15d91c20caec9cf272ef948f35b8fa3a448531c36d6341b4315a\": container with ID starting with 111fd6003d8d15d91c20caec9cf272ef948f35b8fa3a448531c36d6341b4315a not found: ID does not exist" containerID="111fd6003d8d15d91c20caec9cf272ef948f35b8fa3a448531c36d6341b4315a" Sep 17 11:58:17.396264 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:17.396210 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"111fd6003d8d15d91c20caec9cf272ef948f35b8fa3a448531c36d6341b4315a"} err="failed to get container status \"111fd6003d8d15d91c20caec9cf272ef948f35b8fa3a448531c36d6341b4315a\": rpc error: code = NotFound desc = could not find container \"111fd6003d8d15d91c20caec9cf272ef948f35b8fa3a448531c36d6341b4315a\": container with ID starting with 111fd6003d8d15d91c20caec9cf272ef948f35b8fa3a448531c36d6341b4315a not found: ID does not exist" Sep 17 11:58:17.396264 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:17.396233 2581 scope.go:117] "RemoveContainer" containerID="b62d09b45a0cea19b35db723bceb925f1b05ebf02a7c7e3429b1156f355d0a75" Sep 17 11:58:17.404215 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:17.404199 2581 scope.go:117] "RemoveContainer" containerID="b62d09b45a0cea19b35db723bceb925f1b05ebf02a7c7e3429b1156f355d0a75" Sep 17 11:58:17.404475 ip-10-0-140-196 kubenswrapper[2581]: E0917 11:58:17.404458 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b62d09b45a0cea19b35db723bceb925f1b05ebf02a7c7e3429b1156f355d0a75\": container with ID starting with b62d09b45a0cea19b35db723bceb925f1b05ebf02a7c7e3429b1156f355d0a75 not found: ID does not exist" containerID="b62d09b45a0cea19b35db723bceb925f1b05ebf02a7c7e3429b1156f355d0a75" Sep 17 11:58:17.404516 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:17.404481 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b62d09b45a0cea19b35db723bceb925f1b05ebf02a7c7e3429b1156f355d0a75"} err="failed to get container status \"b62d09b45a0cea19b35db723bceb925f1b05ebf02a7c7e3429b1156f355d0a75\": rpc error: code = NotFound desc = could not find container \"b62d09b45a0cea19b35db723bceb925f1b05ebf02a7c7e3429b1156f355d0a75\": container with ID starting with b62d09b45a0cea19b35db723bceb925f1b05ebf02a7c7e3429b1156f355d0a75 not found: ID does not exist" Sep 17 11:58:17.411351 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:17.411331 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-6d5jq"] Sep 17 11:58:17.415849 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:17.415831 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-6d5jq"] Sep 17 11:58:17.427617 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:17.427594 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-fzc2s"] Sep 17 11:58:17.429737 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:17.429716 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-fzc2s"] Sep 17 11:58:18.428452 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:18.428421 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17838de6-c210-42d3-b242-270940ba03a6" path="/var/lib/kubelet/pods/17838de6-c210-42d3-b242-270940ba03a6/volumes" Sep 17 11:58:18.428828 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:18.428717 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a65d8168-139c-4ddc-a7b4-c069e8b6396f" path="/var/lib/kubelet/pods/a65d8168-139c-4ddc-a7b4-c069e8b6396f/volumes" Sep 17 11:58:26.668634 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:26.668597 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-25hv7"] Sep 17 11:58:26.669013 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:26.668942 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a65d8168-139c-4ddc-a7b4-c069e8b6396f" containerName="redis" Sep 17 11:58:26.669013 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:26.668954 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a65d8168-139c-4ddc-a7b4-c069e8b6396f" containerName="redis" Sep 17 11:58:26.669013 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:26.668962 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17838de6-c210-42d3-b242-270940ba03a6" containerName="postgres" Sep 17 11:58:26.669013 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:26.668967 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="17838de6-c210-42d3-b242-270940ba03a6" containerName="postgres" Sep 17 11:58:26.669159 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:26.669031 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a65d8168-139c-4ddc-a7b4-c069e8b6396f" containerName="redis" Sep 17 11:58:26.669159 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:26.669043 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="17838de6-c210-42d3-b242-270940ba03a6" containerName="postgres" Sep 17 11:58:26.673362 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:26.673345 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-25hv7" Sep 17 11:58:26.677401 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:26.677380 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-n87gs\"" Sep 17 11:58:26.677618 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:26.677605 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 17 11:58:26.677675 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:26.677641 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 17 11:58:26.688982 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:26.688958 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-25hv7"] Sep 17 11:58:26.822826 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:26.819582 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27dz4\" (UniqueName: \"kubernetes.io/projected/a21333e3-a759-4b18-83f5-1df25c72abb6-kube-api-access-27dz4\") pod \"feast-operator-controller-manager-689b877b4b-25hv7\" (UID: \"a21333e3-a759-4b18-83f5-1df25c72abb6\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-25hv7" Sep 17 11:58:26.921137 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:26.921021 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27dz4\" (UniqueName: \"kubernetes.io/projected/a21333e3-a759-4b18-83f5-1df25c72abb6-kube-api-access-27dz4\") pod \"feast-operator-controller-manager-689b877b4b-25hv7\" (UID: \"a21333e3-a759-4b18-83f5-1df25c72abb6\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-25hv7" Sep 17 11:58:26.944104 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:26.944070 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27dz4\" (UniqueName: \"kubernetes.io/projected/a21333e3-a759-4b18-83f5-1df25c72abb6-kube-api-access-27dz4\") pod \"feast-operator-controller-manager-689b877b4b-25hv7\" (UID: \"a21333e3-a759-4b18-83f5-1df25c72abb6\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-25hv7" Sep 17 11:58:26.983729 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:26.983702 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-25hv7" Sep 17 11:58:27.258355 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:27.258325 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-25hv7"] Sep 17 11:58:27.261920 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:58:27.261895 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda21333e3_a759_4b18_83f5_1df25c72abb6.slice/crio-cea6112b99dd5fce52b8f5f03256582245f61adb9cd57222640ad3390ab1edb1 WatchSource:0}: Error finding container cea6112b99dd5fce52b8f5f03256582245f61adb9cd57222640ad3390ab1edb1: Status 404 returned error can't find the container with id cea6112b99dd5fce52b8f5f03256582245f61adb9cd57222640ad3390ab1edb1 Sep 17 11:58:27.435368 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:27.435334 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-25hv7" event={"ID":"a21333e3-a759-4b18-83f5-1df25c72abb6","Type":"ContainerStarted","Data":"cea6112b99dd5fce52b8f5f03256582245f61adb9cd57222640ad3390ab1edb1"} Sep 17 11:58:29.449877 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:29.449834 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-25hv7" event={"ID":"a21333e3-a759-4b18-83f5-1df25c72abb6","Type":"ContainerStarted","Data":"f02984691ea4d69622ea100a55e4cef796ac73459f154bd1c7d032057a8cbde7"} Sep 17 11:58:29.450378 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:29.450026 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-25hv7" Sep 17 11:58:29.472524 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:29.472478 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-25hv7" podStartSLOduration=2.030703467 podStartE2EDuration="3.472461892s" totalImagesPullingTime="1.441758425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:58:29.470824284 +0000 UTC m=+1179.478271317" watchObservedRunningTime="2026-09-17 11:58:29.472461892 +0000 UTC m=+1179.479908922" Sep 17 11:58:40.456747 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:40.456715 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-25hv7" Sep 17 11:58:44.627348 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.627316 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k"] Sep 17 11:58:44.631398 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.631382 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:58:44.634654 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.634628 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 17 11:58:44.634654 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.634641 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 17 11:58:44.634818 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.634738 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-f228r\"" Sep 17 11:58:44.634872 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.634845 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 17 11:58:44.634872 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.634862 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 17 11:58:44.634948 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.634894 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 17 11:58:44.635095 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.635082 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 17 11:58:44.646839 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.646818 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k"] Sep 17 11:58:44.772358 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.772322 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tp56k\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:58:44.772553 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.772365 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tp56k\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:58:44.772553 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.772392 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtnrt\" (UniqueName: \"kubernetes.io/projected/88a8d4bb-9326-4dc2-8410-69406b04f082-kube-api-access-wtnrt\") pod \"feast-simple-feast-setup-57576b8fcf-tp56k\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:58:44.772553 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.772493 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tp56k\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:58:44.772553 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.772534 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/88a8d4bb-9326-4dc2-8410-69406b04f082-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-tp56k\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:58:44.772718 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.772618 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tp56k\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:58:44.873068 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.873033 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tp56k\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:58:44.873068 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.873072 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtnrt\" (UniqueName: \"kubernetes.io/projected/88a8d4bb-9326-4dc2-8410-69406b04f082-kube-api-access-wtnrt\") pod \"feast-simple-feast-setup-57576b8fcf-tp56k\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:58:44.873340 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.873095 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tp56k\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:58:44.873340 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.873134 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/88a8d4bb-9326-4dc2-8410-69406b04f082-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-tp56k\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:58:44.873340 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.873184 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tp56k\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:58:44.873340 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.873220 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tp56k\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:58:44.874079 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.873980 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/88a8d4bb-9326-4dc2-8410-69406b04f082-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-tp56k\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:58:44.875815 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.875788 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tp56k\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:58:44.875929 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.875824 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tp56k\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:58:44.876295 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.876266 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tp56k\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:58:44.876382 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.876292 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tp56k\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:58:44.882657 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.882593 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtnrt\" (UniqueName: \"kubernetes.io/projected/88a8d4bb-9326-4dc2-8410-69406b04f082-kube-api-access-wtnrt\") pod \"feast-simple-feast-setup-57576b8fcf-tp56k\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:58:44.940553 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:44.940513 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:58:45.221254 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:45.221229 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k"] Sep 17 11:58:45.223654 ip-10-0-140-196 kubenswrapper[2581]: W0917 11:58:45.223627 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88a8d4bb_9326_4dc2_8410_69406b04f082.slice/crio-da73ee042e90c3c80785237a165a9b1d2bb81a0a8710536c4950565f16b5a36e WatchSource:0}: Error finding container da73ee042e90c3c80785237a165a9b1d2bb81a0a8710536c4950565f16b5a36e: Status 404 returned error can't find the container with id da73ee042e90c3c80785237a165a9b1d2bb81a0a8710536c4950565f16b5a36e Sep 17 11:58:45.520207 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:45.520162 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" event={"ID":"88a8d4bb-9326-4dc2-8410-69406b04f082","Type":"ContainerStarted","Data":"da73ee042e90c3c80785237a165a9b1d2bb81a0a8710536c4950565f16b5a36e"} Sep 17 11:58:50.424387 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:50.424353 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/2.log" Sep 17 11:58:50.425363 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:50.425337 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/2.log" Sep 17 11:58:57.584982 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:58:57.584914 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" event={"ID":"88a8d4bb-9326-4dc2-8410-69406b04f082","Type":"ContainerStarted","Data":"4a786cbebf9c759ce90a46cf9b0d54c629c5aa070f911647c5b1ca41cf0fe391"} Sep 17 11:59:02.608062 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:02.607970 2581 generic.go:356] "Generic (PLEG): container finished" podID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerID="4a786cbebf9c759ce90a46cf9b0d54c629c5aa070f911647c5b1ca41cf0fe391" exitCode=0 Sep 17 11:59:02.608062 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:02.608020 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" event={"ID":"88a8d4bb-9326-4dc2-8410-69406b04f082","Type":"ContainerDied","Data":"4a786cbebf9c759ce90a46cf9b0d54c629c5aa070f911647c5b1ca41cf0fe391"} Sep 17 11:59:03.614525 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:03.614486 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" event={"ID":"88a8d4bb-9326-4dc2-8410-69406b04f082","Type":"ContainerStarted","Data":"b1b1ddc599fa9e464a3948f339da6a19e5dff77023c87339ba813409b4722196"} Sep 17 11:59:08.639008 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:08.638965 2581 generic.go:356] "Generic (PLEG): container finished" podID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerID="b1b1ddc599fa9e464a3948f339da6a19e5dff77023c87339ba813409b4722196" exitCode=0 Sep 17 11:59:08.639417 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:08.639040 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" event={"ID":"88a8d4bb-9326-4dc2-8410-69406b04f082","Type":"ContainerDied","Data":"b1b1ddc599fa9e464a3948f339da6a19e5dff77023c87339ba813409b4722196"} Sep 17 11:59:09.651055 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:09.651012 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" event={"ID":"88a8d4bb-9326-4dc2-8410-69406b04f082","Type":"ContainerStarted","Data":"fb20f3ebf45ec1fd8740630058ce86f8b2d4c3defe5fefafdfac50b5fdf1be37"} Sep 17 11:59:09.651722 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:09.651100 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" event={"ID":"88a8d4bb-9326-4dc2-8410-69406b04f082","Type":"ContainerStarted","Data":"8a757a0c75c193fbfec7f47c1c5dc011cc5e7f1998dd1ebcbce48c9898479bca"} Sep 17 11:59:09.651722 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:09.651148 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" event={"ID":"88a8d4bb-9326-4dc2-8410-69406b04f082","Type":"ContainerStarted","Data":"9f9afe00c97bdf52e490062af7781c8a1641f2a16cc172ade8f5121fa282df33"} Sep 17 11:59:09.651722 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:09.651173 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" event={"ID":"88a8d4bb-9326-4dc2-8410-69406b04f082","Type":"ContainerStarted","Data":"c192f25d20b58001ca346fe864ab29378d9e0eeee1a696ec3cb436bd0b518e03"} Sep 17 11:59:09.687312 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:09.687242 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" podStartSLOduration=13.532927325 podStartE2EDuration="25.687216448s" totalImagesPullingTime="12.154289123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:58:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:59:09.683289997 +0000 UTC m=+1219.690737028" watchObservedRunningTime="2026-09-17 11:59:09.687216448 +0000 UTC m=+1219.694663481" Sep 17 11:59:11.941090 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:11.941047 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:59:11.941738 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:11.941145 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:59:11.941738 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:11.941169 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:59:11.941738 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:11.941187 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:59:11.943568 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:11.943519 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.31:8443: connect: connection refused" Sep 17 11:59:11.943707 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:11.943558 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" Sep 17 11:59:11.943707 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:11.943659 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.31:8816: connect: connection refused" Sep 17 11:59:11.943856 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:11.943550 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.31:6571: connect: connection refused" Sep 17 11:59:14.941183 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:14.940774 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:59:14.941183 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:14.940972 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.31:8443: connect: connection refused" Sep 17 11:59:14.941183 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:14.941028 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.31:8816: connect: connection refused" Sep 17 11:59:14.941183 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:14.941036 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.31:6571: connect: connection refused" Sep 17 11:59:14.941872 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:14.941382 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:59:14.941872 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:14.941422 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:59:14.941872 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:14.941442 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:59:14.941872 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:14.941482 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" Sep 17 11:59:17.942571 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:17.942533 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:59:17.943065 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:17.942870 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:59:17.943065 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:17.943008 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:59:17.943333 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:17.943311 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:59:17.943417 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:17.943387 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:59:17.948167 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:17.948148 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:59:18.717295 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:18.717266 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 11:59:18.719984 ip-10-0-140-196 kubenswrapper[2581]: I0917 11:59:18.719965 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 12:00:21.805557 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:21.805525 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k"] Sep 17 12:00:21.806046 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:21.805756 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="registry" containerID="cri-o://c192f25d20b58001ca346fe864ab29378d9e0eeee1a696ec3cb436bd0b518e03" gracePeriod=30 Sep 17 12:00:21.806046 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:21.805816 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="ui" containerID="cri-o://fb20f3ebf45ec1fd8740630058ce86f8b2d4c3defe5fefafdfac50b5fdf1be37" gracePeriod=30 Sep 17 12:00:21.806046 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:21.805827 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="offline" containerID="cri-o://8a757a0c75c193fbfec7f47c1c5dc011cc5e7f1998dd1ebcbce48c9898479bca" gracePeriod=30 Sep 17 12:00:21.806046 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:21.805827 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="online" containerID="cri-o://9f9afe00c97bdf52e490062af7781c8a1641f2a16cc172ade8f5121fa282df33" gracePeriod=30 Sep 17 12:00:22.419394 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.419364 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98"] Sep 17 12:00:22.423338 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.423311 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:22.429761 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.429736 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-7bmw4\"" Sep 17 12:00:22.444673 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.444642 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98"] Sep 17 12:00:22.512961 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.512914 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7017c5c-f264-43e5-ae29-05b7cdf223c8-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-xnh98\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:22.513183 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.512986 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xnh98\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:22.513183 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.513032 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2pfl\" (UniqueName: \"kubernetes.io/projected/e7017c5c-f264-43e5-ae29-05b7cdf223c8-kube-api-access-x2pfl\") pod \"feast-simple-feast-setup-57576b8fcf-xnh98\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:22.513183 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.513087 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xnh98\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:22.513183 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.513173 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xnh98\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:22.513405 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.513198 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xnh98\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:22.614332 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.614297 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7017c5c-f264-43e5-ae29-05b7cdf223c8-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-xnh98\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:22.614471 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.614373 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xnh98\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:22.614471 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.614417 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2pfl\" (UniqueName: \"kubernetes.io/projected/e7017c5c-f264-43e5-ae29-05b7cdf223c8-kube-api-access-x2pfl\") pod \"feast-simple-feast-setup-57576b8fcf-xnh98\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:22.614589 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.614473 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xnh98\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:22.614589 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.614540 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xnh98\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:22.614589 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.614565 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xnh98\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:22.614747 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.614636 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7017c5c-f264-43e5-ae29-05b7cdf223c8-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-xnh98\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:22.614747 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:00:22.614682 2581 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 17 12:00:22.614747 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:00:22.614710 2581 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 17 12:00:22.614898 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:00:22.614760 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-ui-tls podName:e7017c5c-f264-43e5-ae29-05b7cdf223c8 nodeName:}" failed. No retries permitted until 2026-09-17 12:00:23.114739608 +0000 UTC m=+1293.122186638 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-ui-tls") pod "feast-simple-feast-setup-57576b8fcf-xnh98" (UID: "e7017c5c-f264-43e5-ae29-05b7cdf223c8") : secret "feast-simple-feast-setup-ui-tls" not found Sep 17 12:00:22.614898 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:00:22.614778 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-offline-tls podName:e7017c5c-f264-43e5-ae29-05b7cdf223c8 nodeName:}" failed. No retries permitted until 2026-09-17 12:00:23.114768497 +0000 UTC m=+1293.122215552 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-offline-tls") pod "feast-simple-feast-setup-57576b8fcf-xnh98" (UID: "e7017c5c-f264-43e5-ae29-05b7cdf223c8") : secret "feast-simple-feast-setup-offline-tls" not found Sep 17 12:00:22.616903 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.616880 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xnh98\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:22.617446 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.617423 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xnh98\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:22.633276 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.633255 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2pfl\" (UniqueName: \"kubernetes.io/projected/e7017c5c-f264-43e5-ae29-05b7cdf223c8-kube-api-access-x2pfl\") pod \"feast-simple-feast-setup-57576b8fcf-xnh98\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:22.998856 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.998823 2581 generic.go:356] "Generic (PLEG): container finished" podID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerID="fb20f3ebf45ec1fd8740630058ce86f8b2d4c3defe5fefafdfac50b5fdf1be37" exitCode=0 Sep 17 12:00:22.998856 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.998846 2581 generic.go:356] "Generic (PLEG): container finished" podID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerID="9f9afe00c97bdf52e490062af7781c8a1641f2a16cc172ade8f5121fa282df33" exitCode=0 Sep 17 12:00:22.999267 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.998892 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" event={"ID":"88a8d4bb-9326-4dc2-8410-69406b04f082","Type":"ContainerDied","Data":"fb20f3ebf45ec1fd8740630058ce86f8b2d4c3defe5fefafdfac50b5fdf1be37"} Sep 17 12:00:22.999267 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:22.998932 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" event={"ID":"88a8d4bb-9326-4dc2-8410-69406b04f082","Type":"ContainerDied","Data":"9f9afe00c97bdf52e490062af7781c8a1641f2a16cc172ade8f5121fa282df33"} Sep 17 12:00:23.119077 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:23.119042 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xnh98\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:23.119077 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:23.119080 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xnh98\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:23.121616 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:23.121592 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xnh98\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:23.121737 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:23.121602 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xnh98\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:23.335783 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:23.335697 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:23.619150 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:23.619107 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98"] Sep 17 12:00:23.620522 ip-10-0-140-196 kubenswrapper[2581]: W0917 12:00:23.620491 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7017c5c_f264_43e5_ae29_05b7cdf223c8.slice/crio-535a703d9afe188ab380ccc8ef928768f950ca8e1c24683b38cfba6efd596e68 WatchSource:0}: Error finding container 535a703d9afe188ab380ccc8ef928768f950ca8e1c24683b38cfba6efd596e68: Status 404 returned error can't find the container with id 535a703d9afe188ab380ccc8ef928768f950ca8e1c24683b38cfba6efd596e68 Sep 17 12:00:24.004623 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:24.004580 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" event={"ID":"e7017c5c-f264-43e5-ae29-05b7cdf223c8","Type":"ContainerStarted","Data":"2b3cd3862f20547ef0fc58e98c253105cda5ef32bd82b9bd220222ccb363a001"} Sep 17 12:00:24.005006 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:24.004633 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" event={"ID":"e7017c5c-f264-43e5-ae29-05b7cdf223c8","Type":"ContainerStarted","Data":"535a703d9afe188ab380ccc8ef928768f950ca8e1c24683b38cfba6efd596e68"} Sep 17 12:00:27.943537 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:27.943490 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.31:8443: connect: connection refused" Sep 17 12:00:28.717172 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:28.717098 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" Sep 17 12:00:29.027376 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:29.027337 2581 generic.go:356] "Generic (PLEG): container finished" podID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerID="2b3cd3862f20547ef0fc58e98c253105cda5ef32bd82b9bd220222ccb363a001" exitCode=0 Sep 17 12:00:29.027799 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:29.027393 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" event={"ID":"e7017c5c-f264-43e5-ae29-05b7cdf223c8","Type":"ContainerDied","Data":"2b3cd3862f20547ef0fc58e98c253105cda5ef32bd82b9bd220222ccb363a001"} Sep 17 12:00:30.034742 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:30.034708 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" event={"ID":"e7017c5c-f264-43e5-ae29-05b7cdf223c8","Type":"ContainerStarted","Data":"f36543bd3734fbb489c6e2879ce3b94126c3a3ed557da9f98cd5d482b76de9f5"} Sep 17 12:00:35.061151 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:35.061095 2581 generic.go:356] "Generic (PLEG): container finished" podID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerID="f36543bd3734fbb489c6e2879ce3b94126c3a3ed557da9f98cd5d482b76de9f5" exitCode=0 Sep 17 12:00:35.061551 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:35.061182 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" event={"ID":"e7017c5c-f264-43e5-ae29-05b7cdf223c8","Type":"ContainerDied","Data":"f36543bd3734fbb489c6e2879ce3b94126c3a3ed557da9f98cd5d482b76de9f5"} Sep 17 12:00:36.072377 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:36.072336 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" event={"ID":"e7017c5c-f264-43e5-ae29-05b7cdf223c8","Type":"ContainerStarted","Data":"c03d4997f7d61e25721256a61acc590787d377d70f97b2666060b8f2a3fe519f"} Sep 17 12:00:36.072886 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:36.072400 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" event={"ID":"e7017c5c-f264-43e5-ae29-05b7cdf223c8","Type":"ContainerStarted","Data":"c00d842c8047cca91a4ffda1285344b934195947dbe88a5c60bbce4e17b6eecd"} Sep 17 12:00:36.072886 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:36.072423 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" event={"ID":"e7017c5c-f264-43e5-ae29-05b7cdf223c8","Type":"ContainerStarted","Data":"c1d484da68803bb045faeae86330abb38c51cb81a331ea9f85aa677704c3f3d1"} Sep 17 12:00:36.072886 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:36.072440 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" event={"ID":"e7017c5c-f264-43e5-ae29-05b7cdf223c8","Type":"ContainerStarted","Data":"0018acb0d9351c5af3a8fecfc4bb0995ea325904c216e8e7173e6187453df300"} Sep 17 12:00:36.113953 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:36.113888 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" podStartSLOduration=14.113866054 podStartE2EDuration="14.113866054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 12:00:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 12:00:36.11275716 +0000 UTC m=+1306.120204194" watchObservedRunningTime="2026-09-17 12:00:36.113866054 +0000 UTC m=+1306.121313088" Sep 17 12:00:37.943331 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:37.943286 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.31:8443: connect: connection refused" Sep 17 12:00:38.336001 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:38.335961 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:38.336329 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:38.336306 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:38.336473 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:38.336338 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:38.336473 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:38.336357 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:38.338482 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:38.338450 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.32:8443: connect: connection refused" Sep 17 12:00:38.338704 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:38.338476 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.32:8816: connect: connection refused" Sep 17 12:00:38.338790 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:38.338498 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused" Sep 17 12:00:38.338790 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:38.338578 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Sep 17 12:00:38.717637 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:38.717492 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" Sep 17 12:00:41.336851 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:41.336799 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.32:8443: connect: connection refused" Sep 17 12:00:41.336851 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:41.336799 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.32:8816: connect: connection refused" Sep 17 12:00:41.337611 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:41.336821 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Sep 17 12:00:41.337611 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:41.337034 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused" Sep 17 12:00:43.335883 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:43.335842 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:43.336400 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:43.335927 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:43.336400 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:43.335953 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:43.336400 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:43.335988 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:44.337052 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:44.337025 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:44.337495 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:44.337275 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:44.337495 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:44.337469 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:44.342058 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:44.342038 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:45.128913 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:45.128879 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:45.129136 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:45.129046 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:45.129136 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:45.129103 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:45.131604 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:45.131583 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:00:47.943686 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:47.943569 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.31:8443: connect: connection refused" Sep 17 12:00:47.949787 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:47.943724 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 12:00:48.717027 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:48.716986 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" Sep 17 12:00:48.717254 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:48.717143 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 12:00:52.162504 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.162417 2581 generic.go:356] "Generic (PLEG): container finished" podID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerID="8a757a0c75c193fbfec7f47c1c5dc011cc5e7f1998dd1ebcbce48c9898479bca" exitCode=137 Sep 17 12:00:52.162504 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.162440 2581 generic.go:356] "Generic (PLEG): container finished" podID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerID="c192f25d20b58001ca346fe864ab29378d9e0eeee1a696ec3cb436bd0b518e03" exitCode=137 Sep 17 12:00:52.162504 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.162486 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" event={"ID":"88a8d4bb-9326-4dc2-8410-69406b04f082","Type":"ContainerDied","Data":"8a757a0c75c193fbfec7f47c1c5dc011cc5e7f1998dd1ebcbce48c9898479bca"} Sep 17 12:00:52.163012 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.162524 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" event={"ID":"88a8d4bb-9326-4dc2-8410-69406b04f082","Type":"ContainerDied","Data":"c192f25d20b58001ca346fe864ab29378d9e0eeee1a696ec3cb436bd0b518e03"} Sep 17 12:00:52.457418 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.457390 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 12:00:52.476925 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.476902 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-ui-tls\") pod \"88a8d4bb-9326-4dc2-8410-69406b04f082\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " Sep 17 12:00:52.477048 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.476974 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-online-tls\") pod \"88a8d4bb-9326-4dc2-8410-69406b04f082\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " Sep 17 12:00:52.477048 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.477007 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-offline-tls\") pod \"88a8d4bb-9326-4dc2-8410-69406b04f082\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " Sep 17 12:00:52.477142 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.477071 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtnrt\" (UniqueName: \"kubernetes.io/projected/88a8d4bb-9326-4dc2-8410-69406b04f082-kube-api-access-wtnrt\") pod \"88a8d4bb-9326-4dc2-8410-69406b04f082\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " Sep 17 12:00:52.477142 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.477137 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/88a8d4bb-9326-4dc2-8410-69406b04f082-feast-data\") pod \"88a8d4bb-9326-4dc2-8410-69406b04f082\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " Sep 17 12:00:52.477243 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.477163 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-registry-tls\") pod \"88a8d4bb-9326-4dc2-8410-69406b04f082\" (UID: \"88a8d4bb-9326-4dc2-8410-69406b04f082\") " Sep 17 12:00:52.477747 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.477714 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a8d4bb-9326-4dc2-8410-69406b04f082-feast-data" (OuterVolumeSpecName: "feast-data") pod "88a8d4bb-9326-4dc2-8410-69406b04f082" (UID: "88a8d4bb-9326-4dc2-8410-69406b04f082"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 12:00:52.479725 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.479684 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "88a8d4bb-9326-4dc2-8410-69406b04f082" (UID: "88a8d4bb-9326-4dc2-8410-69406b04f082"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:00:52.479725 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.479701 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-online-tls" (OuterVolumeSpecName: "online-tls") pod "88a8d4bb-9326-4dc2-8410-69406b04f082" (UID: "88a8d4bb-9326-4dc2-8410-69406b04f082"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:00:52.479876 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.479762 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "88a8d4bb-9326-4dc2-8410-69406b04f082" (UID: "88a8d4bb-9326-4dc2-8410-69406b04f082"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:00:52.479959 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.479940 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "88a8d4bb-9326-4dc2-8410-69406b04f082" (UID: "88a8d4bb-9326-4dc2-8410-69406b04f082"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:00:52.480001 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.479959 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88a8d4bb-9326-4dc2-8410-69406b04f082-kube-api-access-wtnrt" (OuterVolumeSpecName: "kube-api-access-wtnrt") pod "88a8d4bb-9326-4dc2-8410-69406b04f082" (UID: "88a8d4bb-9326-4dc2-8410-69406b04f082"). InnerVolumeSpecName "kube-api-access-wtnrt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 12:00:52.578569 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.578537 2581 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-ui-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:00:52.578569 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.578562 2581 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-online-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:00:52.578569 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.578574 2581 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-offline-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:00:52.578857 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.578582 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtnrt\" (UniqueName: \"kubernetes.io/projected/88a8d4bb-9326-4dc2-8410-69406b04f082-kube-api-access-wtnrt\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:00:52.578857 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.578591 2581 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/88a8d4bb-9326-4dc2-8410-69406b04f082-feast-data\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:00:52.578857 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:52.578599 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/88a8d4bb-9326-4dc2-8410-69406b04f082-registry-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:00:53.169169 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:53.169144 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" Sep 17 12:00:53.169605 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:53.169139 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k" event={"ID":"88a8d4bb-9326-4dc2-8410-69406b04f082","Type":"ContainerDied","Data":"da73ee042e90c3c80785237a165a9b1d2bb81a0a8710536c4950565f16b5a36e"} Sep 17 12:00:53.169605 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:53.169266 2581 scope.go:117] "RemoveContainer" containerID="fb20f3ebf45ec1fd8740630058ce86f8b2d4c3defe5fefafdfac50b5fdf1be37" Sep 17 12:00:53.178983 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:53.178966 2581 scope.go:117] "RemoveContainer" containerID="8a757a0c75c193fbfec7f47c1c5dc011cc5e7f1998dd1ebcbce48c9898479bca" Sep 17 12:00:53.186876 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:53.186860 2581 scope.go:117] "RemoveContainer" containerID="9f9afe00c97bdf52e490062af7781c8a1641f2a16cc172ade8f5121fa282df33" Sep 17 12:00:53.194820 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:53.194804 2581 scope.go:117] "RemoveContainer" containerID="c192f25d20b58001ca346fe864ab29378d9e0eeee1a696ec3cb436bd0b518e03" Sep 17 12:00:53.205948 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:53.205793 2581 scope.go:117] "RemoveContainer" containerID="b1b1ddc599fa9e464a3948f339da6a19e5dff77023c87339ba813409b4722196" Sep 17 12:00:53.208508 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:53.208488 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k"] Sep 17 12:00:53.216154 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:53.216133 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-tp56k"] Sep 17 12:00:53.216480 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:53.216468 2581 scope.go:117] "RemoveContainer" containerID="4a786cbebf9c759ce90a46cf9b0d54c629c5aa070f911647c5b1ca41cf0fe391" Sep 17 12:00:54.428288 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:00:54.428255 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" path="/var/lib/kubelet/pods/88a8d4bb-9326-4dc2-8410-69406b04f082/volumes" Sep 17 12:01:47.363390 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.363358 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78"] Sep 17 12:01:47.363915 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.363688 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="feast-init" Sep 17 12:01:47.363915 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.363699 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="feast-init" Sep 17 12:01:47.363915 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.363707 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="online" Sep 17 12:01:47.363915 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.363714 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="online" Sep 17 12:01:47.363915 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.363723 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="feast-apply" Sep 17 12:01:47.363915 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.363728 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="feast-apply" Sep 17 12:01:47.363915 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.363735 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="ui" Sep 17 12:01:47.363915 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.363740 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="ui" Sep 17 12:01:47.363915 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.363755 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="registry" Sep 17 12:01:47.363915 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.363760 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="registry" Sep 17 12:01:47.363915 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.363769 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="offline" Sep 17 12:01:47.363915 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.363773 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="offline" Sep 17 12:01:47.363915 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.363832 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="online" Sep 17 12:01:47.363915 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.363841 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="registry" Sep 17 12:01:47.363915 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.363846 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="ui" Sep 17 12:01:47.363915 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.363854 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="88a8d4bb-9326-4dc2-8410-69406b04f082" containerName="offline" Sep 17 12:01:47.367625 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.367607 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:01:47.371457 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.371418 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 17 12:01:47.371658 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.371628 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 17 12:01:47.371658 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.371636 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 17 12:01:47.371658 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.371649 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 17 12:01:47.371981 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.371965 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 17 12:01:47.372038 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.371965 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 17 12:01:47.372768 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.372751 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-jp45l\"" Sep 17 12:01:47.388779 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.388755 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78"] Sep 17 12:01:47.447045 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.447010 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jqz78\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:01:47.447224 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.447057 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhpf7\" (UniqueName: \"kubernetes.io/projected/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-kube-api-access-fhpf7\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jqz78\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:01:47.447224 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.447132 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jqz78\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:01:47.447224 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.447166 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jqz78\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:01:47.447224 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.447205 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jqz78\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:01:47.447366 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.447233 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jqz78\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:01:47.548482 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.548453 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jqz78\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:01:47.548645 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.548496 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhpf7\" (UniqueName: \"kubernetes.io/projected/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-kube-api-access-fhpf7\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jqz78\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:01:47.548645 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.548520 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jqz78\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:01:47.548645 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.548542 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jqz78\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:01:47.548645 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.548583 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jqz78\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:01:47.548645 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.548606 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jqz78\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:01:47.549161 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.549105 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jqz78\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:01:47.549429 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.549334 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jqz78\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:01:47.551132 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.551091 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jqz78\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:01:47.551231 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.551215 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jqz78\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:01:47.551282 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.551254 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jqz78\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:01:47.558303 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.558275 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhpf7\" (UniqueName: \"kubernetes.io/projected/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-kube-api-access-fhpf7\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-jqz78\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:01:47.677695 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.677614 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:01:47.964895 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:47.964869 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78"] Sep 17 12:01:47.967461 ip-10-0-140-196 kubenswrapper[2581]: W0917 12:01:47.967431 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d898cc6_2ba3_46fe_b336_ff3e0420bd64.slice/crio-d86d04fbe2197e3b4f39707d0818ac12278347e63ef21506182d813e93947575 WatchSource:0}: Error finding container d86d04fbe2197e3b4f39707d0818ac12278347e63ef21506182d813e93947575: Status 404 returned error can't find the container with id d86d04fbe2197e3b4f39707d0818ac12278347e63ef21506182d813e93947575 Sep 17 12:01:48.415167 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:48.415102 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" event={"ID":"8d898cc6-2ba3-46fe-b336-ff3e0420bd64","Type":"ContainerStarted","Data":"ba3d55ee01d71c636bcdb147969ddd3e3797bf0058856e005e5721e746518da9"} Sep 17 12:01:48.415167 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:48.415172 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" event={"ID":"8d898cc6-2ba3-46fe-b336-ff3e0420bd64","Type":"ContainerStarted","Data":"d86d04fbe2197e3b4f39707d0818ac12278347e63ef21506182d813e93947575"} Sep 17 12:01:53.440365 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:53.440328 2581 generic.go:356] "Generic (PLEG): container finished" podID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerID="ba3d55ee01d71c636bcdb147969ddd3e3797bf0058856e005e5721e746518da9" exitCode=0 Sep 17 12:01:53.441020 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:53.440407 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" event={"ID":"8d898cc6-2ba3-46fe-b336-ff3e0420bd64","Type":"ContainerDied","Data":"ba3d55ee01d71c636bcdb147969ddd3e3797bf0058856e005e5721e746518da9"} Sep 17 12:01:54.446763 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:01:54.446725 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" event={"ID":"8d898cc6-2ba3-46fe-b336-ff3e0420bd64","Type":"ContainerStarted","Data":"22294cb715b3079de80553ebe3dc9484b92b1e698df9310f571dfe3f825ddfea"} Sep 17 12:02:22.575792 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:22.575698 2581 generic.go:356] "Generic (PLEG): container finished" podID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerID="22294cb715b3079de80553ebe3dc9484b92b1e698df9310f571dfe3f825ddfea" exitCode=0 Sep 17 12:02:22.575792 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:22.575739 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" event={"ID":"8d898cc6-2ba3-46fe-b336-ff3e0420bd64","Type":"ContainerDied","Data":"22294cb715b3079de80553ebe3dc9484b92b1e698df9310f571dfe3f825ddfea"} Sep 17 12:02:23.583737 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:23.583696 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" event={"ID":"8d898cc6-2ba3-46fe-b336-ff3e0420bd64","Type":"ContainerStarted","Data":"2ec806fb85314c0760bdc88ef8e98d463b423808b6d718e52eb1acb4a3be18d5"} Sep 17 12:02:23.584147 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:23.583762 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" event={"ID":"8d898cc6-2ba3-46fe-b336-ff3e0420bd64","Type":"ContainerStarted","Data":"e1f4b425e08eb6efef417dfb4418180063cf5e262b2e1fda7c0e83b3e08bd498"} Sep 17 12:02:23.584147 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:23.583783 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" event={"ID":"8d898cc6-2ba3-46fe-b336-ff3e0420bd64","Type":"ContainerStarted","Data":"10d529d62303fe2f85875639e84e176a3dd6fa9284009413cfbc4daef155dd36"} Sep 17 12:02:23.615561 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:23.615469 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podStartSLOduration=36.61544571 podStartE2EDuration="36.61544571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 12:01:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 12:02:23.615060413 +0000 UTC m=+1413.622507467" watchObservedRunningTime="2026-09-17 12:02:23.61544571 +0000 UTC m=+1413.622892742" Sep 17 12:02:23.678073 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:23.677959 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:02:23.678073 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:23.678033 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:02:23.678073 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:23.678053 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:02:23.680028 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:23.679992 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.33:8443: connect: connection refused" Sep 17 12:02:23.680028 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:23.680016 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" Sep 17 12:02:23.680222 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:23.680000 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.33:8816: connect: connection refused" Sep 17 12:02:26.678308 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:26.678264 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.33:8816: connect: connection refused" Sep 17 12:02:26.678703 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:26.678263 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.33:8443: connect: connection refused" Sep 17 12:02:26.678703 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:26.678274 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" Sep 17 12:02:27.677993 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:27.677945 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:02:27.678193 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:27.678025 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:02:27.678193 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:27.678070 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:02:29.677981 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:29.677936 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.33:8816: connect: connection refused" Sep 17 12:02:29.677981 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:29.677948 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.33:8443: connect: connection refused" Sep 17 12:02:29.677981 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:29.677976 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" Sep 17 12:02:32.678561 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:32.678523 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:02:32.678972 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:32.678781 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:02:32.679290 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:32.679272 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:02:33.635274 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:33.635251 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:02:33.682240 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:33.682207 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 17 12:02:36.681247 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:36.681204 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 17 12:02:39.622861 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:39.622828 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:02:40.578155 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:40.578104 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:02:48.684293 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:02:48.684213 2581 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 17 12:03:08.683281 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:08.683230 2581 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 17 12:03:10.628256 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:10.628220 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 17 12:03:40.629254 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:40.629212 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 17 12:03:41.865329 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:41.865298 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78"] Sep 17 12:03:41.865725 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:41.865537 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="online" containerID="cri-o://10d529d62303fe2f85875639e84e176a3dd6fa9284009413cfbc4daef155dd36" gracePeriod=30 Sep 17 12:03:41.865725 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:41.865590 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="ui" containerID="cri-o://2ec806fb85314c0760bdc88ef8e98d463b423808b6d718e52eb1acb4a3be18d5" gracePeriod=30 Sep 17 12:03:41.865725 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:41.865622 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="offline" containerID="cri-o://e1f4b425e08eb6efef417dfb4418180063cf5e262b2e1fda7c0e83b3e08bd498" gracePeriod=30 Sep 17 12:03:42.088744 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:42.088712 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98"] Sep 17 12:03:42.089022 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:42.088971 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="registry" containerID="cri-o://0018acb0d9351c5af3a8fecfc4bb0995ea325904c216e8e7173e6187453df300" gracePeriod=30 Sep 17 12:03:42.089022 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:42.089005 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="offline" containerID="cri-o://c00d842c8047cca91a4ffda1285344b934195947dbe88a5c60bbce4e17b6eecd" gracePeriod=30 Sep 17 12:03:42.089241 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:42.089010 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="online" containerID="cri-o://c1d484da68803bb045faeae86330abb38c51cb81a331ea9f85aa677704c3f3d1" gracePeriod=30 Sep 17 12:03:42.089241 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:42.089032 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="ui" containerID="cri-o://c03d4997f7d61e25721256a61acc590787d377d70f97b2666060b8f2a3fe519f" gracePeriod=30 Sep 17 12:03:42.679619 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:42.679529 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.33:8443: connect: connection refused" Sep 17 12:03:42.946240 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:42.946177 2581 generic.go:356] "Generic (PLEG): container finished" podID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerID="c03d4997f7d61e25721256a61acc590787d377d70f97b2666060b8f2a3fe519f" exitCode=0 Sep 17 12:03:42.946240 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:42.946199 2581 generic.go:356] "Generic (PLEG): container finished" podID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerID="c1d484da68803bb045faeae86330abb38c51cb81a331ea9f85aa677704c3f3d1" exitCode=0 Sep 17 12:03:42.946581 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:42.946255 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" event={"ID":"e7017c5c-f264-43e5-ae29-05b7cdf223c8","Type":"ContainerDied","Data":"c03d4997f7d61e25721256a61acc590787d377d70f97b2666060b8f2a3fe519f"} Sep 17 12:03:42.946581 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:42.946295 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" event={"ID":"e7017c5c-f264-43e5-ae29-05b7cdf223c8","Type":"ContainerDied","Data":"c1d484da68803bb045faeae86330abb38c51cb81a331ea9f85aa677704c3f3d1"} Sep 17 12:03:42.949035 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:42.949016 2581 generic.go:356] "Generic (PLEG): container finished" podID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerID="2ec806fb85314c0760bdc88ef8e98d463b423808b6d718e52eb1acb4a3be18d5" exitCode=0 Sep 17 12:03:42.949035 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:42.949032 2581 generic.go:356] "Generic (PLEG): container finished" podID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerID="10d529d62303fe2f85875639e84e176a3dd6fa9284009413cfbc4daef155dd36" exitCode=0 Sep 17 12:03:42.949196 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:42.949071 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" event={"ID":"8d898cc6-2ba3-46fe-b336-ff3e0420bd64","Type":"ContainerDied","Data":"2ec806fb85314c0760bdc88ef8e98d463b423808b6d718e52eb1acb4a3be18d5"} Sep 17 12:03:42.949196 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:42.949103 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" event={"ID":"8d898cc6-2ba3-46fe-b336-ff3e0420bd64","Type":"ContainerDied","Data":"10d529d62303fe2f85875639e84e176a3dd6fa9284009413cfbc4daef155dd36"} Sep 17 12:03:45.128694 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:45.128621 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.32:8443: connect: connection refused" Sep 17 12:03:45.129161 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:45.128707 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Sep 17 12:03:49.623872 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:49.623837 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" Sep 17 12:03:50.459569 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:50.459543 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/2.log" Sep 17 12:03:50.461994 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:50.461975 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/2.log" Sep 17 12:03:52.678908 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:52.678874 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.33:8443: connect: connection refused" Sep 17 12:03:55.128658 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:55.128615 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.32:8443: connect: connection refused" Sep 17 12:03:55.129062 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:55.128704 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Sep 17 12:03:59.623785 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:59.623749 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" Sep 17 12:03:59.624248 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:03:59.623884 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:04:02.679134 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:02.679092 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.33:8443: connect: connection refused" Sep 17 12:04:02.679547 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:02.679250 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:04:05.129515 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:05.129478 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.32:8443: connect: connection refused" Sep 17 12:04:05.129926 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:05.129519 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Sep 17 12:04:05.129926 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:05.129601 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:04:05.129926 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:05.129804 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:04:09.624410 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:09.624374 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" Sep 17 12:04:12.073892 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.073865 2581 generic.go:356] "Generic (PLEG): container finished" podID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerID="e1f4b425e08eb6efef417dfb4418180063cf5e262b2e1fda7c0e83b3e08bd498" exitCode=137 Sep 17 12:04:12.074259 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.073899 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" event={"ID":"8d898cc6-2ba3-46fe-b336-ff3e0420bd64","Type":"ContainerDied","Data":"e1f4b425e08eb6efef417dfb4418180063cf5e262b2e1fda7c0e83b3e08bd498"} Sep 17 12:04:12.515077 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.515052 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:04:12.616702 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.616680 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-registry-tls\") pod \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " Sep 17 12:04:12.616813 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.616733 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-offline-tls\") pod \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " Sep 17 12:04:12.616813 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.616779 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-online-tls\") pod \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " Sep 17 12:04:12.616897 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.616827 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-ui-tls\") pod \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " Sep 17 12:04:12.616897 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.616874 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-feast-data\") pod \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " Sep 17 12:04:12.616967 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.616931 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhpf7\" (UniqueName: \"kubernetes.io/projected/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-kube-api-access-fhpf7\") pod \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\" (UID: \"8d898cc6-2ba3-46fe-b336-ff3e0420bd64\") " Sep 17 12:04:12.617056 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.617030 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8d898cc6-2ba3-46fe-b336-ff3e0420bd64" (UID: "8d898cc6-2ba3-46fe-b336-ff3e0420bd64"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 12:04:12.617221 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.617199 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-registry-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:04:12.617549 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.617520 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-feast-data" (OuterVolumeSpecName: "feast-data") pod "8d898cc6-2ba3-46fe-b336-ff3e0420bd64" (UID: "8d898cc6-2ba3-46fe-b336-ff3e0420bd64"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 12:04:12.619702 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.619674 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "8d898cc6-2ba3-46fe-b336-ff3e0420bd64" (UID: "8d898cc6-2ba3-46fe-b336-ff3e0420bd64"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:04:12.619702 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.619690 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-online-tls" (OuterVolumeSpecName: "online-tls") pod "8d898cc6-2ba3-46fe-b336-ff3e0420bd64" (UID: "8d898cc6-2ba3-46fe-b336-ff3e0420bd64"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:04:12.619848 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.619723 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-kube-api-access-fhpf7" (OuterVolumeSpecName: "kube-api-access-fhpf7") pod "8d898cc6-2ba3-46fe-b336-ff3e0420bd64" (UID: "8d898cc6-2ba3-46fe-b336-ff3e0420bd64"). InnerVolumeSpecName "kube-api-access-fhpf7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 12:04:12.619848 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.619775 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "8d898cc6-2ba3-46fe-b336-ff3e0420bd64" (UID: "8d898cc6-2ba3-46fe-b336-ff3e0420bd64"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:04:12.717824 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.717798 2581 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-feast-data\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:04:12.717906 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.717833 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhpf7\" (UniqueName: \"kubernetes.io/projected/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-kube-api-access-fhpf7\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:04:12.717906 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.717848 2581 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-offline-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:04:12.717906 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.717861 2581 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-online-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:04:12.717906 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.717873 2581 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8d898cc6-2ba3-46fe-b336-ff3e0420bd64-ui-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:04:12.727656 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.727637 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:04:12.818310 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.818286 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2pfl\" (UniqueName: \"kubernetes.io/projected/e7017c5c-f264-43e5-ae29-05b7cdf223c8-kube-api-access-x2pfl\") pod \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " Sep 17 12:04:12.818426 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.818335 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-online-tls\") pod \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " Sep 17 12:04:12.818426 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.818406 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-ui-tls\") pod \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " Sep 17 12:04:12.818533 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.818426 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-registry-tls\") pod \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " Sep 17 12:04:12.818533 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.818467 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7017c5c-f264-43e5-ae29-05b7cdf223c8-feast-data\") pod \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " Sep 17 12:04:12.818533 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.818490 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-offline-tls\") pod \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\" (UID: \"e7017c5c-f264-43e5-ae29-05b7cdf223c8\") " Sep 17 12:04:12.819260 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.819214 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7017c5c-f264-43e5-ae29-05b7cdf223c8-feast-data" (OuterVolumeSpecName: "feast-data") pod "e7017c5c-f264-43e5-ae29-05b7cdf223c8" (UID: "e7017c5c-f264-43e5-ae29-05b7cdf223c8"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 12:04:12.820622 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.820588 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-online-tls" (OuterVolumeSpecName: "online-tls") pod "e7017c5c-f264-43e5-ae29-05b7cdf223c8" (UID: "e7017c5c-f264-43e5-ae29-05b7cdf223c8"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:04:12.820708 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.820630 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7017c5c-f264-43e5-ae29-05b7cdf223c8-kube-api-access-x2pfl" (OuterVolumeSpecName: "kube-api-access-x2pfl") pod "e7017c5c-f264-43e5-ae29-05b7cdf223c8" (UID: "e7017c5c-f264-43e5-ae29-05b7cdf223c8"). InnerVolumeSpecName "kube-api-access-x2pfl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 12:04:12.820773 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.820706 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e7017c5c-f264-43e5-ae29-05b7cdf223c8" (UID: "e7017c5c-f264-43e5-ae29-05b7cdf223c8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:04:12.821027 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.821012 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "e7017c5c-f264-43e5-ae29-05b7cdf223c8" (UID: "e7017c5c-f264-43e5-ae29-05b7cdf223c8"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:04:12.821085 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.821029 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "e7017c5c-f264-43e5-ae29-05b7cdf223c8" (UID: "e7017c5c-f264-43e5-ae29-05b7cdf223c8"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:04:12.919231 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.919187 2581 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7017c5c-f264-43e5-ae29-05b7cdf223c8-feast-data\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:04:12.919231 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.919204 2581 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-offline-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:04:12.919231 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.919213 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2pfl\" (UniqueName: \"kubernetes.io/projected/e7017c5c-f264-43e5-ae29-05b7cdf223c8-kube-api-access-x2pfl\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:04:12.919231 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.919221 2581 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-online-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:04:12.919231 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.919228 2581 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-ui-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:04:12.919421 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:12.919236 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7017c5c-f264-43e5-ae29-05b7cdf223c8-registry-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:04:13.081210 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.081190 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" Sep 17 12:04:13.081210 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.081191 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78" event={"ID":"8d898cc6-2ba3-46fe-b336-ff3e0420bd64","Type":"ContainerDied","Data":"d86d04fbe2197e3b4f39707d0818ac12278347e63ef21506182d813e93947575"} Sep 17 12:04:13.081566 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.081253 2581 scope.go:117] "RemoveContainer" containerID="2ec806fb85314c0760bdc88ef8e98d463b423808b6d718e52eb1acb4a3be18d5" Sep 17 12:04:13.084216 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.084192 2581 generic.go:356] "Generic (PLEG): container finished" podID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerID="c00d842c8047cca91a4ffda1285344b934195947dbe88a5c60bbce4e17b6eecd" exitCode=137 Sep 17 12:04:13.084216 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.084215 2581 generic.go:356] "Generic (PLEG): container finished" podID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerID="0018acb0d9351c5af3a8fecfc4bb0995ea325904c216e8e7173e6187453df300" exitCode=137 Sep 17 12:04:13.084359 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.084232 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" event={"ID":"e7017c5c-f264-43e5-ae29-05b7cdf223c8","Type":"ContainerDied","Data":"c00d842c8047cca91a4ffda1285344b934195947dbe88a5c60bbce4e17b6eecd"} Sep 17 12:04:13.084359 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.084250 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" Sep 17 12:04:13.084359 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.084260 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" event={"ID":"e7017c5c-f264-43e5-ae29-05b7cdf223c8","Type":"ContainerDied","Data":"0018acb0d9351c5af3a8fecfc4bb0995ea325904c216e8e7173e6187453df300"} Sep 17 12:04:13.084359 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.084276 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98" event={"ID":"e7017c5c-f264-43e5-ae29-05b7cdf223c8","Type":"ContainerDied","Data":"535a703d9afe188ab380ccc8ef928768f950ca8e1c24683b38cfba6efd596e68"} Sep 17 12:04:13.092594 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.092575 2581 scope.go:117] "RemoveContainer" containerID="e1f4b425e08eb6efef417dfb4418180063cf5e262b2e1fda7c0e83b3e08bd498" Sep 17 12:04:13.099941 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.099925 2581 scope.go:117] "RemoveContainer" containerID="10d529d62303fe2f85875639e84e176a3dd6fa9284009413cfbc4daef155dd36" Sep 17 12:04:13.106852 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.106835 2581 scope.go:117] "RemoveContainer" containerID="22294cb715b3079de80553ebe3dc9484b92b1e698df9310f571dfe3f825ddfea" Sep 17 12:04:13.113848 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.113834 2581 scope.go:117] "RemoveContainer" containerID="ba3d55ee01d71c636bcdb147969ddd3e3797bf0058856e005e5721e746518da9" Sep 17 12:04:13.121129 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.121101 2581 scope.go:117] "RemoveContainer" containerID="c03d4997f7d61e25721256a61acc590787d377d70f97b2666060b8f2a3fe519f" Sep 17 12:04:13.128276 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.128259 2581 scope.go:117] "RemoveContainer" containerID="c00d842c8047cca91a4ffda1285344b934195947dbe88a5c60bbce4e17b6eecd" Sep 17 12:04:13.135702 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.135688 2581 scope.go:117] "RemoveContainer" containerID="c1d484da68803bb045faeae86330abb38c51cb81a331ea9f85aa677704c3f3d1" Sep 17 12:04:13.142783 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.142768 2581 scope.go:117] "RemoveContainer" containerID="0018acb0d9351c5af3a8fecfc4bb0995ea325904c216e8e7173e6187453df300" Sep 17 12:04:13.149851 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.149833 2581 scope.go:117] "RemoveContainer" containerID="f36543bd3734fbb489c6e2879ce3b94126c3a3ed557da9f98cd5d482b76de9f5" Sep 17 12:04:13.157149 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.156908 2581 scope.go:117] "RemoveContainer" containerID="2b3cd3862f20547ef0fc58e98c253105cda5ef32bd82b9bd220222ccb363a001" Sep 17 12:04:13.162680 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.162661 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98"] Sep 17 12:04:13.164070 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.164056 2581 scope.go:117] "RemoveContainer" containerID="c03d4997f7d61e25721256a61acc590787d377d70f97b2666060b8f2a3fe519f" Sep 17 12:04:13.164364 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:04:13.164343 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c03d4997f7d61e25721256a61acc590787d377d70f97b2666060b8f2a3fe519f\": container with ID starting with c03d4997f7d61e25721256a61acc590787d377d70f97b2666060b8f2a3fe519f not found: ID does not exist" containerID="c03d4997f7d61e25721256a61acc590787d377d70f97b2666060b8f2a3fe519f" Sep 17 12:04:13.164415 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.164369 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c03d4997f7d61e25721256a61acc590787d377d70f97b2666060b8f2a3fe519f"} err="failed to get container status \"c03d4997f7d61e25721256a61acc590787d377d70f97b2666060b8f2a3fe519f\": rpc error: code = NotFound desc = could not find container \"c03d4997f7d61e25721256a61acc590787d377d70f97b2666060b8f2a3fe519f\": container with ID starting with c03d4997f7d61e25721256a61acc590787d377d70f97b2666060b8f2a3fe519f not found: ID does not exist" Sep 17 12:04:13.164415 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.164384 2581 scope.go:117] "RemoveContainer" containerID="c00d842c8047cca91a4ffda1285344b934195947dbe88a5c60bbce4e17b6eecd" Sep 17 12:04:13.164638 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:04:13.164622 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c00d842c8047cca91a4ffda1285344b934195947dbe88a5c60bbce4e17b6eecd\": container with ID starting with c00d842c8047cca91a4ffda1285344b934195947dbe88a5c60bbce4e17b6eecd not found: ID does not exist" containerID="c00d842c8047cca91a4ffda1285344b934195947dbe88a5c60bbce4e17b6eecd" Sep 17 12:04:13.164676 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.164644 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c00d842c8047cca91a4ffda1285344b934195947dbe88a5c60bbce4e17b6eecd"} err="failed to get container status \"c00d842c8047cca91a4ffda1285344b934195947dbe88a5c60bbce4e17b6eecd\": rpc error: code = NotFound desc = could not find container \"c00d842c8047cca91a4ffda1285344b934195947dbe88a5c60bbce4e17b6eecd\": container with ID starting with c00d842c8047cca91a4ffda1285344b934195947dbe88a5c60bbce4e17b6eecd not found: ID does not exist" Sep 17 12:04:13.164676 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.164661 2581 scope.go:117] "RemoveContainer" containerID="c1d484da68803bb045faeae86330abb38c51cb81a331ea9f85aa677704c3f3d1" Sep 17 12:04:13.164885 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:04:13.164869 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1d484da68803bb045faeae86330abb38c51cb81a331ea9f85aa677704c3f3d1\": container with ID starting with c1d484da68803bb045faeae86330abb38c51cb81a331ea9f85aa677704c3f3d1 not found: ID does not exist" containerID="c1d484da68803bb045faeae86330abb38c51cb81a331ea9f85aa677704c3f3d1" Sep 17 12:04:13.164925 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.164890 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1d484da68803bb045faeae86330abb38c51cb81a331ea9f85aa677704c3f3d1"} err="failed to get container status \"c1d484da68803bb045faeae86330abb38c51cb81a331ea9f85aa677704c3f3d1\": rpc error: code = NotFound desc = could not find container \"c1d484da68803bb045faeae86330abb38c51cb81a331ea9f85aa677704c3f3d1\": container with ID starting with c1d484da68803bb045faeae86330abb38c51cb81a331ea9f85aa677704c3f3d1 not found: ID does not exist" Sep 17 12:04:13.164925 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.164904 2581 scope.go:117] "RemoveContainer" containerID="0018acb0d9351c5af3a8fecfc4bb0995ea325904c216e8e7173e6187453df300" Sep 17 12:04:13.165151 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:04:13.165137 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0018acb0d9351c5af3a8fecfc4bb0995ea325904c216e8e7173e6187453df300\": container with ID starting with 0018acb0d9351c5af3a8fecfc4bb0995ea325904c216e8e7173e6187453df300 not found: ID does not exist" containerID="0018acb0d9351c5af3a8fecfc4bb0995ea325904c216e8e7173e6187453df300" Sep 17 12:04:13.165194 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.165155 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0018acb0d9351c5af3a8fecfc4bb0995ea325904c216e8e7173e6187453df300"} err="failed to get container status \"0018acb0d9351c5af3a8fecfc4bb0995ea325904c216e8e7173e6187453df300\": rpc error: code = NotFound desc = could not find container \"0018acb0d9351c5af3a8fecfc4bb0995ea325904c216e8e7173e6187453df300\": container with ID starting with 0018acb0d9351c5af3a8fecfc4bb0995ea325904c216e8e7173e6187453df300 not found: ID does not exist" Sep 17 12:04:13.165194 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.165167 2581 scope.go:117] "RemoveContainer" containerID="f36543bd3734fbb489c6e2879ce3b94126c3a3ed557da9f98cd5d482b76de9f5" Sep 17 12:04:13.165389 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:04:13.165374 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f36543bd3734fbb489c6e2879ce3b94126c3a3ed557da9f98cd5d482b76de9f5\": container with ID starting with f36543bd3734fbb489c6e2879ce3b94126c3a3ed557da9f98cd5d482b76de9f5 not found: ID does not exist" containerID="f36543bd3734fbb489c6e2879ce3b94126c3a3ed557da9f98cd5d482b76de9f5" Sep 17 12:04:13.165426 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.165394 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f36543bd3734fbb489c6e2879ce3b94126c3a3ed557da9f98cd5d482b76de9f5"} err="failed to get container status \"f36543bd3734fbb489c6e2879ce3b94126c3a3ed557da9f98cd5d482b76de9f5\": rpc error: code = NotFound desc = could not find container \"f36543bd3734fbb489c6e2879ce3b94126c3a3ed557da9f98cd5d482b76de9f5\": container with ID starting with f36543bd3734fbb489c6e2879ce3b94126c3a3ed557da9f98cd5d482b76de9f5 not found: ID does not exist" Sep 17 12:04:13.165426 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.165407 2581 scope.go:117] "RemoveContainer" containerID="2b3cd3862f20547ef0fc58e98c253105cda5ef32bd82b9bd220222ccb363a001" Sep 17 12:04:13.165611 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:04:13.165593 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b3cd3862f20547ef0fc58e98c253105cda5ef32bd82b9bd220222ccb363a001\": container with ID starting with 2b3cd3862f20547ef0fc58e98c253105cda5ef32bd82b9bd220222ccb363a001 not found: ID does not exist" containerID="2b3cd3862f20547ef0fc58e98c253105cda5ef32bd82b9bd220222ccb363a001" Sep 17 12:04:13.165699 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.165611 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b3cd3862f20547ef0fc58e98c253105cda5ef32bd82b9bd220222ccb363a001"} err="failed to get container status \"2b3cd3862f20547ef0fc58e98c253105cda5ef32bd82b9bd220222ccb363a001\": rpc error: code = NotFound desc = could not find container \"2b3cd3862f20547ef0fc58e98c253105cda5ef32bd82b9bd220222ccb363a001\": container with ID starting with 2b3cd3862f20547ef0fc58e98c253105cda5ef32bd82b9bd220222ccb363a001 not found: ID does not exist" Sep 17 12:04:13.165699 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.165623 2581 scope.go:117] "RemoveContainer" containerID="c03d4997f7d61e25721256a61acc590787d377d70f97b2666060b8f2a3fe519f" Sep 17 12:04:13.165851 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.165833 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c03d4997f7d61e25721256a61acc590787d377d70f97b2666060b8f2a3fe519f"} err="failed to get container status \"c03d4997f7d61e25721256a61acc590787d377d70f97b2666060b8f2a3fe519f\": rpc error: code = NotFound desc = could not find container \"c03d4997f7d61e25721256a61acc590787d377d70f97b2666060b8f2a3fe519f\": container with ID starting with c03d4997f7d61e25721256a61acc590787d377d70f97b2666060b8f2a3fe519f not found: ID does not exist" Sep 17 12:04:13.165892 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.165852 2581 scope.go:117] "RemoveContainer" containerID="c00d842c8047cca91a4ffda1285344b934195947dbe88a5c60bbce4e17b6eecd" Sep 17 12:04:13.166073 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.166056 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c00d842c8047cca91a4ffda1285344b934195947dbe88a5c60bbce4e17b6eecd"} err="failed to get container status \"c00d842c8047cca91a4ffda1285344b934195947dbe88a5c60bbce4e17b6eecd\": rpc error: code = NotFound desc = could not find container \"c00d842c8047cca91a4ffda1285344b934195947dbe88a5c60bbce4e17b6eecd\": container with ID starting with c00d842c8047cca91a4ffda1285344b934195947dbe88a5c60bbce4e17b6eecd not found: ID does not exist" Sep 17 12:04:13.166073 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.166072 2581 scope.go:117] "RemoveContainer" containerID="c1d484da68803bb045faeae86330abb38c51cb81a331ea9f85aa677704c3f3d1" Sep 17 12:04:13.166312 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.166298 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1d484da68803bb045faeae86330abb38c51cb81a331ea9f85aa677704c3f3d1"} err="failed to get container status \"c1d484da68803bb045faeae86330abb38c51cb81a331ea9f85aa677704c3f3d1\": rpc error: code = NotFound desc = could not find container \"c1d484da68803bb045faeae86330abb38c51cb81a331ea9f85aa677704c3f3d1\": container with ID starting with c1d484da68803bb045faeae86330abb38c51cb81a331ea9f85aa677704c3f3d1 not found: ID does not exist" Sep 17 12:04:13.166373 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.166315 2581 scope.go:117] "RemoveContainer" containerID="0018acb0d9351c5af3a8fecfc4bb0995ea325904c216e8e7173e6187453df300" Sep 17 12:04:13.166613 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.166596 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0018acb0d9351c5af3a8fecfc4bb0995ea325904c216e8e7173e6187453df300"} err="failed to get container status \"0018acb0d9351c5af3a8fecfc4bb0995ea325904c216e8e7173e6187453df300\": rpc error: code = NotFound desc = could not find container \"0018acb0d9351c5af3a8fecfc4bb0995ea325904c216e8e7173e6187453df300\": container with ID starting with 0018acb0d9351c5af3a8fecfc4bb0995ea325904c216e8e7173e6187453df300 not found: ID does not exist" Sep 17 12:04:13.166613 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.166612 2581 scope.go:117] "RemoveContainer" containerID="f36543bd3734fbb489c6e2879ce3b94126c3a3ed557da9f98cd5d482b76de9f5" Sep 17 12:04:13.166864 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.166848 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f36543bd3734fbb489c6e2879ce3b94126c3a3ed557da9f98cd5d482b76de9f5"} err="failed to get container status \"f36543bd3734fbb489c6e2879ce3b94126c3a3ed557da9f98cd5d482b76de9f5\": rpc error: code = NotFound desc = could not find container \"f36543bd3734fbb489c6e2879ce3b94126c3a3ed557da9f98cd5d482b76de9f5\": container with ID starting with f36543bd3734fbb489c6e2879ce3b94126c3a3ed557da9f98cd5d482b76de9f5 not found: ID does not exist" Sep 17 12:04:13.166909 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.166864 2581 scope.go:117] "RemoveContainer" containerID="2b3cd3862f20547ef0fc58e98c253105cda5ef32bd82b9bd220222ccb363a001" Sep 17 12:04:13.167079 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.167062 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b3cd3862f20547ef0fc58e98c253105cda5ef32bd82b9bd220222ccb363a001"} err="failed to get container status \"2b3cd3862f20547ef0fc58e98c253105cda5ef32bd82b9bd220222ccb363a001\": rpc error: code = NotFound desc = could not find container \"2b3cd3862f20547ef0fc58e98c253105cda5ef32bd82b9bd220222ccb363a001\": container with ID starting with 2b3cd3862f20547ef0fc58e98c253105cda5ef32bd82b9bd220222ccb363a001 not found: ID does not exist" Sep 17 12:04:13.181293 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.181251 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-xnh98"] Sep 17 12:04:13.228448 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.228426 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78"] Sep 17 12:04:13.236048 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:13.236028 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-jqz78"] Sep 17 12:04:14.427479 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:14.427447 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" path="/var/lib/kubelet/pods/8d898cc6-2ba3-46fe-b336-ff3e0420bd64/volumes" Sep 17 12:04:14.427960 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:14.427947 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" path="/var/lib/kubelet/pods/e7017c5c-f264-43e5-ae29-05b7cdf223c8/volumes" Sep 17 12:04:31.161625 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:31.161597 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-25hv7"] Sep 17 12:04:31.161999 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:31.161834 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-25hv7" podUID="a21333e3-a759-4b18-83f5-1df25c72abb6" containerName="manager" containerID="cri-o://f02984691ea4d69622ea100a55e4cef796ac73459f154bd1c7d032057a8cbde7" gracePeriod=10 Sep 17 12:04:31.405317 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:31.405293 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-25hv7" Sep 17 12:04:31.551567 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:31.551541 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27dz4\" (UniqueName: \"kubernetes.io/projected/a21333e3-a759-4b18-83f5-1df25c72abb6-kube-api-access-27dz4\") pod \"a21333e3-a759-4b18-83f5-1df25c72abb6\" (UID: \"a21333e3-a759-4b18-83f5-1df25c72abb6\") " Sep 17 12:04:31.553636 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:31.553606 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a21333e3-a759-4b18-83f5-1df25c72abb6-kube-api-access-27dz4" (OuterVolumeSpecName: "kube-api-access-27dz4") pod "a21333e3-a759-4b18-83f5-1df25c72abb6" (UID: "a21333e3-a759-4b18-83f5-1df25c72abb6"). InnerVolumeSpecName "kube-api-access-27dz4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 12:04:31.652005 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:31.651980 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27dz4\" (UniqueName: \"kubernetes.io/projected/a21333e3-a759-4b18-83f5-1df25c72abb6-kube-api-access-27dz4\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:04:32.164232 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:32.164196 2581 generic.go:356] "Generic (PLEG): container finished" podID="a21333e3-a759-4b18-83f5-1df25c72abb6" containerID="f02984691ea4d69622ea100a55e4cef796ac73459f154bd1c7d032057a8cbde7" exitCode=0 Sep 17 12:04:32.164672 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:32.164242 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-25hv7" Sep 17 12:04:32.164672 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:32.164275 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-25hv7" event={"ID":"a21333e3-a759-4b18-83f5-1df25c72abb6","Type":"ContainerDied","Data":"f02984691ea4d69622ea100a55e4cef796ac73459f154bd1c7d032057a8cbde7"} Sep 17 12:04:32.164672 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:32.164320 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-25hv7" event={"ID":"a21333e3-a759-4b18-83f5-1df25c72abb6","Type":"ContainerDied","Data":"cea6112b99dd5fce52b8f5f03256582245f61adb9cd57222640ad3390ab1edb1"} Sep 17 12:04:32.164672 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:32.164342 2581 scope.go:117] "RemoveContainer" containerID="f02984691ea4d69622ea100a55e4cef796ac73459f154bd1c7d032057a8cbde7" Sep 17 12:04:32.173005 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:32.172990 2581 scope.go:117] "RemoveContainer" containerID="f02984691ea4d69622ea100a55e4cef796ac73459f154bd1c7d032057a8cbde7" Sep 17 12:04:32.173318 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:04:32.173298 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f02984691ea4d69622ea100a55e4cef796ac73459f154bd1c7d032057a8cbde7\": container with ID starting with f02984691ea4d69622ea100a55e4cef796ac73459f154bd1c7d032057a8cbde7 not found: ID does not exist" containerID="f02984691ea4d69622ea100a55e4cef796ac73459f154bd1c7d032057a8cbde7" Sep 17 12:04:32.173382 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:32.173329 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f02984691ea4d69622ea100a55e4cef796ac73459f154bd1c7d032057a8cbde7"} err="failed to get container status \"f02984691ea4d69622ea100a55e4cef796ac73459f154bd1c7d032057a8cbde7\": rpc error: code = NotFound desc = could not find container \"f02984691ea4d69622ea100a55e4cef796ac73459f154bd1c7d032057a8cbde7\": container with ID starting with f02984691ea4d69622ea100a55e4cef796ac73459f154bd1c7d032057a8cbde7 not found: ID does not exist" Sep 17 12:04:32.190949 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:32.190922 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-25hv7"] Sep 17 12:04:32.195773 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:32.195748 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-25hv7"] Sep 17 12:04:32.431276 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:32.431213 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a21333e3-a759-4b18-83f5-1df25c72abb6" path="/var/lib/kubelet/pods/a21333e3-a759-4b18-83f5-1df25c72abb6/volumes" Sep 17 12:04:36.669928 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.669896 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-lznhg"] Sep 17 12:04:36.670319 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670208 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="registry" Sep 17 12:04:36.670319 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670219 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="registry" Sep 17 12:04:36.670319 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670226 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a21333e3-a759-4b18-83f5-1df25c72abb6" containerName="manager" Sep 17 12:04:36.670319 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670231 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a21333e3-a759-4b18-83f5-1df25c72abb6" containerName="manager" Sep 17 12:04:36.670319 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670237 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="feast-init" Sep 17 12:04:36.670319 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670242 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="feast-init" Sep 17 12:04:36.670319 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670248 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="offline" Sep 17 12:04:36.670319 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670252 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="offline" Sep 17 12:04:36.670319 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670257 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="feast-apply" Sep 17 12:04:36.670319 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670262 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="feast-apply" Sep 17 12:04:36.670319 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670268 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="online" Sep 17 12:04:36.670319 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670272 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="online" Sep 17 12:04:36.670319 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670280 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="feast-init" Sep 17 12:04:36.670319 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670284 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="feast-init" Sep 17 12:04:36.670319 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670290 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="offline" Sep 17 12:04:36.670319 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670294 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="offline" Sep 17 12:04:36.670319 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670306 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="feast-apply" Sep 17 12:04:36.670319 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670311 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="feast-apply" Sep 17 12:04:36.670319 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670316 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="ui" Sep 17 12:04:36.670319 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670321 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="ui" Sep 17 12:04:36.670319 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670329 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="ui" Sep 17 12:04:36.671008 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670334 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="ui" Sep 17 12:04:36.671008 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670341 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="online" Sep 17 12:04:36.671008 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670346 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="online" Sep 17 12:04:36.671008 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670392 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a21333e3-a759-4b18-83f5-1df25c72abb6" containerName="manager" Sep 17 12:04:36.671008 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670398 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="ui" Sep 17 12:04:36.671008 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670404 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="registry" Sep 17 12:04:36.671008 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670411 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="online" Sep 17 12:04:36.671008 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670416 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="offline" Sep 17 12:04:36.671008 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670420 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7017c5c-f264-43e5-ae29-05b7cdf223c8" containerName="ui" Sep 17 12:04:36.671008 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670426 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="online" Sep 17 12:04:36.671008 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.670431 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d898cc6-2ba3-46fe-b336-ff3e0420bd64" containerName="offline" Sep 17 12:04:36.674533 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.674518 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-lznhg" Sep 17 12:04:36.683668 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.678323 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-n87gs\"" Sep 17 12:04:36.683668 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.678624 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 17 12:04:36.683668 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.678898 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 17 12:04:36.686909 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.686880 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-lznhg"] Sep 17 12:04:36.789711 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.789688 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66v5j\" (UniqueName: \"kubernetes.io/projected/20d73c28-353d-428e-a7a8-dcd0a0ae5499-kube-api-access-66v5j\") pod \"feast-operator-controller-manager-689b877b4b-lznhg\" (UID: \"20d73c28-353d-428e-a7a8-dcd0a0ae5499\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-lznhg" Sep 17 12:04:36.890181 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.890145 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66v5j\" (UniqueName: \"kubernetes.io/projected/20d73c28-353d-428e-a7a8-dcd0a0ae5499-kube-api-access-66v5j\") pod \"feast-operator-controller-manager-689b877b4b-lznhg\" (UID: \"20d73c28-353d-428e-a7a8-dcd0a0ae5499\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-lznhg" Sep 17 12:04:36.908437 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.908416 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66v5j\" (UniqueName: \"kubernetes.io/projected/20d73c28-353d-428e-a7a8-dcd0a0ae5499-kube-api-access-66v5j\") pod \"feast-operator-controller-manager-689b877b4b-lznhg\" (UID: \"20d73c28-353d-428e-a7a8-dcd0a0ae5499\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-lznhg" Sep 17 12:04:36.992008 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:36.991948 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-lznhg" Sep 17 12:04:37.285317 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:37.285290 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-lznhg"] Sep 17 12:04:37.288423 ip-10-0-140-196 kubenswrapper[2581]: W0917 12:04:37.288398 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20d73c28_353d_428e_a7a8_dcd0a0ae5499.slice/crio-47756e4442fd7bda2e443d167f808ce032b1fc00b87870d89a632b1e397af2a7 WatchSource:0}: Error finding container 47756e4442fd7bda2e443d167f808ce032b1fc00b87870d89a632b1e397af2a7: Status 404 returned error can't find the container with id 47756e4442fd7bda2e443d167f808ce032b1fc00b87870d89a632b1e397af2a7 Sep 17 12:04:38.192694 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:38.192657 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-lznhg" event={"ID":"20d73c28-353d-428e-a7a8-dcd0a0ae5499","Type":"ContainerStarted","Data":"7292713932acf7631534a48ae1601f35ec012c0eb8fd1c41d1a230d67ee8d487"} Sep 17 12:04:38.193182 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:38.192716 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-lznhg" event={"ID":"20d73c28-353d-428e-a7a8-dcd0a0ae5499","Type":"ContainerStarted","Data":"47756e4442fd7bda2e443d167f808ce032b1fc00b87870d89a632b1e397af2a7"} Sep 17 12:04:38.193182 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:38.192773 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-lznhg" Sep 17 12:04:38.244641 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:38.244593 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-lznhg" podStartSLOduration=2.244579137 podStartE2EDuration="2.244579137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 12:04:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 12:04:38.243959166 +0000 UTC m=+1548.251406195" watchObservedRunningTime="2026-09-17 12:04:38.244579137 +0000 UTC m=+1548.252026165" Sep 17 12:04:49.198937 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:49.198907 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-lznhg" Sep 17 12:04:55.328346 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.328305 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq"] Sep 17 12:04:55.333358 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.333332 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:04:55.336353 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.336334 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 17 12:04:55.336515 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.336504 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 17 12:04:55.336648 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.336634 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-pzlq4\"" Sep 17 12:04:55.336960 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.336931 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 17 12:04:55.337094 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.336997 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 17 12:04:55.337094 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.337043 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 17 12:04:55.337337 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.337322 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 17 12:04:55.353592 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.353571 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq"] Sep 17 12:04:55.523255 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.523231 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-99lfq\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:04:55.523372 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.523266 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-99lfq\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:04:55.523372 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.523284 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-99lfq\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:04:55.523372 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.523298 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-99lfq\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:04:55.523372 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.523356 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-99lfq\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:04:55.523502 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.523424 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bprd\" (UniqueName: \"kubernetes.io/projected/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-kube-api-access-6bprd\") pod \"feast-simple-feast-setup-57576b8fcf-99lfq\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:04:55.624664 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.624610 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-99lfq\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:04:55.624664 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.624637 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-99lfq\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:04:55.624664 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.624652 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-99lfq\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:04:55.624664 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.624666 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-99lfq\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:04:55.624879 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.624720 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bprd\" (UniqueName: \"kubernetes.io/projected/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-kube-api-access-6bprd\") pod \"feast-simple-feast-setup-57576b8fcf-99lfq\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:04:55.624879 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:04:55.624755 2581 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 17 12:04:55.624879 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.624757 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-99lfq\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:04:55.624879 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:04:55.624818 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-registry-tls podName:55ffd1a2-c3c5-4478-aae6-8f322b0d1c17 nodeName:}" failed. No retries permitted until 2026-09-17 12:04:56.124798124 +0000 UTC m=+1566.132245134 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-registry-tls") pod "feast-simple-feast-setup-57576b8fcf-99lfq" (UID: "55ffd1a2-c3c5-4478-aae6-8f322b0d1c17") : secret "feast-simple-feast-setup-registry-tls" not found Sep 17 12:04:55.625068 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.625048 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-99lfq\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:04:55.627257 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.627239 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-99lfq\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:04:55.627356 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.627300 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-99lfq\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:04:55.627437 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.627416 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-99lfq\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:04:55.633283 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:55.633267 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bprd\" (UniqueName: \"kubernetes.io/projected/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-kube-api-access-6bprd\") pod \"feast-simple-feast-setup-57576b8fcf-99lfq\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:04:56.130508 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:56.130472 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-99lfq\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:04:56.132919 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:56.132897 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-99lfq\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:04:56.242260 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:56.242236 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:04:56.524304 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:56.524279 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq"] Sep 17 12:04:56.526143 ip-10-0-140-196 kubenswrapper[2581]: W0917 12:04:56.526098 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55ffd1a2_c3c5_4478_aae6_8f322b0d1c17.slice/crio-e7290cb0717215a094492155e8f7914b9479b7273a4e6c86e79d0efea654f353 WatchSource:0}: Error finding container e7290cb0717215a094492155e8f7914b9479b7273a4e6c86e79d0efea654f353: Status 404 returned error can't find the container with id e7290cb0717215a094492155e8f7914b9479b7273a4e6c86e79d0efea654f353 Sep 17 12:04:57.279599 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:57.279561 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" event={"ID":"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17","Type":"ContainerStarted","Data":"81d1a1f2e1bae0cbec17ad5dd6d49872601e08b749db1ba942a0ee1a1a9de15e"} Sep 17 12:04:57.279771 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:04:57.279615 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" event={"ID":"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17","Type":"ContainerStarted","Data":"e7290cb0717215a094492155e8f7914b9479b7273a4e6c86e79d0efea654f353"} Sep 17 12:05:02.303098 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:02.303065 2581 generic.go:356] "Generic (PLEG): container finished" podID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerID="81d1a1f2e1bae0cbec17ad5dd6d49872601e08b749db1ba942a0ee1a1a9de15e" exitCode=0 Sep 17 12:05:02.303509 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:02.303140 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" event={"ID":"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17","Type":"ContainerDied","Data":"81d1a1f2e1bae0cbec17ad5dd6d49872601e08b749db1ba942a0ee1a1a9de15e"} Sep 17 12:05:03.309870 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:03.309839 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" event={"ID":"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17","Type":"ContainerStarted","Data":"89929aa66c97048e8bfd985a4cdd375ddf0f3278727952cbc0fbedddc0135a17"} Sep 17 12:05:08.333972 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:08.333890 2581 generic.go:356] "Generic (PLEG): container finished" podID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerID="89929aa66c97048e8bfd985a4cdd375ddf0f3278727952cbc0fbedddc0135a17" exitCode=0 Sep 17 12:05:08.334462 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:08.333971 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" event={"ID":"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17","Type":"ContainerDied","Data":"89929aa66c97048e8bfd985a4cdd375ddf0f3278727952cbc0fbedddc0135a17"} Sep 17 12:05:09.345343 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:09.345297 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" event={"ID":"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17","Type":"ContainerStarted","Data":"22c1b34fab5b196b15ea76dbf3b352e78efc8408bae308610077865dfe1d8254"} Sep 17 12:05:09.345847 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:09.345371 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" event={"ID":"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17","Type":"ContainerStarted","Data":"425d44844b8ac11fd522dbd6414b2d19e2598af6ca33ae1f8737b9e03809b3e0"} Sep 17 12:05:09.345847 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:09.345395 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" event={"ID":"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17","Type":"ContainerStarted","Data":"b4faee45209b8679d7c43921031edbdfc45ced19b4f7061282e0cae26d2f54a8"} Sep 17 12:05:09.345847 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:09.345413 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" event={"ID":"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17","Type":"ContainerStarted","Data":"76acc9427ed939a68cb82a1aa254aec5409ac9dabdf5864e39b37f2249566aba"} Sep 17 12:05:09.374003 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:09.373937 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" podStartSLOduration=14.37391626 podStartE2EDuration="14.37391626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 12:04:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 12:05:09.371879569 +0000 UTC m=+1579.379326600" watchObservedRunningTime="2026-09-17 12:05:09.37391626 +0000 UTC m=+1579.381363292" Sep 17 12:05:11.243305 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:11.243267 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:05:11.243774 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:11.243331 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:05:11.243774 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:11.243351 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:05:11.243774 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:11.243370 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:05:11.245743 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:11.245709 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused" Sep 17 12:05:11.245743 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:11.245725 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.35:6571: connect: connection refused" Sep 17 12:05:11.245930 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:11.245837 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused" Sep 17 12:05:11.246026 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:11.246000 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.35:8816: connect: connection refused" Sep 17 12:05:14.243157 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:14.243083 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.35:8816: connect: connection refused" Sep 17 12:05:14.243772 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:14.243083 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.35:6571: connect: connection refused" Sep 17 12:05:14.243908 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:14.243083 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused" Sep 17 12:05:14.243908 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:14.243196 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused" Sep 17 12:05:16.242895 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:16.242851 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:05:16.243270 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:16.242913 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:05:16.243270 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:16.242926 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:05:16.243270 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:16.242937 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:05:17.244465 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:17.244431 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:05:17.244917 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:17.244648 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:05:17.244917 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:17.244832 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:05:17.245733 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:17.245710 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:05:17.245842 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:17.245782 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:05:17.245842 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:17.245827 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:05:17.250109 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:17.250090 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:05:17.398430 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:05:17.398404 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:06:20.003147 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.002668 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq"] Sep 17 12:06:20.003147 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.002939 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="registry" containerID="cri-o://76acc9427ed939a68cb82a1aa254aec5409ac9dabdf5864e39b37f2249566aba" gracePeriod=30 Sep 17 12:06:20.005509 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.005049 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="ui" containerID="cri-o://22c1b34fab5b196b15ea76dbf3b352e78efc8408bae308610077865dfe1d8254" gracePeriod=30 Sep 17 12:06:20.005509 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.005217 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="offline" containerID="cri-o://425d44844b8ac11fd522dbd6414b2d19e2598af6ca33ae1f8737b9e03809b3e0" gracePeriod=30 Sep 17 12:06:20.005509 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.005295 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="online" containerID="cri-o://b4faee45209b8679d7c43921031edbdfc45ced19b4f7061282e0cae26d2f54a8" gracePeriod=30 Sep 17 12:06:20.437986 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.437894 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn"] Sep 17 12:06:20.444240 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.444210 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:20.447089 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.447062 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-nhrck\"" Sep 17 12:06:20.456813 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.456781 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn"] Sep 17 12:06:20.558336 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.558292 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e08b89bd-f172-45a7-b468-bf1194e69658-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-rw5hn\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:20.558550 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.558364 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-rw5hn\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:20.558550 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.558423 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-rw5hn\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:20.558550 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.558511 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6tgx\" (UniqueName: \"kubernetes.io/projected/e08b89bd-f172-45a7-b468-bf1194e69658-kube-api-access-l6tgx\") pod \"feast-simple-feast-setup-57576b8fcf-rw5hn\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:20.558731 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.558575 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-rw5hn\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:20.558731 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.558615 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-rw5hn\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:20.659347 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.659309 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6tgx\" (UniqueName: \"kubernetes.io/projected/e08b89bd-f172-45a7-b468-bf1194e69658-kube-api-access-l6tgx\") pod \"feast-simple-feast-setup-57576b8fcf-rw5hn\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:20.659535 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.659362 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-rw5hn\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:20.659535 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.659500 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-rw5hn\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:20.659673 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.659550 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e08b89bd-f172-45a7-b468-bf1194e69658-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-rw5hn\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:20.659673 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.659615 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-rw5hn\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:20.659768 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.659680 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-rw5hn\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:20.659965 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.659908 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e08b89bd-f172-45a7-b468-bf1194e69658-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-rw5hn\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:20.662135 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.662095 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-rw5hn\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:20.662250 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.662178 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-rw5hn\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:20.662603 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.662583 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-rw5hn\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:20.662706 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.662619 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-rw5hn\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:20.669887 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.669865 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6tgx\" (UniqueName: \"kubernetes.io/projected/e08b89bd-f172-45a7-b468-bf1194e69658-kube-api-access-l6tgx\") pod \"feast-simple-feast-setup-57576b8fcf-rw5hn\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:20.754347 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:20.754308 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:21.054804 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:21.054776 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn"] Sep 17 12:06:21.673067 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:21.673032 2581 generic.go:356] "Generic (PLEG): container finished" podID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerID="22c1b34fab5b196b15ea76dbf3b352e78efc8408bae308610077865dfe1d8254" exitCode=0 Sep 17 12:06:21.673067 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:21.673063 2581 generic.go:356] "Generic (PLEG): container finished" podID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerID="b4faee45209b8679d7c43921031edbdfc45ced19b4f7061282e0cae26d2f54a8" exitCode=0 Sep 17 12:06:21.673318 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:21.673097 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" event={"ID":"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17","Type":"ContainerDied","Data":"22c1b34fab5b196b15ea76dbf3b352e78efc8408bae308610077865dfe1d8254"} Sep 17 12:06:21.673318 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:21.673162 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" event={"ID":"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17","Type":"ContainerDied","Data":"b4faee45209b8679d7c43921031edbdfc45ced19b4f7061282e0cae26d2f54a8"} Sep 17 12:06:21.674672 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:21.674651 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" event={"ID":"e08b89bd-f172-45a7-b468-bf1194e69658","Type":"ContainerStarted","Data":"a82841caae4199c802a4a800bbc9d29ede333a5d932c1d30e001e28830804efe"} Sep 17 12:06:21.674783 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:21.674680 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" event={"ID":"e08b89bd-f172-45a7-b468-bf1194e69658","Type":"ContainerStarted","Data":"ed776fb6217249420dec2143bb3a840f6d60af341ebfe1eef20f2f17dca8e487"} Sep 17 12:06:26.700463 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:26.700425 2581 generic.go:356] "Generic (PLEG): container finished" podID="e08b89bd-f172-45a7-b468-bf1194e69658" containerID="a82841caae4199c802a4a800bbc9d29ede333a5d932c1d30e001e28830804efe" exitCode=0 Sep 17 12:06:26.700867 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:26.700503 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" event={"ID":"e08b89bd-f172-45a7-b468-bf1194e69658","Type":"ContainerDied","Data":"a82841caae4199c802a4a800bbc9d29ede333a5d932c1d30e001e28830804efe"} Sep 17 12:06:27.245773 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:27.245722 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused" Sep 17 12:06:27.396009 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:27.395961 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused" Sep 17 12:06:27.708020 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:27.707981 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" event={"ID":"e08b89bd-f172-45a7-b468-bf1194e69658","Type":"ContainerStarted","Data":"692668d9343d81d28b189634b7fa59b3da025efe6fc7af77a4e2e59cc63370e4"} Sep 17 12:06:32.730602 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:32.730518 2581 generic.go:356] "Generic (PLEG): container finished" podID="e08b89bd-f172-45a7-b468-bf1194e69658" containerID="692668d9343d81d28b189634b7fa59b3da025efe6fc7af77a4e2e59cc63370e4" exitCode=0 Sep 17 12:06:32.731019 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:32.730593 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" event={"ID":"e08b89bd-f172-45a7-b468-bf1194e69658","Type":"ContainerDied","Data":"692668d9343d81d28b189634b7fa59b3da025efe6fc7af77a4e2e59cc63370e4"} Sep 17 12:06:33.742025 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:33.741987 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" event={"ID":"e08b89bd-f172-45a7-b468-bf1194e69658","Type":"ContainerStarted","Data":"fec21e210ddd7393cd10768aa59b078eb4e177f153d48b2469af49fd02d3917f"} Sep 17 12:06:33.742566 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:33.742053 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" event={"ID":"e08b89bd-f172-45a7-b468-bf1194e69658","Type":"ContainerStarted","Data":"a7e50b1255a3bb32d392f7478a9adce16e95d6cc9d98aebf2a3afc13c810862c"} Sep 17 12:06:33.742566 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:33.742077 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" event={"ID":"e08b89bd-f172-45a7-b468-bf1194e69658","Type":"ContainerStarted","Data":"e89ea3f8edcb8bee7a08924882e2563c5ac56ad3c01040971573fd3b678d60db"} Sep 17 12:06:33.742566 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:33.742097 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" event={"ID":"e08b89bd-f172-45a7-b468-bf1194e69658","Type":"ContainerStarted","Data":"dbac67fc14155dad393a5753a66b7794b5d71ba21610a32544b942bb2a32d590"} Sep 17 12:06:33.779209 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:33.779150 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" podStartSLOduration=13.779109066 podStartE2EDuration="13.779109066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 12:06:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 12:06:33.775675702 +0000 UTC m=+1663.783122733" watchObservedRunningTime="2026-09-17 12:06:33.779109066 +0000 UTC m=+1663.786556097" Sep 17 12:06:35.754860 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:35.754644 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:35.754860 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:35.754704 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:35.754860 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:35.754727 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:35.754860 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:35.754745 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:35.757252 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:35.757070 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.36:6571: connect: connection refused" Sep 17 12:06:35.757252 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:35.757102 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused" Sep 17 12:06:35.757252 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:35.757176 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused" Sep 17 12:06:35.757591 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:35.757565 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.36:8816: connect: connection refused" Sep 17 12:06:37.246281 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:37.246237 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused" Sep 17 12:06:37.396132 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:37.396069 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused" Sep 17 12:06:38.755610 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:38.755565 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused" Sep 17 12:06:38.755610 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:38.755590 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.36:6571: connect: connection refused" Sep 17 12:06:38.756246 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:38.755565 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.36:8816: connect: connection refused" Sep 17 12:06:38.756421 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:38.756386 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused" Sep 17 12:06:40.754683 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:40.754647 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:40.755085 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:40.754701 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:40.755085 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:40.754714 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:40.755085 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:40.754727 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:41.755178 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:41.755142 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:41.755644 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:41.755368 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:41.755644 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:41.755511 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:41.756138 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:41.756098 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:41.756260 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:41.756200 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:41.756260 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:41.756245 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:41.760656 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:41.760636 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:41.794706 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:41.794682 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:06:47.246352 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:47.246308 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused" Sep 17 12:06:47.246839 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:47.246462 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:06:47.395795 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:47.395758 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused" Sep 17 12:06:47.395945 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:47.395865 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:06:50.654814 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.654789 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:06:50.741082 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.741060 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-online-tls\") pod \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " Sep 17 12:06:50.741240 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.741088 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-feast-data\") pod \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " Sep 17 12:06:50.741240 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.741138 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-offline-tls\") pod \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " Sep 17 12:06:50.741240 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.741205 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-registry-tls\") pod \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " Sep 17 12:06:50.741240 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.741234 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-ui-tls\") pod \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " Sep 17 12:06:50.741469 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.741276 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bprd\" (UniqueName: \"kubernetes.io/projected/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-kube-api-access-6bprd\") pod \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\" (UID: \"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17\") " Sep 17 12:06:50.741666 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.741630 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-feast-data" (OuterVolumeSpecName: "feast-data") pod "55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" (UID: "55ffd1a2-c3c5-4478-aae6-8f322b0d1c17"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 12:06:50.743644 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.743620 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-online-tls" (OuterVolumeSpecName: "online-tls") pod "55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" (UID: "55ffd1a2-c3c5-4478-aae6-8f322b0d1c17"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:06:50.743764 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.743662 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-kube-api-access-6bprd" (OuterVolumeSpecName: "kube-api-access-6bprd") pod "55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" (UID: "55ffd1a2-c3c5-4478-aae6-8f322b0d1c17"). InnerVolumeSpecName "kube-api-access-6bprd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 12:06:50.743764 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.743720 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" (UID: "55ffd1a2-c3c5-4478-aae6-8f322b0d1c17"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:06:50.743887 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.743863 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" (UID: "55ffd1a2-c3c5-4478-aae6-8f322b0d1c17"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:06:50.743930 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.743888 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" (UID: "55ffd1a2-c3c5-4478-aae6-8f322b0d1c17"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:06:50.833802 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.833731 2581 generic.go:356] "Generic (PLEG): container finished" podID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerID="425d44844b8ac11fd522dbd6414b2d19e2598af6ca33ae1f8737b9e03809b3e0" exitCode=137 Sep 17 12:06:50.833802 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.833753 2581 generic.go:356] "Generic (PLEG): container finished" podID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerID="76acc9427ed939a68cb82a1aa254aec5409ac9dabdf5864e39b37f2249566aba" exitCode=137 Sep 17 12:06:50.833802 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.833799 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" Sep 17 12:06:50.833984 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.833808 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" event={"ID":"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17","Type":"ContainerDied","Data":"425d44844b8ac11fd522dbd6414b2d19e2598af6ca33ae1f8737b9e03809b3e0"} Sep 17 12:06:50.833984 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.833857 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" event={"ID":"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17","Type":"ContainerDied","Data":"76acc9427ed939a68cb82a1aa254aec5409ac9dabdf5864e39b37f2249566aba"} Sep 17 12:06:50.833984 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.833872 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq" event={"ID":"55ffd1a2-c3c5-4478-aae6-8f322b0d1c17","Type":"ContainerDied","Data":"e7290cb0717215a094492155e8f7914b9479b7273a4e6c86e79d0efea654f353"} Sep 17 12:06:50.833984 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.833891 2581 scope.go:117] "RemoveContainer" containerID="22c1b34fab5b196b15ea76dbf3b352e78efc8408bae308610077865dfe1d8254" Sep 17 12:06:50.841895 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.841880 2581 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-feast-data\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:06:50.841977 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.841900 2581 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-offline-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:06:50.841977 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.841911 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-registry-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:06:50.841977 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.841919 2581 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-ui-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:06:50.841977 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.841928 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bprd\" (UniqueName: \"kubernetes.io/projected/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-kube-api-access-6bprd\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:06:50.841977 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.841958 2581 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17-online-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:06:50.842858 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.842844 2581 scope.go:117] "RemoveContainer" containerID="425d44844b8ac11fd522dbd6414b2d19e2598af6ca33ae1f8737b9e03809b3e0" Sep 17 12:06:50.851057 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.851043 2581 scope.go:117] "RemoveContainer" containerID="b4faee45209b8679d7c43921031edbdfc45ced19b4f7061282e0cae26d2f54a8" Sep 17 12:06:50.858735 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.858714 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq"] Sep 17 12:06:50.861742 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.861733 2581 scope.go:117] "RemoveContainer" containerID="76acc9427ed939a68cb82a1aa254aec5409ac9dabdf5864e39b37f2249566aba" Sep 17 12:06:50.865558 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.865540 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-99lfq"] Sep 17 12:06:50.871989 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.871976 2581 scope.go:117] "RemoveContainer" containerID="89929aa66c97048e8bfd985a4cdd375ddf0f3278727952cbc0fbedddc0135a17" Sep 17 12:06:50.879703 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.879686 2581 scope.go:117] "RemoveContainer" containerID="81d1a1f2e1bae0cbec17ad5dd6d49872601e08b749db1ba942a0ee1a1a9de15e" Sep 17 12:06:50.889267 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.889256 2581 scope.go:117] "RemoveContainer" containerID="22c1b34fab5b196b15ea76dbf3b352e78efc8408bae308610077865dfe1d8254" Sep 17 12:06:50.889535 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:06:50.889516 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22c1b34fab5b196b15ea76dbf3b352e78efc8408bae308610077865dfe1d8254\": container with ID starting with 22c1b34fab5b196b15ea76dbf3b352e78efc8408bae308610077865dfe1d8254 not found: ID does not exist" containerID="22c1b34fab5b196b15ea76dbf3b352e78efc8408bae308610077865dfe1d8254" Sep 17 12:06:50.889588 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.889540 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22c1b34fab5b196b15ea76dbf3b352e78efc8408bae308610077865dfe1d8254"} err="failed to get container status \"22c1b34fab5b196b15ea76dbf3b352e78efc8408bae308610077865dfe1d8254\": rpc error: code = NotFound desc = could not find container \"22c1b34fab5b196b15ea76dbf3b352e78efc8408bae308610077865dfe1d8254\": container with ID starting with 22c1b34fab5b196b15ea76dbf3b352e78efc8408bae308610077865dfe1d8254 not found: ID does not exist" Sep 17 12:06:50.889588 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.889557 2581 scope.go:117] "RemoveContainer" containerID="425d44844b8ac11fd522dbd6414b2d19e2598af6ca33ae1f8737b9e03809b3e0" Sep 17 12:06:50.889786 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:06:50.889770 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"425d44844b8ac11fd522dbd6414b2d19e2598af6ca33ae1f8737b9e03809b3e0\": container with ID starting with 425d44844b8ac11fd522dbd6414b2d19e2598af6ca33ae1f8737b9e03809b3e0 not found: ID does not exist" containerID="425d44844b8ac11fd522dbd6414b2d19e2598af6ca33ae1f8737b9e03809b3e0" Sep 17 12:06:50.889836 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.889789 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"425d44844b8ac11fd522dbd6414b2d19e2598af6ca33ae1f8737b9e03809b3e0"} err="failed to get container status \"425d44844b8ac11fd522dbd6414b2d19e2598af6ca33ae1f8737b9e03809b3e0\": rpc error: code = NotFound desc = could not find container \"425d44844b8ac11fd522dbd6414b2d19e2598af6ca33ae1f8737b9e03809b3e0\": container with ID starting with 425d44844b8ac11fd522dbd6414b2d19e2598af6ca33ae1f8737b9e03809b3e0 not found: ID does not exist" Sep 17 12:06:50.889836 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.889802 2581 scope.go:117] "RemoveContainer" containerID="b4faee45209b8679d7c43921031edbdfc45ced19b4f7061282e0cae26d2f54a8" Sep 17 12:06:50.890047 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:06:50.890033 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4faee45209b8679d7c43921031edbdfc45ced19b4f7061282e0cae26d2f54a8\": container with ID starting with b4faee45209b8679d7c43921031edbdfc45ced19b4f7061282e0cae26d2f54a8 not found: ID does not exist" containerID="b4faee45209b8679d7c43921031edbdfc45ced19b4f7061282e0cae26d2f54a8" Sep 17 12:06:50.890144 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.890050 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4faee45209b8679d7c43921031edbdfc45ced19b4f7061282e0cae26d2f54a8"} err="failed to get container status \"b4faee45209b8679d7c43921031edbdfc45ced19b4f7061282e0cae26d2f54a8\": rpc error: code = NotFound desc = could not find container \"b4faee45209b8679d7c43921031edbdfc45ced19b4f7061282e0cae26d2f54a8\": container with ID starting with b4faee45209b8679d7c43921031edbdfc45ced19b4f7061282e0cae26d2f54a8 not found: ID does not exist" Sep 17 12:06:50.890144 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.890062 2581 scope.go:117] "RemoveContainer" containerID="76acc9427ed939a68cb82a1aa254aec5409ac9dabdf5864e39b37f2249566aba" Sep 17 12:06:50.890324 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:06:50.890298 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76acc9427ed939a68cb82a1aa254aec5409ac9dabdf5864e39b37f2249566aba\": container with ID starting with 76acc9427ed939a68cb82a1aa254aec5409ac9dabdf5864e39b37f2249566aba not found: ID does not exist" containerID="76acc9427ed939a68cb82a1aa254aec5409ac9dabdf5864e39b37f2249566aba" Sep 17 12:06:50.890390 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.890330 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76acc9427ed939a68cb82a1aa254aec5409ac9dabdf5864e39b37f2249566aba"} err="failed to get container status \"76acc9427ed939a68cb82a1aa254aec5409ac9dabdf5864e39b37f2249566aba\": rpc error: code = NotFound desc = could not find container \"76acc9427ed939a68cb82a1aa254aec5409ac9dabdf5864e39b37f2249566aba\": container with ID starting with 76acc9427ed939a68cb82a1aa254aec5409ac9dabdf5864e39b37f2249566aba not found: ID does not exist" Sep 17 12:06:50.890390 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.890345 2581 scope.go:117] "RemoveContainer" containerID="89929aa66c97048e8bfd985a4cdd375ddf0f3278727952cbc0fbedddc0135a17" Sep 17 12:06:50.890562 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:06:50.890548 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89929aa66c97048e8bfd985a4cdd375ddf0f3278727952cbc0fbedddc0135a17\": container with ID starting with 89929aa66c97048e8bfd985a4cdd375ddf0f3278727952cbc0fbedddc0135a17 not found: ID does not exist" containerID="89929aa66c97048e8bfd985a4cdd375ddf0f3278727952cbc0fbedddc0135a17" Sep 17 12:06:50.890612 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.890566 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89929aa66c97048e8bfd985a4cdd375ddf0f3278727952cbc0fbedddc0135a17"} err="failed to get container status \"89929aa66c97048e8bfd985a4cdd375ddf0f3278727952cbc0fbedddc0135a17\": rpc error: code = NotFound desc = could not find container \"89929aa66c97048e8bfd985a4cdd375ddf0f3278727952cbc0fbedddc0135a17\": container with ID starting with 89929aa66c97048e8bfd985a4cdd375ddf0f3278727952cbc0fbedddc0135a17 not found: ID does not exist" Sep 17 12:06:50.890612 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.890578 2581 scope.go:117] "RemoveContainer" containerID="81d1a1f2e1bae0cbec17ad5dd6d49872601e08b749db1ba942a0ee1a1a9de15e" Sep 17 12:06:50.890800 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:06:50.890782 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81d1a1f2e1bae0cbec17ad5dd6d49872601e08b749db1ba942a0ee1a1a9de15e\": container with ID starting with 81d1a1f2e1bae0cbec17ad5dd6d49872601e08b749db1ba942a0ee1a1a9de15e not found: ID does not exist" containerID="81d1a1f2e1bae0cbec17ad5dd6d49872601e08b749db1ba942a0ee1a1a9de15e" Sep 17 12:06:50.890847 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.890802 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81d1a1f2e1bae0cbec17ad5dd6d49872601e08b749db1ba942a0ee1a1a9de15e"} err="failed to get container status \"81d1a1f2e1bae0cbec17ad5dd6d49872601e08b749db1ba942a0ee1a1a9de15e\": rpc error: code = NotFound desc = could not find container \"81d1a1f2e1bae0cbec17ad5dd6d49872601e08b749db1ba942a0ee1a1a9de15e\": container with ID starting with 81d1a1f2e1bae0cbec17ad5dd6d49872601e08b749db1ba942a0ee1a1a9de15e not found: ID does not exist" Sep 17 12:06:50.890847 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.890813 2581 scope.go:117] "RemoveContainer" containerID="22c1b34fab5b196b15ea76dbf3b352e78efc8408bae308610077865dfe1d8254" Sep 17 12:06:50.890993 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.890978 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22c1b34fab5b196b15ea76dbf3b352e78efc8408bae308610077865dfe1d8254"} err="failed to get container status \"22c1b34fab5b196b15ea76dbf3b352e78efc8408bae308610077865dfe1d8254\": rpc error: code = NotFound desc = could not find container \"22c1b34fab5b196b15ea76dbf3b352e78efc8408bae308610077865dfe1d8254\": container with ID starting with 22c1b34fab5b196b15ea76dbf3b352e78efc8408bae308610077865dfe1d8254 not found: ID does not exist" Sep 17 12:06:50.891041 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.890995 2581 scope.go:117] "RemoveContainer" containerID="425d44844b8ac11fd522dbd6414b2d19e2598af6ca33ae1f8737b9e03809b3e0" Sep 17 12:06:50.891235 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.891219 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"425d44844b8ac11fd522dbd6414b2d19e2598af6ca33ae1f8737b9e03809b3e0"} err="failed to get container status \"425d44844b8ac11fd522dbd6414b2d19e2598af6ca33ae1f8737b9e03809b3e0\": rpc error: code = NotFound desc = could not find container \"425d44844b8ac11fd522dbd6414b2d19e2598af6ca33ae1f8737b9e03809b3e0\": container with ID starting with 425d44844b8ac11fd522dbd6414b2d19e2598af6ca33ae1f8737b9e03809b3e0 not found: ID does not exist" Sep 17 12:06:50.891235 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.891235 2581 scope.go:117] "RemoveContainer" containerID="b4faee45209b8679d7c43921031edbdfc45ced19b4f7061282e0cae26d2f54a8" Sep 17 12:06:50.891463 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.891447 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4faee45209b8679d7c43921031edbdfc45ced19b4f7061282e0cae26d2f54a8"} err="failed to get container status \"b4faee45209b8679d7c43921031edbdfc45ced19b4f7061282e0cae26d2f54a8\": rpc error: code = NotFound desc = could not find container \"b4faee45209b8679d7c43921031edbdfc45ced19b4f7061282e0cae26d2f54a8\": container with ID starting with b4faee45209b8679d7c43921031edbdfc45ced19b4f7061282e0cae26d2f54a8 not found: ID does not exist" Sep 17 12:06:50.891463 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.891463 2581 scope.go:117] "RemoveContainer" containerID="76acc9427ed939a68cb82a1aa254aec5409ac9dabdf5864e39b37f2249566aba" Sep 17 12:06:50.891701 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.891687 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76acc9427ed939a68cb82a1aa254aec5409ac9dabdf5864e39b37f2249566aba"} err="failed to get container status \"76acc9427ed939a68cb82a1aa254aec5409ac9dabdf5864e39b37f2249566aba\": rpc error: code = NotFound desc = could not find container \"76acc9427ed939a68cb82a1aa254aec5409ac9dabdf5864e39b37f2249566aba\": container with ID starting with 76acc9427ed939a68cb82a1aa254aec5409ac9dabdf5864e39b37f2249566aba not found: ID does not exist" Sep 17 12:06:50.891701 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.891701 2581 scope.go:117] "RemoveContainer" containerID="89929aa66c97048e8bfd985a4cdd375ddf0f3278727952cbc0fbedddc0135a17" Sep 17 12:06:50.891879 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.891865 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89929aa66c97048e8bfd985a4cdd375ddf0f3278727952cbc0fbedddc0135a17"} err="failed to get container status \"89929aa66c97048e8bfd985a4cdd375ddf0f3278727952cbc0fbedddc0135a17\": rpc error: code = NotFound desc = could not find container \"89929aa66c97048e8bfd985a4cdd375ddf0f3278727952cbc0fbedddc0135a17\": container with ID starting with 89929aa66c97048e8bfd985a4cdd375ddf0f3278727952cbc0fbedddc0135a17 not found: ID does not exist" Sep 17 12:06:50.891879 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.891880 2581 scope.go:117] "RemoveContainer" containerID="81d1a1f2e1bae0cbec17ad5dd6d49872601e08b749db1ba942a0ee1a1a9de15e" Sep 17 12:06:50.892103 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:50.892088 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81d1a1f2e1bae0cbec17ad5dd6d49872601e08b749db1ba942a0ee1a1a9de15e"} err="failed to get container status \"81d1a1f2e1bae0cbec17ad5dd6d49872601e08b749db1ba942a0ee1a1a9de15e\": rpc error: code = NotFound desc = could not find container \"81d1a1f2e1bae0cbec17ad5dd6d49872601e08b749db1ba942a0ee1a1a9de15e\": container with ID starting with 81d1a1f2e1bae0cbec17ad5dd6d49872601e08b749db1ba942a0ee1a1a9de15e not found: ID does not exist" Sep 17 12:06:52.428422 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:06:52.428391 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" path="/var/lib/kubelet/pods/55ffd1a2-c3c5-4478-aae6-8f322b0d1c17/volumes" Sep 17 12:07:43.261303 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.261263 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk"] Sep 17 12:07:43.261888 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.261747 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="feast-init" Sep 17 12:07:43.261888 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.261766 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="feast-init" Sep 17 12:07:43.261888 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.261789 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="online" Sep 17 12:07:43.261888 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.261798 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="online" Sep 17 12:07:43.261888 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.261813 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="feast-apply" Sep 17 12:07:43.261888 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.261822 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="feast-apply" Sep 17 12:07:43.261888 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.261837 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="offline" Sep 17 12:07:43.261888 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.261845 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="offline" Sep 17 12:07:43.261888 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.261855 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="registry" Sep 17 12:07:43.261888 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.261863 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="registry" Sep 17 12:07:43.261888 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.261887 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="ui" Sep 17 12:07:43.261888 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.261895 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="ui" Sep 17 12:07:43.262543 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.261981 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="ui" Sep 17 12:07:43.262543 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.261995 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="online" Sep 17 12:07:43.262543 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.262006 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="registry" Sep 17 12:07:43.262543 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.262016 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="55ffd1a2-c3c5-4478-aae6-8f322b0d1c17" containerName="offline" Sep 17 12:07:43.265730 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.265708 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:43.268495 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.268471 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 17 12:07:43.268623 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.268549 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 17 12:07:43.268736 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.268722 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 17 12:07:43.268957 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.268939 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 17 12:07:43.269146 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.269132 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-8dwjj\"" Sep 17 12:07:43.269406 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.269391 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 17 12:07:43.269484 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.269435 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 17 12:07:43.280750 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.280732 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk"] Sep 17 12:07:43.327943 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.327917 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/3def2b46-d728-4694-8c34-7ad26bd2c3ae-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wthdk\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:43.328056 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.327956 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wthdk\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:43.328056 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.327992 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wthdk\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:43.328056 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.328026 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrkps\" (UniqueName: \"kubernetes.io/projected/3def2b46-d728-4694-8c34-7ad26bd2c3ae-kube-api-access-wrkps\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wthdk\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:43.328227 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.328061 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3def2b46-d728-4694-8c34-7ad26bd2c3ae-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wthdk\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:43.328227 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.328106 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wthdk\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:43.428637 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.428614 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/3def2b46-d728-4694-8c34-7ad26bd2c3ae-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wthdk\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:43.428739 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.428644 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wthdk\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:43.428739 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.428663 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wthdk\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:43.428739 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.428681 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrkps\" (UniqueName: \"kubernetes.io/projected/3def2b46-d728-4694-8c34-7ad26bd2c3ae-kube-api-access-wrkps\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wthdk\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:43.428908 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:07:43.428767 2581 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 17 12:07:43.428908 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:07:43.428831 2581 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found Sep 17 12:07:43.428908 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:07:43.428841 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-ui-tls podName:3def2b46-d728-4694-8c34-7ad26bd2c3ae nodeName:}" failed. No retries permitted until 2026-09-17 12:07:43.92882005 +0000 UTC m=+1733.936267059 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-ui-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-wthdk" (UID: "3def2b46-d728-4694-8c34-7ad26bd2c3ae") : secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 17 12:07:43.428908 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:07:43.428901 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-offline-tls podName:3def2b46-d728-4694-8c34-7ad26bd2c3ae nodeName:}" failed. No retries permitted until 2026-09-17 12:07:43.928880965 +0000 UTC m=+1733.936327973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-offline-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-wthdk" (UID: "3def2b46-d728-4694-8c34-7ad26bd2c3ae") : secret "feast-simple-feast-remote-setup-offline-tls" not found Sep 17 12:07:43.429148 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.428965 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3def2b46-d728-4694-8c34-7ad26bd2c3ae-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wthdk\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:43.429148 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.429003 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wthdk\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:43.429248 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.429231 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3def2b46-d728-4694-8c34-7ad26bd2c3ae-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wthdk\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:43.429395 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.429376 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/3def2b46-d728-4694-8c34-7ad26bd2c3ae-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wthdk\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:43.431435 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.431411 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wthdk\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:43.441167 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.441143 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrkps\" (UniqueName: \"kubernetes.io/projected/3def2b46-d728-4694-8c34-7ad26bd2c3ae-kube-api-access-wrkps\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wthdk\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:43.932277 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.932248 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wthdk\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:43.932429 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.932285 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wthdk\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:43.934676 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.934654 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wthdk\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:43.934788 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:43.934707 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wthdk\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:44.175877 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:44.175846 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:07:44.458823 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:44.458796 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk"] Sep 17 12:07:44.460365 ip-10-0-140-196 kubenswrapper[2581]: W0917 12:07:44.460338 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3def2b46_d728_4694_8c34_7ad26bd2c3ae.slice/crio-95d328f985940c77afab661b1c1bd7935af7c468a50c212557c03a37326b4798 WatchSource:0}: Error finding container 95d328f985940c77afab661b1c1bd7935af7c468a50c212557c03a37326b4798: Status 404 returned error can't find the container with id 95d328f985940c77afab661b1c1bd7935af7c468a50c212557c03a37326b4798 Sep 17 12:07:45.079032 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:45.078992 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" event={"ID":"3def2b46-d728-4694-8c34-7ad26bd2c3ae","Type":"ContainerStarted","Data":"834048c396cff7e196d478984065417c01a3e81865674fca696666d8f36bf713"} Sep 17 12:07:45.079262 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:45.079041 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" event={"ID":"3def2b46-d728-4694-8c34-7ad26bd2c3ae","Type":"ContainerStarted","Data":"95d328f985940c77afab661b1c1bd7935af7c468a50c212557c03a37326b4798"} Sep 17 12:07:50.104267 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:50.104233 2581 generic.go:356] "Generic (PLEG): container finished" podID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerID="834048c396cff7e196d478984065417c01a3e81865674fca696666d8f36bf713" exitCode=0 Sep 17 12:07:50.104702 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:50.104288 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" event={"ID":"3def2b46-d728-4694-8c34-7ad26bd2c3ae","Type":"ContainerDied","Data":"834048c396cff7e196d478984065417c01a3e81865674fca696666d8f36bf713"} Sep 17 12:07:51.110155 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:07:51.110107 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" event={"ID":"3def2b46-d728-4694-8c34-7ad26bd2c3ae","Type":"ContainerStarted","Data":"57c6aedbd338c2500f855e38a17644c0f966ecd0c7c4d508b7fecec36cf8d0d3"} Sep 17 12:08:13.210929 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:13.210852 2581 generic.go:356] "Generic (PLEG): container finished" podID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerID="57c6aedbd338c2500f855e38a17644c0f966ecd0c7c4d508b7fecec36cf8d0d3" exitCode=0 Sep 17 12:08:13.211300 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:13.210926 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" event={"ID":"3def2b46-d728-4694-8c34-7ad26bd2c3ae","Type":"ContainerDied","Data":"57c6aedbd338c2500f855e38a17644c0f966ecd0c7c4d508b7fecec36cf8d0d3"} Sep 17 12:08:14.219757 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:14.219703 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" event={"ID":"3def2b46-d728-4694-8c34-7ad26bd2c3ae","Type":"ContainerStarted","Data":"6d69ea7b5f3599c0f057ec8a685aa9a199c0b117eccc12ebd09c94a75be98630"} Sep 17 12:08:14.220273 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:14.219776 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" event={"ID":"3def2b46-d728-4694-8c34-7ad26bd2c3ae","Type":"ContainerStarted","Data":"2fcf0478507c1c22d23fc9136d9b77ffc5f6ada83d4037eb643795497954eb60"} Sep 17 12:08:14.220273 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:14.219799 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" event={"ID":"3def2b46-d728-4694-8c34-7ad26bd2c3ae","Type":"ContainerStarted","Data":"96aec1fb8c5689fa4413baafee7c33721ab1f6c2e6c88f9d0a9d090e8078e13a"} Sep 17 12:08:14.250406 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:14.250349 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" podStartSLOduration=31.25032775 podStartE2EDuration="31.25032775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 12:07:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 12:08:14.247798029 +0000 UTC m=+1764.255245087" watchObservedRunningTime="2026-09-17 12:08:14.25032775 +0000 UTC m=+1764.257774780" Sep 17 12:08:17.176511 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:17.176480 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:08:17.177081 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:17.176718 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:08:17.177081 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:17.176743 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:08:17.178633 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:17.178587 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused" Sep 17 12:08:17.178823 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:17.178646 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused" Sep 17 12:08:17.178823 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:17.178596 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.37:8816: connect: connection refused" Sep 17 12:08:20.176743 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:20.176695 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused" Sep 17 12:08:20.177271 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:20.176705 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.37:8816: connect: connection refused" Sep 17 12:08:20.177271 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:20.176709 2581 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused" Sep 17 12:08:23.177398 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:23.177367 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:08:23.178007 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:23.177459 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:08:23.178007 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:23.177803 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:08:23.178007 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:23.177839 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:08:23.178217 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:23.178164 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:08:23.178217 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:23.178217 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:08:23.985883 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:23.985848 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:08:24.176228 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:24.176190 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:08:25.217289 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:25.217240 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 17 12:08:26.222291 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:26.222248 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 17 12:08:27.038038 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:27.038007 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:08:50.491042 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:50.491009 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/2.log" Sep 17 12:08:50.495184 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:08:50.495161 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/2.log" Sep 17 12:09:25.180247 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:25.180207 2581 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 17 12:09:29.353371 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:29.353332 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk"] Sep 17 12:09:29.353861 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:29.353596 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="offline" containerID="cri-o://2fcf0478507c1c22d23fc9136d9b77ffc5f6ada83d4037eb643795497954eb60" gracePeriod=30 Sep 17 12:09:29.353861 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:29.353597 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="ui" containerID="cri-o://6d69ea7b5f3599c0f057ec8a685aa9a199c0b117eccc12ebd09c94a75be98630" gracePeriod=30 Sep 17 12:09:29.353861 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:29.353814 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="online" containerID="cri-o://96aec1fb8c5689fa4413baafee7c33721ab1f6c2e6c88f9d0a9d090e8078e13a" gracePeriod=30 Sep 17 12:09:29.563944 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:29.563910 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn"] Sep 17 12:09:29.564238 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:29.564206 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="registry" containerID="cri-o://dbac67fc14155dad393a5753a66b7794b5d71ba21610a32544b942bb2a32d590" gracePeriod=30 Sep 17 12:09:29.564500 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:29.564256 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="offline" containerID="cri-o://a7e50b1255a3bb32d392f7478a9adce16e95d6cc9d98aebf2a3afc13c810862c" gracePeriod=30 Sep 17 12:09:29.564617 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:29.564286 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="ui" containerID="cri-o://fec21e210ddd7393cd10768aa59b078eb4e177f153d48b2469af49fd02d3917f" gracePeriod=30 Sep 17 12:09:29.564692 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:29.564310 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="online" containerID="cri-o://e89ea3f8edcb8bee7a08924882e2563c5ac56ad3c01040971573fd3b678d60db" gracePeriod=30 Sep 17 12:09:30.559454 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:30.559381 2581 generic.go:356] "Generic (PLEG): container finished" podID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerID="6d69ea7b5f3599c0f057ec8a685aa9a199c0b117eccc12ebd09c94a75be98630" exitCode=0 Sep 17 12:09:30.559454 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:30.559410 2581 generic.go:356] "Generic (PLEG): container finished" podID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerID="96aec1fb8c5689fa4413baafee7c33721ab1f6c2e6c88f9d0a9d090e8078e13a" exitCode=0 Sep 17 12:09:30.559454 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:30.559439 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" event={"ID":"3def2b46-d728-4694-8c34-7ad26bd2c3ae","Type":"ContainerDied","Data":"6d69ea7b5f3599c0f057ec8a685aa9a199c0b117eccc12ebd09c94a75be98630"} Sep 17 12:09:30.559878 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:30.559483 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" event={"ID":"3def2b46-d728-4694-8c34-7ad26bd2c3ae","Type":"ContainerDied","Data":"96aec1fb8c5689fa4413baafee7c33721ab1f6c2e6c88f9d0a9d090e8078e13a"} Sep 17 12:09:30.562062 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:30.562045 2581 generic.go:356] "Generic (PLEG): container finished" podID="e08b89bd-f172-45a7-b468-bf1194e69658" containerID="fec21e210ddd7393cd10768aa59b078eb4e177f153d48b2469af49fd02d3917f" exitCode=0 Sep 17 12:09:30.562062 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:30.562060 2581 generic.go:356] "Generic (PLEG): container finished" podID="e08b89bd-f172-45a7-b468-bf1194e69658" containerID="e89ea3f8edcb8bee7a08924882e2563c5ac56ad3c01040971573fd3b678d60db" exitCode=0 Sep 17 12:09:30.562219 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:30.562139 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" event={"ID":"e08b89bd-f172-45a7-b468-bf1194e69658","Type":"ContainerDied","Data":"fec21e210ddd7393cd10768aa59b078eb4e177f153d48b2469af49fd02d3917f"} Sep 17 12:09:30.562219 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:30.562179 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" event={"ID":"e08b89bd-f172-45a7-b468-bf1194e69658","Type":"ContainerDied","Data":"e89ea3f8edcb8bee7a08924882e2563c5ac56ad3c01040971573fd3b678d60db"} Sep 17 12:09:31.756628 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:31.756583 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused" Sep 17 12:09:31.792644 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:31.792606 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused" Sep 17 12:09:33.178682 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:33.178638 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused" Sep 17 12:09:36.222727 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:36.222692 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused" Sep 17 12:09:41.756198 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:41.756147 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused" Sep 17 12:09:41.792486 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:41.792453 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused" Sep 17 12:09:43.178703 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:43.178646 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused" Sep 17 12:09:46.223718 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:46.223678 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused" Sep 17 12:09:51.756315 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:51.756269 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused" Sep 17 12:09:51.756769 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:51.756413 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:09:51.792041 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:51.792000 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused" Sep 17 12:09:51.792229 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:51.792156 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:09:53.178446 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:53.178404 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused" Sep 17 12:09:53.178891 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:53.178531 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:09:56.222840 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:56.222802 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused" Sep 17 12:09:56.223272 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:56.222909 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:09:59.537244 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.537221 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:09:59.647710 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.647635 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/3def2b46-d728-4694-8c34-7ad26bd2c3ae-registry-tls\") pod \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " Sep 17 12:09:59.647710 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.647672 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-offline-tls\") pod \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " Sep 17 12:09:59.647901 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.647724 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-online-tls\") pod \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " Sep 17 12:09:59.647901 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.647750 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3def2b46-d728-4694-8c34-7ad26bd2c3ae-feast-data\") pod \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " Sep 17 12:09:59.647901 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.647781 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrkps\" (UniqueName: \"kubernetes.io/projected/3def2b46-d728-4694-8c34-7ad26bd2c3ae-kube-api-access-wrkps\") pod \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " Sep 17 12:09:59.647901 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.647811 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-ui-tls\") pod \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\" (UID: \"3def2b46-d728-4694-8c34-7ad26bd2c3ae\") " Sep 17 12:09:59.648150 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.648037 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3def2b46-d728-4694-8c34-7ad26bd2c3ae-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3def2b46-d728-4694-8c34-7ad26bd2c3ae" (UID: "3def2b46-d728-4694-8c34-7ad26bd2c3ae"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 12:09:59.648317 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.648283 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3def2b46-d728-4694-8c34-7ad26bd2c3ae-feast-data" (OuterVolumeSpecName: "feast-data") pod "3def2b46-d728-4694-8c34-7ad26bd2c3ae" (UID: "3def2b46-d728-4694-8c34-7ad26bd2c3ae"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 12:09:59.650034 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.650012 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-online-tls" (OuterVolumeSpecName: "online-tls") pod "3def2b46-d728-4694-8c34-7ad26bd2c3ae" (UID: "3def2b46-d728-4694-8c34-7ad26bd2c3ae"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:09:59.650245 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.650100 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "3def2b46-d728-4694-8c34-7ad26bd2c3ae" (UID: "3def2b46-d728-4694-8c34-7ad26bd2c3ae"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:09:59.650448 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.650425 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "3def2b46-d728-4694-8c34-7ad26bd2c3ae" (UID: "3def2b46-d728-4694-8c34-7ad26bd2c3ae"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:09:59.650494 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.650441 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3def2b46-d728-4694-8c34-7ad26bd2c3ae-kube-api-access-wrkps" (OuterVolumeSpecName: "kube-api-access-wrkps") pod "3def2b46-d728-4694-8c34-7ad26bd2c3ae" (UID: "3def2b46-d728-4694-8c34-7ad26bd2c3ae"). InnerVolumeSpecName "kube-api-access-wrkps". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 12:09:59.689940 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.689906 2581 generic.go:356] "Generic (PLEG): container finished" podID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerID="2fcf0478507c1c22d23fc9136d9b77ffc5f6ada83d4037eb643795497954eb60" exitCode=137 Sep 17 12:09:59.690104 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.689977 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" Sep 17 12:09:59.690104 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.689984 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" event={"ID":"3def2b46-d728-4694-8c34-7ad26bd2c3ae","Type":"ContainerDied","Data":"2fcf0478507c1c22d23fc9136d9b77ffc5f6ada83d4037eb643795497954eb60"} Sep 17 12:09:59.690104 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.690035 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk" event={"ID":"3def2b46-d728-4694-8c34-7ad26bd2c3ae","Type":"ContainerDied","Data":"95d328f985940c77afab661b1c1bd7935af7c468a50c212557c03a37326b4798"} Sep 17 12:09:59.690104 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.690061 2581 scope.go:117] "RemoveContainer" containerID="6d69ea7b5f3599c0f057ec8a685aa9a199c0b117eccc12ebd09c94a75be98630" Sep 17 12:09:59.693536 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.693503 2581 generic.go:356] "Generic (PLEG): container finished" podID="e08b89bd-f172-45a7-b468-bf1194e69658" containerID="a7e50b1255a3bb32d392f7478a9adce16e95d6cc9d98aebf2a3afc13c810862c" exitCode=137 Sep 17 12:09:59.693659 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.693538 2581 generic.go:356] "Generic (PLEG): container finished" podID="e08b89bd-f172-45a7-b468-bf1194e69658" containerID="dbac67fc14155dad393a5753a66b7794b5d71ba21610a32544b942bb2a32d590" exitCode=137 Sep 17 12:09:59.693659 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.693536 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" event={"ID":"e08b89bd-f172-45a7-b468-bf1194e69658","Type":"ContainerDied","Data":"a7e50b1255a3bb32d392f7478a9adce16e95d6cc9d98aebf2a3afc13c810862c"} Sep 17 12:09:59.693659 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.693580 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" event={"ID":"e08b89bd-f172-45a7-b468-bf1194e69658","Type":"ContainerDied","Data":"dbac67fc14155dad393a5753a66b7794b5d71ba21610a32544b942bb2a32d590"} Sep 17 12:09:59.699930 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.699914 2581 scope.go:117] "RemoveContainer" containerID="2fcf0478507c1c22d23fc9136d9b77ffc5f6ada83d4037eb643795497954eb60" Sep 17 12:09:59.708085 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.708067 2581 scope.go:117] "RemoveContainer" containerID="96aec1fb8c5689fa4413baafee7c33721ab1f6c2e6c88f9d0a9d090e8078e13a" Sep 17 12:09:59.713345 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.713322 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk"] Sep 17 12:09:59.716950 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.716931 2581 scope.go:117] "RemoveContainer" containerID="57c6aedbd338c2500f855e38a17644c0f966ecd0c7c4d508b7fecec36cf8d0d3" Sep 17 12:09:59.718760 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.718739 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wthdk"] Sep 17 12:09:59.724818 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.724799 2581 scope.go:117] "RemoveContainer" containerID="834048c396cff7e196d478984065417c01a3e81865674fca696666d8f36bf713" Sep 17 12:09:59.734271 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.733052 2581 scope.go:117] "RemoveContainer" containerID="6d69ea7b5f3599c0f057ec8a685aa9a199c0b117eccc12ebd09c94a75be98630" Sep 17 12:09:59.734766 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:09:59.734741 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d69ea7b5f3599c0f057ec8a685aa9a199c0b117eccc12ebd09c94a75be98630\": container with ID starting with 6d69ea7b5f3599c0f057ec8a685aa9a199c0b117eccc12ebd09c94a75be98630 not found: ID does not exist" containerID="6d69ea7b5f3599c0f057ec8a685aa9a199c0b117eccc12ebd09c94a75be98630" Sep 17 12:09:59.734820 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.734778 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d69ea7b5f3599c0f057ec8a685aa9a199c0b117eccc12ebd09c94a75be98630"} err="failed to get container status \"6d69ea7b5f3599c0f057ec8a685aa9a199c0b117eccc12ebd09c94a75be98630\": rpc error: code = NotFound desc = could not find container \"6d69ea7b5f3599c0f057ec8a685aa9a199c0b117eccc12ebd09c94a75be98630\": container with ID starting with 6d69ea7b5f3599c0f057ec8a685aa9a199c0b117eccc12ebd09c94a75be98630 not found: ID does not exist" Sep 17 12:09:59.734820 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.734806 2581 scope.go:117] "RemoveContainer" containerID="2fcf0478507c1c22d23fc9136d9b77ffc5f6ada83d4037eb643795497954eb60" Sep 17 12:09:59.735107 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:09:59.735089 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fcf0478507c1c22d23fc9136d9b77ffc5f6ada83d4037eb643795497954eb60\": container with ID starting with 2fcf0478507c1c22d23fc9136d9b77ffc5f6ada83d4037eb643795497954eb60 not found: ID does not exist" containerID="2fcf0478507c1c22d23fc9136d9b77ffc5f6ada83d4037eb643795497954eb60" Sep 17 12:09:59.735179 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.735137 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fcf0478507c1c22d23fc9136d9b77ffc5f6ada83d4037eb643795497954eb60"} err="failed to get container status \"2fcf0478507c1c22d23fc9136d9b77ffc5f6ada83d4037eb643795497954eb60\": rpc error: code = NotFound desc = could not find container \"2fcf0478507c1c22d23fc9136d9b77ffc5f6ada83d4037eb643795497954eb60\": container with ID starting with 2fcf0478507c1c22d23fc9136d9b77ffc5f6ada83d4037eb643795497954eb60 not found: ID does not exist" Sep 17 12:09:59.735179 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.735157 2581 scope.go:117] "RemoveContainer" containerID="96aec1fb8c5689fa4413baafee7c33721ab1f6c2e6c88f9d0a9d090e8078e13a" Sep 17 12:09:59.735433 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:09:59.735413 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96aec1fb8c5689fa4413baafee7c33721ab1f6c2e6c88f9d0a9d090e8078e13a\": container with ID starting with 96aec1fb8c5689fa4413baafee7c33721ab1f6c2e6c88f9d0a9d090e8078e13a not found: ID does not exist" containerID="96aec1fb8c5689fa4413baafee7c33721ab1f6c2e6c88f9d0a9d090e8078e13a" Sep 17 12:09:59.735489 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.735439 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96aec1fb8c5689fa4413baafee7c33721ab1f6c2e6c88f9d0a9d090e8078e13a"} err="failed to get container status \"96aec1fb8c5689fa4413baafee7c33721ab1f6c2e6c88f9d0a9d090e8078e13a\": rpc error: code = NotFound desc = could not find container \"96aec1fb8c5689fa4413baafee7c33721ab1f6c2e6c88f9d0a9d090e8078e13a\": container with ID starting with 96aec1fb8c5689fa4413baafee7c33721ab1f6c2e6c88f9d0a9d090e8078e13a not found: ID does not exist" Sep 17 12:09:59.735489 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.735458 2581 scope.go:117] "RemoveContainer" containerID="57c6aedbd338c2500f855e38a17644c0f966ecd0c7c4d508b7fecec36cf8d0d3" Sep 17 12:09:59.735712 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:09:59.735693 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57c6aedbd338c2500f855e38a17644c0f966ecd0c7c4d508b7fecec36cf8d0d3\": container with ID starting with 57c6aedbd338c2500f855e38a17644c0f966ecd0c7c4d508b7fecec36cf8d0d3 not found: ID does not exist" containerID="57c6aedbd338c2500f855e38a17644c0f966ecd0c7c4d508b7fecec36cf8d0d3" Sep 17 12:09:59.735778 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.735718 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57c6aedbd338c2500f855e38a17644c0f966ecd0c7c4d508b7fecec36cf8d0d3"} err="failed to get container status \"57c6aedbd338c2500f855e38a17644c0f966ecd0c7c4d508b7fecec36cf8d0d3\": rpc error: code = NotFound desc = could not find container \"57c6aedbd338c2500f855e38a17644c0f966ecd0c7c4d508b7fecec36cf8d0d3\": container with ID starting with 57c6aedbd338c2500f855e38a17644c0f966ecd0c7c4d508b7fecec36cf8d0d3 not found: ID does not exist" Sep 17 12:09:59.735778 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.735740 2581 scope.go:117] "RemoveContainer" containerID="834048c396cff7e196d478984065417c01a3e81865674fca696666d8f36bf713" Sep 17 12:09:59.735985 ip-10-0-140-196 kubenswrapper[2581]: E0917 12:09:59.735970 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"834048c396cff7e196d478984065417c01a3e81865674fca696666d8f36bf713\": container with ID starting with 834048c396cff7e196d478984065417c01a3e81865674fca696666d8f36bf713 not found: ID does not exist" containerID="834048c396cff7e196d478984065417c01a3e81865674fca696666d8f36bf713" Sep 17 12:09:59.736036 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.735988 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"834048c396cff7e196d478984065417c01a3e81865674fca696666d8f36bf713"} err="failed to get container status \"834048c396cff7e196d478984065417c01a3e81865674fca696666d8f36bf713\": rpc error: code = NotFound desc = could not find container \"834048c396cff7e196d478984065417c01a3e81865674fca696666d8f36bf713\": container with ID starting with 834048c396cff7e196d478984065417c01a3e81865674fca696666d8f36bf713 not found: ID does not exist" Sep 17 12:09:59.749441 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.749413 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/3def2b46-d728-4694-8c34-7ad26bd2c3ae-registry-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:09:59.749441 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.749439 2581 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-offline-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:09:59.749578 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.749479 2581 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-online-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:09:59.749578 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.749493 2581 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3def2b46-d728-4694-8c34-7ad26bd2c3ae-feast-data\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:09:59.749578 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.749506 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrkps\" (UniqueName: \"kubernetes.io/projected/3def2b46-d728-4694-8c34-7ad26bd2c3ae-kube-api-access-wrkps\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:09:59.749578 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:09:59.749518 2581 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3def2b46-d728-4694-8c34-7ad26bd2c3ae-ui-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:10:00.215470 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.215439 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:10:00.254212 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.254181 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e08b89bd-f172-45a7-b468-bf1194e69658-feast-data\") pod \"e08b89bd-f172-45a7-b468-bf1194e69658\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " Sep 17 12:10:00.254404 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.254219 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-ui-tls\") pod \"e08b89bd-f172-45a7-b468-bf1194e69658\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " Sep 17 12:10:00.254404 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.254260 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6tgx\" (UniqueName: \"kubernetes.io/projected/e08b89bd-f172-45a7-b468-bf1194e69658-kube-api-access-l6tgx\") pod \"e08b89bd-f172-45a7-b468-bf1194e69658\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " Sep 17 12:10:00.254404 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.254285 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-online-tls\") pod \"e08b89bd-f172-45a7-b468-bf1194e69658\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " Sep 17 12:10:00.254404 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.254326 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-offline-tls\") pod \"e08b89bd-f172-45a7-b468-bf1194e69658\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " Sep 17 12:10:00.254624 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.254401 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-registry-tls\") pod \"e08b89bd-f172-45a7-b468-bf1194e69658\" (UID: \"e08b89bd-f172-45a7-b468-bf1194e69658\") " Sep 17 12:10:00.254862 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.254816 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e08b89bd-f172-45a7-b468-bf1194e69658-feast-data" (OuterVolumeSpecName: "feast-data") pod "e08b89bd-f172-45a7-b468-bf1194e69658" (UID: "e08b89bd-f172-45a7-b468-bf1194e69658"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 12:10:00.256726 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.256689 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-online-tls" (OuterVolumeSpecName: "online-tls") pod "e08b89bd-f172-45a7-b468-bf1194e69658" (UID: "e08b89bd-f172-45a7-b468-bf1194e69658"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:10:00.256853 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.256725 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e08b89bd-f172-45a7-b468-bf1194e69658" (UID: "e08b89bd-f172-45a7-b468-bf1194e69658"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:10:00.257029 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.257008 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "e08b89bd-f172-45a7-b468-bf1194e69658" (UID: "e08b89bd-f172-45a7-b468-bf1194e69658"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:10:00.257158 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.257054 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e08b89bd-f172-45a7-b468-bf1194e69658-kube-api-access-l6tgx" (OuterVolumeSpecName: "kube-api-access-l6tgx") pod "e08b89bd-f172-45a7-b468-bf1194e69658" (UID: "e08b89bd-f172-45a7-b468-bf1194e69658"). InnerVolumeSpecName "kube-api-access-l6tgx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 12:10:00.257226 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.257155 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "e08b89bd-f172-45a7-b468-bf1194e69658" (UID: "e08b89bd-f172-45a7-b468-bf1194e69658"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 12:10:00.356068 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.355977 2581 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e08b89bd-f172-45a7-b468-bf1194e69658-feast-data\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:10:00.356068 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.356005 2581 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-ui-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:10:00.356068 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.356018 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6tgx\" (UniqueName: \"kubernetes.io/projected/e08b89bd-f172-45a7-b468-bf1194e69658-kube-api-access-l6tgx\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:10:00.356068 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.356031 2581 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-online-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:10:00.356068 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.356057 2581 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-offline-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:10:00.356068 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.356069 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e08b89bd-f172-45a7-b468-bf1194e69658-registry-tls\") on node \"ip-10-0-140-196.ec2.internal\" DevicePath \"\"" Sep 17 12:10:00.428961 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.428928 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" path="/var/lib/kubelet/pods/3def2b46-d728-4694-8c34-7ad26bd2c3ae/volumes" Sep 17 12:10:00.702471 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.702384 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" event={"ID":"e08b89bd-f172-45a7-b468-bf1194e69658","Type":"ContainerDied","Data":"ed776fb6217249420dec2143bb3a840f6d60af341ebfe1eef20f2f17dca8e487"} Sep 17 12:10:00.702471 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.702441 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn" Sep 17 12:10:00.703111 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.702442 2581 scope.go:117] "RemoveContainer" containerID="fec21e210ddd7393cd10768aa59b078eb4e177f153d48b2469af49fd02d3917f" Sep 17 12:10:00.711287 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.711168 2581 scope.go:117] "RemoveContainer" containerID="a7e50b1255a3bb32d392f7478a9adce16e95d6cc9d98aebf2a3afc13c810862c" Sep 17 12:10:00.718767 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.718751 2581 scope.go:117] "RemoveContainer" containerID="e89ea3f8edcb8bee7a08924882e2563c5ac56ad3c01040971573fd3b678d60db" Sep 17 12:10:00.721679 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.721657 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn"] Sep 17 12:10:00.726205 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.726181 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-rw5hn"] Sep 17 12:10:00.728449 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.728432 2581 scope.go:117] "RemoveContainer" containerID="dbac67fc14155dad393a5753a66b7794b5d71ba21610a32544b942bb2a32d590" Sep 17 12:10:00.735972 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.735954 2581 scope.go:117] "RemoveContainer" containerID="692668d9343d81d28b189634b7fa59b3da025efe6fc7af77a4e2e59cc63370e4" Sep 17 12:10:00.743267 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:00.743246 2581 scope.go:117] "RemoveContainer" containerID="a82841caae4199c802a4a800bbc9d29ede333a5d932c1d30e001e28830804efe" Sep 17 12:10:02.428504 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:02.428466 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" path="/var/lib/kubelet/pods/e08b89bd-f172-45a7-b468-bf1194e69658/volumes" Sep 17 12:10:37.181607 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.181572 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8xmpg/must-gather-8pvh2"] Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.181895 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="feast-init" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.181906 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="feast-init" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.181915 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="registry" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.181920 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="registry" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.181929 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="offline" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.181934 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="offline" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.181941 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="ui" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.181946 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="ui" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.181959 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="feast-init" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.181964 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="feast-init" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.181972 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="online" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.181977 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="online" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.181983 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="offline" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.181988 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="offline" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.181994 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="feast-apply" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.181999 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="feast-apply" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.182006 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="online" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.182012 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="online" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.182022 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="feast-apply" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.182027 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="feast-apply" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.182032 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="ui" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.182036 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="ui" Sep 17 12:10:37.182080 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.182088 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="online" Sep 17 12:10:37.182967 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.182095 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="ui" Sep 17 12:10:37.182967 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.182101 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="registry" Sep 17 12:10:37.182967 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.182106 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="ui" Sep 17 12:10:37.182967 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.182112 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3def2b46-d728-4694-8c34-7ad26bd2c3ae" containerName="offline" Sep 17 12:10:37.182967 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.182136 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="offline" Sep 17 12:10:37.182967 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.182141 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e08b89bd-f172-45a7-b468-bf1194e69658" containerName="online" Sep 17 12:10:37.185233 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.185216 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8xmpg/must-gather-8pvh2" Sep 17 12:10:37.188013 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.187987 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8xmpg\"/\"default-dockercfg-vckhs\"" Sep 17 12:10:37.188168 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.188022 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8xmpg\"/\"kube-root-ca.crt\"" Sep 17 12:10:37.188168 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.188027 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8xmpg\"/\"openshift-service-ca.crt\"" Sep 17 12:10:37.194602 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.194574 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8xmpg/must-gather-8pvh2"] Sep 17 12:10:37.335989 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.335961 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9d69d5b0-6a2a-43f4-b8f0-b355379ecb5e-must-gather-output\") pod \"must-gather-8pvh2\" (UID: \"9d69d5b0-6a2a-43f4-b8f0-b355379ecb5e\") " pod="openshift-must-gather-8xmpg/must-gather-8pvh2" Sep 17 12:10:37.336103 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.336016 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4sxr\" (UniqueName: \"kubernetes.io/projected/9d69d5b0-6a2a-43f4-b8f0-b355379ecb5e-kube-api-access-l4sxr\") pod \"must-gather-8pvh2\" (UID: \"9d69d5b0-6a2a-43f4-b8f0-b355379ecb5e\") " pod="openshift-must-gather-8xmpg/must-gather-8pvh2" Sep 17 12:10:37.437110 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.437049 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4sxr\" (UniqueName: \"kubernetes.io/projected/9d69d5b0-6a2a-43f4-b8f0-b355379ecb5e-kube-api-access-l4sxr\") pod \"must-gather-8pvh2\" (UID: \"9d69d5b0-6a2a-43f4-b8f0-b355379ecb5e\") " pod="openshift-must-gather-8xmpg/must-gather-8pvh2" Sep 17 12:10:37.437110 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.437105 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9d69d5b0-6a2a-43f4-b8f0-b355379ecb5e-must-gather-output\") pod \"must-gather-8pvh2\" (UID: \"9d69d5b0-6a2a-43f4-b8f0-b355379ecb5e\") " pod="openshift-must-gather-8xmpg/must-gather-8pvh2" Sep 17 12:10:37.437389 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.437370 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9d69d5b0-6a2a-43f4-b8f0-b355379ecb5e-must-gather-output\") pod \"must-gather-8pvh2\" (UID: \"9d69d5b0-6a2a-43f4-b8f0-b355379ecb5e\") " pod="openshift-must-gather-8xmpg/must-gather-8pvh2" Sep 17 12:10:37.446504 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.446481 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4sxr\" (UniqueName: \"kubernetes.io/projected/9d69d5b0-6a2a-43f4-b8f0-b355379ecb5e-kube-api-access-l4sxr\") pod \"must-gather-8pvh2\" (UID: \"9d69d5b0-6a2a-43f4-b8f0-b355379ecb5e\") " pod="openshift-must-gather-8xmpg/must-gather-8pvh2" Sep 17 12:10:37.494508 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.494490 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8xmpg/must-gather-8pvh2" Sep 17 12:10:37.764583 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.764536 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8xmpg/must-gather-8pvh2"] Sep 17 12:10:37.766997 ip-10-0-140-196 kubenswrapper[2581]: W0917 12:10:37.766967 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d69d5b0_6a2a_43f4_b8f0_b355379ecb5e.slice/crio-10226423d5ec281156e9523fa6e869454c525d5b66a5671a57a6b242721432aa WatchSource:0}: Error finding container 10226423d5ec281156e9523fa6e869454c525d5b66a5671a57a6b242721432aa: Status 404 returned error can't find the container with id 10226423d5ec281156e9523fa6e869454c525d5b66a5671a57a6b242721432aa Sep 17 12:10:37.768899 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.768880 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 12:10:37.861978 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:37.861954 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8xmpg/must-gather-8pvh2" event={"ID":"9d69d5b0-6a2a-43f4-b8f0-b355379ecb5e","Type":"ContainerStarted","Data":"10226423d5ec281156e9523fa6e869454c525d5b66a5671a57a6b242721432aa"} Sep 17 12:10:39.873554 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:39.873486 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8xmpg/must-gather-8pvh2" event={"ID":"9d69d5b0-6a2a-43f4-b8f0-b355379ecb5e","Type":"ContainerStarted","Data":"f9ac887095fc2573f8331f91920853b82f9324a63c7e84d08632fc181fb3026d"} Sep 17 12:10:39.873554 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:39.873542 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8xmpg/must-gather-8pvh2" event={"ID":"9d69d5b0-6a2a-43f4-b8f0-b355379ecb5e","Type":"ContainerStarted","Data":"1b5c2112a5acd12cb3cc9dd3f599de023a79f8b5f09880b5facdb5e57a6f7460"} Sep 17 12:10:39.889419 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:39.889366 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8xmpg/must-gather-8pvh2" podStartSLOduration=1.204046028 podStartE2EDuration="2.889346619s" totalImagesPullingTime="1.685300591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 12:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 12:10:39.889150784 +0000 UTC m=+1909.896597809" watchObservedRunningTime="2026-09-17 12:10:39.889346619 +0000 UTC m=+1909.896793650" Sep 17 12:10:40.858708 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:40.858675 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6jmh4_7a49d705-8c21-411b-8988-260b0158a9be/global-pull-secret-syncer/0.log" Sep 17 12:10:41.068731 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:41.068703 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-z8klg_17023d9f-cb91-44aa-a873-3c5218fa56ea/konnectivity-agent/0.log" Sep 17 12:10:41.169592 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:41.169525 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-196.ec2.internal_e5a272a923bce547f142a7efecbeccfb/haproxy/0.log" Sep 17 12:10:44.738517 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:44.738478 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6c914b05-45d8-4851-bdb6-bff91edf3780/alertmanager/0.log" Sep 17 12:10:44.764537 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:44.764500 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6c914b05-45d8-4851-bdb6-bff91edf3780/config-reloader/0.log" Sep 17 12:10:44.787184 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:44.787146 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6c914b05-45d8-4851-bdb6-bff91edf3780/kube-rbac-proxy-web/0.log" Sep 17 12:10:44.810282 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:44.810228 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6c914b05-45d8-4851-bdb6-bff91edf3780/kube-rbac-proxy/0.log" Sep 17 12:10:44.839055 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:44.839028 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6c914b05-45d8-4851-bdb6-bff91edf3780/kube-rbac-proxy-metric/0.log" Sep 17 12:10:44.861407 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:44.861371 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6c914b05-45d8-4851-bdb6-bff91edf3780/prom-label-proxy/0.log" Sep 17 12:10:44.890471 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:44.890427 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6c914b05-45d8-4851-bdb6-bff91edf3780/init-config-reloader/0.log" Sep 17 12:10:44.992301 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:44.992196 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-2ptb8_10929b4d-9cbb-45fd-92ee-427667b142c6/kube-state-metrics/0.log" Sep 17 12:10:45.040024 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:45.039990 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-2ptb8_10929b4d-9cbb-45fd-92ee-427667b142c6/kube-rbac-proxy-main/0.log" Sep 17 12:10:45.073512 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:45.073466 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-2ptb8_10929b4d-9cbb-45fd-92ee-427667b142c6/kube-rbac-proxy-self/0.log" Sep 17 12:10:45.145616 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:45.145563 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-nxcqh_3828af14-839e-4afd-908b-ebeb82098f99/monitoring-plugin/0.log" Sep 17 12:10:45.331787 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:45.331729 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pmps2_ddf4132f-a985-4fa4-96dc-1a2d4e0d810e/node-exporter/0.log" Sep 17 12:10:45.351846 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:45.351808 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pmps2_ddf4132f-a985-4fa4-96dc-1a2d4e0d810e/kube-rbac-proxy/0.log" Sep 17 12:10:45.373552 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:45.373532 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pmps2_ddf4132f-a985-4fa4-96dc-1a2d4e0d810e/init-textfile/0.log" Sep 17 12:10:45.768681 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:45.768645 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-687c687b86-k2dpc_73902f0f-5a1d-4102-b852-7795cae6647a/telemeter-client/0.log" Sep 17 12:10:45.795782 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:45.795751 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-687c687b86-k2dpc_73902f0f-5a1d-4102-b852-7795cae6647a/reload/0.log" Sep 17 12:10:45.821392 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:45.821361 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-687c687b86-k2dpc_73902f0f-5a1d-4102-b852-7795cae6647a/kube-rbac-proxy/0.log" Sep 17 12:10:45.862177 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:45.862141 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b877f875b-r8bm6_b3fa9a70-d6af-4872-a2af-49b83baff353/thanos-query/0.log" Sep 17 12:10:45.892822 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:45.892789 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b877f875b-r8bm6_b3fa9a70-d6af-4872-a2af-49b83baff353/kube-rbac-proxy-web/0.log" Sep 17 12:10:45.926703 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:45.926678 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b877f875b-r8bm6_b3fa9a70-d6af-4872-a2af-49b83baff353/kube-rbac-proxy/0.log" Sep 17 12:10:45.953767 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:45.953744 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b877f875b-r8bm6_b3fa9a70-d6af-4872-a2af-49b83baff353/prom-label-proxy/0.log" Sep 17 12:10:45.981551 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:45.981526 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b877f875b-r8bm6_b3fa9a70-d6af-4872-a2af-49b83baff353/kube-rbac-proxy-rules/0.log" Sep 17 12:10:46.010792 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:46.010760 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b877f875b-r8bm6_b3fa9a70-d6af-4872-a2af-49b83baff353/kube-rbac-proxy-metrics/0.log" Sep 17 12:10:47.502449 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:47.502422 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/2.log" Sep 17 12:10:47.507379 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:47.507356 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gn9sh_124b0387-2e6e-4f73-b71a-2ead313e3701/console-operator/3.log" Sep 17 12:10:47.853531 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:47.853458 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-75bfb5fc9-gzdjt_cce32baa-7461-40f2-98cd-84c30f043a53/console/0.log" Sep 17 12:10:47.884024 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:47.883998 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-jbvnp_9ef7feb3-5cc2-4447-9581-29156afc6ca7/download-server/0.log" Sep 17 12:10:48.320801 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:48.320769 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-hkhqg_17c82812-30fc-48a1-b945-d0708daa7629/volume-data-source-validator/0.log" Sep 17 12:10:49.010948 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.010925 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8b6dg_a0d623f4-748b-4d82-9a70-41671b79d346/dns/0.log" Sep 17 12:10:49.031642 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.031620 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8b6dg_a0d623f4-748b-4d82-9a70-41671b79d346/kube-rbac-proxy/0.log" Sep 17 12:10:49.195866 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.195841 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wk6sh_e1152cde-f077-4e81-a902-d0cd724f0495/dns-node-resolver/0.log" Sep 17 12:10:49.639769 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.639741 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g6kpv_67153d8b-662e-4d11-b932-2ec9f96be172/node-ca/0.log" Sep 17 12:10:49.658065 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.658034 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w"] Sep 17 12:10:49.664909 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.664886 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" Sep 17 12:10:49.674035 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.674011 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w"] Sep 17 12:10:49.749858 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.749826 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5f055390-07ff-4d84-8557-1649931671c9-proc\") pod \"perf-node-gather-daemonset-fft4w\" (UID: \"5f055390-07ff-4d84-8557-1649931671c9\") " pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" Sep 17 12:10:49.750142 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.750094 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5f055390-07ff-4d84-8557-1649931671c9-podres\") pod \"perf-node-gather-daemonset-fft4w\" (UID: \"5f055390-07ff-4d84-8557-1649931671c9\") " pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" Sep 17 12:10:49.750322 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.750305 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f055390-07ff-4d84-8557-1649931671c9-sys\") pod \"perf-node-gather-daemonset-fft4w\" (UID: \"5f055390-07ff-4d84-8557-1649931671c9\") " pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" Sep 17 12:10:49.750483 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.750466 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mdw8\" (UniqueName: \"kubernetes.io/projected/5f055390-07ff-4d84-8557-1649931671c9-kube-api-access-2mdw8\") pod \"perf-node-gather-daemonset-fft4w\" (UID: \"5f055390-07ff-4d84-8557-1649931671c9\") " pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" Sep 17 12:10:49.750666 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.750649 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5f055390-07ff-4d84-8557-1649931671c9-lib-modules\") pod \"perf-node-gather-daemonset-fft4w\" (UID: \"5f055390-07ff-4d84-8557-1649931671c9\") " pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" Sep 17 12:10:49.852017 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.851986 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5f055390-07ff-4d84-8557-1649931671c9-lib-modules\") pod \"perf-node-gather-daemonset-fft4w\" (UID: \"5f055390-07ff-4d84-8557-1649931671c9\") " pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" Sep 17 12:10:49.852196 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.852168 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5f055390-07ff-4d84-8557-1649931671c9-lib-modules\") pod \"perf-node-gather-daemonset-fft4w\" (UID: \"5f055390-07ff-4d84-8557-1649931671c9\") " pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" Sep 17 12:10:49.852281 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.852215 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5f055390-07ff-4d84-8557-1649931671c9-proc\") pod \"perf-node-gather-daemonset-fft4w\" (UID: \"5f055390-07ff-4d84-8557-1649931671c9\") " pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" Sep 17 12:10:49.852281 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.852255 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5f055390-07ff-4d84-8557-1649931671c9-podres\") pod \"perf-node-gather-daemonset-fft4w\" (UID: \"5f055390-07ff-4d84-8557-1649931671c9\") " pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" Sep 17 12:10:49.852394 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.852296 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f055390-07ff-4d84-8557-1649931671c9-sys\") pod \"perf-node-gather-daemonset-fft4w\" (UID: \"5f055390-07ff-4d84-8557-1649931671c9\") " pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" Sep 17 12:10:49.852394 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.852323 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5f055390-07ff-4d84-8557-1649931671c9-proc\") pod \"perf-node-gather-daemonset-fft4w\" (UID: \"5f055390-07ff-4d84-8557-1649931671c9\") " pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" Sep 17 12:10:49.852394 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.852324 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mdw8\" (UniqueName: \"kubernetes.io/projected/5f055390-07ff-4d84-8557-1649931671c9-kube-api-access-2mdw8\") pod \"perf-node-gather-daemonset-fft4w\" (UID: \"5f055390-07ff-4d84-8557-1649931671c9\") " pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" Sep 17 12:10:49.852537 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.852405 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f055390-07ff-4d84-8557-1649931671c9-sys\") pod \"perf-node-gather-daemonset-fft4w\" (UID: \"5f055390-07ff-4d84-8557-1649931671c9\") " pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" Sep 17 12:10:49.852537 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.852425 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5f055390-07ff-4d84-8557-1649931671c9-podres\") pod \"perf-node-gather-daemonset-fft4w\" (UID: \"5f055390-07ff-4d84-8557-1649931671c9\") " pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" Sep 17 12:10:49.860756 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.860737 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mdw8\" (UniqueName: \"kubernetes.io/projected/5f055390-07ff-4d84-8557-1649931671c9-kube-api-access-2mdw8\") pod \"perf-node-gather-daemonset-fft4w\" (UID: \"5f055390-07ff-4d84-8557-1649931671c9\") " pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" Sep 17 12:10:49.975927 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:49.975849 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" Sep 17 12:10:50.262462 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:50.256827 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w"] Sep 17 12:10:50.699110 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:50.699083 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nqcnx_d4657dd9-0b00-4a86-af8e-10392c1704f9/serve-healthcheck-canary/0.log" Sep 17 12:10:50.943305 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:50.943230 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" event={"ID":"5f055390-07ff-4d84-8557-1649931671c9","Type":"ContainerStarted","Data":"801e566687eeb20a8b1f89acf808c25c97e76df668a0fd7d255eb04146043398"} Sep 17 12:10:50.943305 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:50.943285 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" event={"ID":"5f055390-07ff-4d84-8557-1649931671c9","Type":"ContainerStarted","Data":"31da4e6b6a19b09f5bc654f641bf45b8d35bb36d6b935f72858e051f3f51feb1"} Sep 17 12:10:50.943492 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:50.943351 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" Sep 17 12:10:51.060855 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:51.060824 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-skhl2_344d4ef6-f336-46de-a0bd-0735aba220ec/insights-operator/0.log" Sep 17 12:10:51.062700 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:51.062679 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-skhl2_344d4ef6-f336-46de-a0bd-0735aba220ec/insights-operator/1.log" Sep 17 12:10:51.209146 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:51.209060 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qksp8_2f8114ad-d2ca-48a8-b9af-349864c248c3/kube-rbac-proxy/0.log" Sep 17 12:10:51.228899 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:51.228873 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qksp8_2f8114ad-d2ca-48a8-b9af-349864c248c3/exporter/0.log" Sep 17 12:10:51.249237 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:51.249219 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qksp8_2f8114ad-d2ca-48a8-b9af-349864c248c3/extractor/0.log" Sep 17 12:10:55.375767 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:55.375738 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-qz2jt_facf9640-adc2-4f43-a3c0-5d0b0521ba7f/kube-storage-version-migrator-operator/1.log" Sep 17 12:10:55.376524 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:55.376505 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-qz2jt_facf9640-adc2-4f43-a3c0-5d0b0521ba7f/kube-storage-version-migrator-operator/0.log" Sep 17 12:10:56.374321 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:56.374289 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z5n4q_185c4318-ad3f-4a65-bdb4-4d8cbb554069/kube-multus-additional-cni-plugins/0.log" Sep 17 12:10:56.395940 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:56.395911 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z5n4q_185c4318-ad3f-4a65-bdb4-4d8cbb554069/egress-router-binary-copy/0.log" Sep 17 12:10:56.437279 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:56.437256 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z5n4q_185c4318-ad3f-4a65-bdb4-4d8cbb554069/cni-plugins/0.log" Sep 17 12:10:56.457695 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:56.457677 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z5n4q_185c4318-ad3f-4a65-bdb4-4d8cbb554069/bond-cni-plugin/0.log" Sep 17 12:10:56.476944 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:56.476926 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z5n4q_185c4318-ad3f-4a65-bdb4-4d8cbb554069/routeoverride-cni/0.log" Sep 17 12:10:56.495705 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:56.495688 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z5n4q_185c4318-ad3f-4a65-bdb4-4d8cbb554069/whereabouts-cni-bincopy/0.log" Sep 17 12:10:56.517555 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:56.517532 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z5n4q_185c4318-ad3f-4a65-bdb4-4d8cbb554069/whereabouts-cni/0.log" Sep 17 12:10:56.576572 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:56.576554 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mljj2_4c40a380-1eb9-46c1-a161-a9973c9b916d/kube-multus/0.log" Sep 17 12:10:56.668845 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:56.668783 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-f5svv_757a11fc-3750-4ef9-94f6-6c6da0d5c95c/network-metrics-daemon/0.log" Sep 17 12:10:56.691299 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:56.691276 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-f5svv_757a11fc-3750-4ef9-94f6-6c6da0d5c95c/kube-rbac-proxy/0.log" Sep 17 12:10:56.960486 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:56.960415 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" Sep 17 12:10:56.978487 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:56.978420 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8xmpg/perf-node-gather-daemonset-fft4w" podStartSLOduration=7.978401245 podStartE2EDuration="7.978401245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 12:10:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 12:10:50.96018271 +0000 UTC m=+1920.967629740" watchObservedRunningTime="2026-09-17 12:10:56.978401245 +0000 UTC m=+1926.985848277" Sep 17 12:10:57.802667 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:57.802638 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nn75c_eb0d7c07-75fb-4d19-b732-a1f9c737540a/ovn-controller/0.log" Sep 17 12:10:57.820597 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:57.820572 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nn75c_eb0d7c07-75fb-4d19-b732-a1f9c737540a/ovn-acl-logging/0.log" Sep 17 12:10:57.843259 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:57.843233 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nn75c_eb0d7c07-75fb-4d19-b732-a1f9c737540a/kube-rbac-proxy-node/0.log" Sep 17 12:10:57.868847 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:57.868821 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nn75c_eb0d7c07-75fb-4d19-b732-a1f9c737540a/kube-rbac-proxy-ovn-metrics/0.log" Sep 17 12:10:57.894381 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:57.894359 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nn75c_eb0d7c07-75fb-4d19-b732-a1f9c737540a/northd/0.log" Sep 17 12:10:57.915609 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:57.915592 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nn75c_eb0d7c07-75fb-4d19-b732-a1f9c737540a/nbdb/0.log" Sep 17 12:10:57.937065 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:57.937041 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nn75c_eb0d7c07-75fb-4d19-b732-a1f9c737540a/sbdb/0.log" Sep 17 12:10:57.980370 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:57.980344 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nn75c_eb0d7c07-75fb-4d19-b732-a1f9c737540a/ovnkube-controller/0.log" Sep 17 12:10:59.109857 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:59.109816 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-6f8b768455-bxwdl_0d51f0af-c4a9-4e64-86c5-07bb72fcf19b/check-endpoints/0.log" Sep 17 12:10:59.198789 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:10:59.198763 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-pk9p9_78be981b-e482-4dcf-a146-ea8549a3a847/network-check-target-container/0.log" Sep 17 12:11:00.073982 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:11:00.073958 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-qbxfn_26f77893-4701-4ac8-bd67-4df96b57deea/iptables-alerter/0.log" Sep 17 12:11:00.745271 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:11:00.745225 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-sdxbb_29bdca3f-fa73-4fad-a99e-1d8dcf4a59f0/tuned/0.log" Sep 17 12:11:02.267814 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:11:02.267778 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-86499c89c4-btxh2_21fcc133-9735-45b0-98c8-ba788b3b783a/cluster-samples-operator/0.log" Sep 17 12:11:02.284375 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:11:02.284345 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-86499c89c4-btxh2_21fcc133-9735-45b0-98c8-ba788b3b783a/cluster-samples-operator-watch/0.log" Sep 17 12:11:04.070365 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:11:04.070334 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-pzqqg_bc2cd604-c635-41ec-a14e-cb5574805469/csi-driver/0.log" Sep 17 12:11:04.105362 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:11:04.105338 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-pzqqg_bc2cd604-c635-41ec-a14e-cb5574805469/csi-node-driver-registrar/0.log" Sep 17 12:11:04.145139 ip-10-0-140-196 kubenswrapper[2581]: I0917 12:11:04.145104 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-pzqqg_bc2cd604-c635-41ec-a14e-cb5574805469/csi-liveness-probe/0.log"