Sep 11 07:57:38.706943 ip-10-0-136-245 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 11 07:57:38.706953 ip-10-0-136-245 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 11 07:57:38.706961 ip-10-0-136-245 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 11 07:57:38.707269 ip-10-0-136-245 systemd[1]: Failed to start Kubernetes Kubelet. Sep 11 07:57:48.376226 ip-10-0-136-245 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Sep 11 07:57:48.376242 ip-10-0-136-245 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 734d5337e72d459c9af457d2a60d378a -- Sep 11 07:59:22.886150 ip-10-0-136-245 systemd[1]: Starting Kubernetes Kubelet... Sep 11 07:59:23.303280 ip-10-0-136-245 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 11 07:59:23.303280 ip-10-0-136-245 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 11 07:59:23.303280 ip-10-0-136-245 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 11 07:59:23.303280 ip-10-0-136-245 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 11 07:59:23.303804 ip-10-0-136-245 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 11 07:59:23.304413 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.304346 2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 11 07:59:23.307559 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307540 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 07:59:23.307559 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307556 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 07:59:23.307559 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307561 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 07:59:23.307745 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307567 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 07:59:23.307745 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307572 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 07:59:23.307745 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307576 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 07:59:23.307745 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307581 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 07:59:23.307745 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307586 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 07:59:23.307745 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307590 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 07:59:23.307745 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307594 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 07:59:23.307745 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307599 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 07:59:23.307745 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307604 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 07:59:23.307745 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307608 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 07:59:23.307745 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307614 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 07:59:23.307745 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307629 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 07:59:23.307745 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307634 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 07:59:23.307745 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307639 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 07:59:23.307745 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307643 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 07:59:23.307745 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307647 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 07:59:23.307745 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307651 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 07:59:23.307745 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307655 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 07:59:23.307745 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307659 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 07:59:23.307745 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307663 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 07:59:23.308615 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307667 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 07:59:23.308615 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307672 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 07:59:23.308615 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307676 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 07:59:23.308615 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307680 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 07:59:23.308615 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307685 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 07:59:23.308615 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307689 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 07:59:23.308615 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307693 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 07:59:23.308615 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307697 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 07:59:23.308615 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307701 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 07:59:23.308615 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307706 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 07:59:23.308615 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307710 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 07:59:23.308615 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307714 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 07:59:23.308615 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307718 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 07:59:23.308615 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307722 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 07:59:23.308615 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307727 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 07:59:23.308615 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307731 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 07:59:23.308615 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307737 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 07:59:23.308615 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307741 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 07:59:23.308615 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307745 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 07:59:23.308615 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307748 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 07:59:23.309468 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307752 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 07:59:23.309468 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307757 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 07:59:23.309468 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307769 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 07:59:23.309468 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307774 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 07:59:23.309468 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307778 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 07:59:23.309468 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307782 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 07:59:23.309468 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307789 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 07:59:23.309468 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307795 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 07:59:23.309468 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307800 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 07:59:23.309468 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307805 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 07:59:23.309468 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307810 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 07:59:23.309468 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307815 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 07:59:23.309468 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307820 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 07:59:23.309468 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307824 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 07:59:23.309468 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307829 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 07:59:23.309468 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307833 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 07:59:23.309468 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307837 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 07:59:23.309468 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307842 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 07:59:23.309468 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307846 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 07:59:23.310284 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307851 2557 feature_gate.go:328] unrecognized feature gate: Example Sep 11 07:59:23.310284 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307856 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 07:59:23.310284 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307860 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 07:59:23.310284 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307864 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 07:59:23.310284 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307868 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 07:59:23.310284 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307872 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 07:59:23.310284 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307879 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 07:59:23.310284 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307883 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 07:59:23.310284 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307887 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 07:59:23.310284 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307891 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 07:59:23.310284 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307910 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 07:59:23.310284 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307915 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 07:59:23.310284 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307920 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 07:59:23.310284 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307923 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 07:59:23.310284 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307928 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 07:59:23.310284 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307932 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 07:59:23.310284 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307941 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 07:59:23.310284 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307946 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 07:59:23.310284 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307950 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 07:59:23.310284 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307954 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 07:59:23.311153 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307958 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 07:59:23.311153 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307962 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 07:59:23.311153 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307967 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 07:59:23.311153 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307971 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 07:59:23.311153 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307976 2557 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 07:59:23.311153 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307982 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 07:59:23.311153 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307987 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 07:59:23.311153 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307991 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 07:59:23.311153 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.307995 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 07:59:23.311153 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.308001 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 07:59:23.311153 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.308008 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 07:59:23.311153 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.308012 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 07:59:23.311153 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.308018 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 07:59:23.311153 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.308022 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 07:59:23.311153 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.308027 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 07:59:23.311153 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.308031 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 07:59:23.311153 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.308035 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 07:59:23.311153 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309148 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 07:59:23.311153 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309159 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 07:59:23.311153 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309163 2557 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 07:59:23.312033 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309168 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 07:59:23.312033 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309172 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 07:59:23.312033 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309177 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 07:59:23.312033 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309181 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 07:59:23.312033 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309185 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 07:59:23.312033 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309190 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 07:59:23.312033 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309195 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 07:59:23.312033 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309199 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 07:59:23.312033 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309204 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 07:59:23.312033 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309209 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 07:59:23.312033 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309213 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 07:59:23.312033 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309218 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 07:59:23.312033 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309222 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 07:59:23.312033 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309226 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 07:59:23.312033 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309231 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 07:59:23.312033 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309235 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 07:59:23.312033 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309239 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 07:59:23.312033 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309243 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 07:59:23.312033 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309248 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 07:59:23.312033 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309252 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 07:59:23.312638 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309256 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 07:59:23.312638 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309260 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 07:59:23.312638 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309264 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 07:59:23.312638 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309268 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 07:59:23.312638 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309272 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 07:59:23.312638 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309277 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 07:59:23.312638 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309281 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 07:59:23.312638 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309285 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 07:59:23.312638 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309289 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 07:59:23.312638 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309293 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 07:59:23.312638 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309298 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 07:59:23.312638 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309303 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 07:59:23.312638 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309307 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 07:59:23.312638 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309311 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 07:59:23.312638 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309318 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 07:59:23.312638 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309325 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 07:59:23.312638 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309330 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 07:59:23.312638 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309334 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 07:59:23.312638 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309338 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 07:59:23.312638 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309342 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 07:59:23.313220 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309347 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 07:59:23.313220 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309353 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 07:59:23.313220 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309358 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 07:59:23.313220 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309362 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 07:59:23.313220 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309367 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 07:59:23.313220 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309371 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 07:59:23.313220 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309375 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 07:59:23.313220 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309379 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 07:59:23.313220 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309383 2557 feature_gate.go:328] unrecognized feature gate: Example Sep 11 07:59:23.313220 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309387 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 07:59:23.313220 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309391 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 07:59:23.313220 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309396 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 07:59:23.313220 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309401 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 07:59:23.313220 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309405 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 07:59:23.313220 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309409 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 07:59:23.313220 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309415 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 07:59:23.313220 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309421 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 07:59:23.313220 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309426 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 07:59:23.313220 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309431 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 07:59:23.313715 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309436 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 07:59:23.313715 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309443 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 07:59:23.313715 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309447 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 07:59:23.313715 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309452 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 07:59:23.313715 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309457 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 07:59:23.313715 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309461 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 07:59:23.313715 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309467 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 07:59:23.313715 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309472 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 07:59:23.313715 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309477 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 07:59:23.313715 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309481 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 07:59:23.313715 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309486 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 07:59:23.313715 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309490 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 07:59:23.313715 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309495 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 07:59:23.313715 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309500 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 07:59:23.313715 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309504 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 07:59:23.313715 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309509 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 07:59:23.313715 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309514 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 07:59:23.313715 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309518 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 07:59:23.313715 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309522 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 07:59:23.314194 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309526 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 07:59:23.314194 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309531 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 07:59:23.314194 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309535 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 07:59:23.314194 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309540 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 07:59:23.314194 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309544 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 07:59:23.314194 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309548 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 07:59:23.314194 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309553 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 07:59:23.314194 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309557 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 07:59:23.314194 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309562 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 07:59:23.314194 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309566 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 07:59:23.314194 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309570 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 07:59:23.314194 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309574 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 07:59:23.314194 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309578 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 07:59:23.314194 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309582 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 07:59:23.314194 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309587 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 07:59:23.314194 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309591 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 07:59:23.314194 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309595 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 07:59:23.314194 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.309600 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 07:59:23.314194 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310180 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 07:59:23.314194 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310189 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 07:59:23.314868 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310193 2557 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 07:59:23.314868 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310197 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 07:59:23.314868 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310205 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 07:59:23.314868 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310211 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 07:59:23.314868 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310216 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 07:59:23.314868 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310220 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 07:59:23.314868 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310224 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 07:59:23.314868 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310229 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 07:59:23.314868 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310233 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 07:59:23.314868 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310238 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 07:59:23.314868 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310242 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 07:59:23.314868 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310246 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 07:59:23.314868 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310251 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 07:59:23.314868 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310255 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 07:59:23.314868 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310259 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 07:59:23.314868 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310264 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 07:59:23.314868 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310268 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 07:59:23.314868 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310273 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 07:59:23.314868 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310277 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 07:59:23.314868 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310281 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 07:59:23.315380 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310285 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 07:59:23.315380 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310289 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 07:59:23.315380 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310294 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 07:59:23.315380 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310298 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 07:59:23.315380 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310302 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 07:59:23.315380 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310306 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 07:59:23.315380 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310310 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 07:59:23.315380 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310315 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 07:59:23.315380 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310320 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 07:59:23.315380 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310324 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 07:59:23.315380 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310328 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 07:59:23.315380 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310333 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 07:59:23.315380 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310337 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 07:59:23.315380 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310341 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 07:59:23.315380 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310346 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 07:59:23.315380 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310350 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 07:59:23.315380 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310354 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 07:59:23.315380 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310358 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 07:59:23.315380 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310362 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 07:59:23.315380 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310367 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 07:59:23.315984 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310371 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 07:59:23.315984 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310375 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 07:59:23.315984 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310381 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 07:59:23.315984 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310386 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 07:59:23.315984 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310390 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 07:59:23.315984 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310394 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 07:59:23.315984 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310398 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 07:59:23.315984 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310403 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 07:59:23.315984 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310407 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 07:59:23.315984 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310411 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 07:59:23.315984 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310415 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 07:59:23.315984 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310420 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 07:59:23.315984 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310424 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 07:59:23.315984 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310428 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 07:59:23.315984 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310433 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 07:59:23.315984 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310437 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 07:59:23.315984 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310442 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 07:59:23.315984 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310446 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 07:59:23.315984 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310451 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 07:59:23.315984 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310455 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 07:59:23.316482 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310460 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 07:59:23.316482 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310464 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 07:59:23.316482 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310468 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 07:59:23.316482 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310472 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 07:59:23.316482 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310476 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 07:59:23.316482 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310480 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 07:59:23.316482 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310485 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 07:59:23.316482 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310489 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 07:59:23.316482 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310493 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 07:59:23.316482 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310498 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 07:59:23.316482 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310502 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 07:59:23.316482 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310506 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 07:59:23.316482 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310512 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 07:59:23.316482 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310517 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 07:59:23.316482 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310521 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 07:59:23.316482 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310526 2557 feature_gate.go:328] unrecognized feature gate: Example Sep 11 07:59:23.316482 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310530 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 07:59:23.316482 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310534 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 07:59:23.316482 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310539 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 07:59:23.316482 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310543 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 07:59:23.317002 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310547 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 07:59:23.317002 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310552 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 07:59:23.317002 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310556 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 07:59:23.317002 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310560 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 07:59:23.317002 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310565 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 07:59:23.317002 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310570 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 07:59:23.317002 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310574 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 07:59:23.317002 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310578 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 07:59:23.317002 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310583 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 07:59:23.317002 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310587 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 07:59:23.317002 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310591 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 07:59:23.317002 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310595 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 07:59:23.317002 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310599 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 07:59:23.317002 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310604 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 07:59:23.317002 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310608 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 07:59:23.317002 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310614 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 07:59:23.317002 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.310620 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 07:59:23.317002 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310722 2557 flags.go:64] FLAG: --address="0.0.0.0" Sep 11 07:59:23.317002 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310768 2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 11 07:59:23.317002 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310777 2557 flags.go:64] FLAG: --anonymous-auth="true" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310783 2557 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310790 2557 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310795 2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310801 2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310807 2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310813 2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310817 2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310823 2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310829 2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310836 2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310841 2557 flags.go:64] FLAG: --cgroup-root="" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310846 2557 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310851 2557 flags.go:64] FLAG: --client-ca-file="" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310856 2557 flags.go:64] FLAG: --cloud-provider="external" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310860 2557 flags.go:64] FLAG: --cluster-dns="[]" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310866 2557 flags.go:64] FLAG: --cluster-domain="" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310870 2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310875 2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310880 2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310885 2557 flags.go:64] FLAG: --container-log-max-files="5" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310893 2557 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310915 2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 11 07:59:23.317528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310920 2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310925 2557 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310930 2557 flags.go:64] FLAG: --contention-profiling="false" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310934 2557 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310939 2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310944 2557 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310948 2557 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310954 2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310959 2557 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310965 2557 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310969 2557 flags.go:64] FLAG: --enable-load-reader="false" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310974 2557 flags.go:64] FLAG: --enable-server="true" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310979 2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310985 2557 flags.go:64] FLAG: --event-burst="100" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310990 2557 flags.go:64] FLAG: --event-qps="50" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.310995 2557 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311000 2557 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311005 2557 flags.go:64] FLAG: --eviction-hard="" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311010 2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311018 2557 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311023 2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311027 2557 flags.go:64] FLAG: --eviction-soft="" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311032 2557 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311037 2557 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311042 2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 11 07:59:23.318098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311046 2557 flags.go:64] FLAG: --experimental-mounter-path="" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311051 2557 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311055 2557 flags.go:64] FLAG: --fail-swap-on="true" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311060 2557 flags.go:64] FLAG: --feature-gates="" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311073 2557 flags.go:64] FLAG: --file-check-frequency="20s" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311080 2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311085 2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311090 2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311094 2557 flags.go:64] FLAG: --healthz-port="10248" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311099 2557 flags.go:64] FLAG: --help="false" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311104 2557 flags.go:64] FLAG: --hostname-override="ip-10-0-136-245.ec2.internal" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311109 2557 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311113 2557 flags.go:64] FLAG: --http-check-frequency="20s" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311118 2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311123 2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311129 2557 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311134 2557 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311138 2557 flags.go:64] FLAG: --image-service-endpoint="" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311143 2557 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311147 2557 flags.go:64] FLAG: --kube-api-burst="100" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311151 2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311156 2557 flags.go:64] FLAG: --kube-api-qps="50" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311161 2557 flags.go:64] FLAG: --kube-reserved="" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311165 2557 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 11 07:59:23.318809 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311170 2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311174 2557 flags.go:64] FLAG: --kubelet-cgroups="" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311180 2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311186 2557 flags.go:64] FLAG: --lock-file="" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311191 2557 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311195 2557 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311200 2557 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311208 2557 flags.go:64] FLAG: --log-json-split-stream="false" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311213 2557 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311217 2557 flags.go:64] FLAG: --log-text-split-stream="false" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311222 2557 flags.go:64] FLAG: --logging-format="text" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311226 2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311231 2557 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311238 2557 flags.go:64] FLAG: --manifest-url="" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311242 2557 flags.go:64] FLAG: --manifest-url-header="" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311249 2557 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311253 2557 flags.go:64] FLAG: --max-open-files="1000000" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311259 2557 flags.go:64] FLAG: --max-pods="110" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311264 2557 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311269 2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311273 2557 flags.go:64] FLAG: --memory-manager-policy="None" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311278 2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311283 2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311288 2557 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311293 2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 11 07:59:23.319768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311303 2557 flags.go:64] FLAG: --node-status-max-images="50" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311307 2557 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311312 2557 flags.go:64] FLAG: --oom-score-adj="-999" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311317 2557 flags.go:64] FLAG: --pod-cidr="" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311322 2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311330 2557 flags.go:64] FLAG: --pod-manifest-path="" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311335 2557 flags.go:64] FLAG: --pod-max-pids="-1" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311340 2557 flags.go:64] FLAG: --pods-per-core="0" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311344 2557 flags.go:64] FLAG: --port="10250" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311349 2557 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311355 2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-039b0dbd1c5d8a309" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311361 2557 flags.go:64] FLAG: --qos-reserved="" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311366 2557 flags.go:64] FLAG: --read-only-port="10255" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311371 2557 flags.go:64] FLAG: --register-node="true" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311376 2557 flags.go:64] FLAG: --register-with-taints="" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311381 2557 flags.go:64] FLAG: --registry-burst="10" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311386 2557 flags.go:64] FLAG: --registry-qps="5" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311390 2557 flags.go:64] FLAG: --reserved-cpus="" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311395 2557 flags.go:64] FLAG: --reserved-memory="" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311402 2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311408 2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311413 2557 flags.go:64] FLAG: --rotate-certificates="false" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311426 2557 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311432 2557 flags.go:64] FLAG: --runonce="false" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311437 2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 11 07:59:23.320399 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311441 2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311446 2557 flags.go:64] FLAG: --seccomp-default="false" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311451 2557 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311455 2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311460 2557 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311465 2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311470 2557 flags.go:64] FLAG: --storage-driver-password="root" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311475 2557 flags.go:64] FLAG: --storage-driver-secure="false" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311479 2557 flags.go:64] FLAG: --storage-driver-table="stats" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311483 2557 flags.go:64] FLAG: --storage-driver-user="root" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311488 2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311494 2557 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311498 2557 flags.go:64] FLAG: --system-cgroups="" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311503 2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311511 2557 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311516 2557 flags.go:64] FLAG: --tls-cert-file="" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311520 2557 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311527 2557 flags.go:64] FLAG: --tls-min-version="" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311532 2557 flags.go:64] FLAG: --tls-private-key-file="" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311537 2557 flags.go:64] FLAG: --topology-manager-policy="none" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311543 2557 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311547 2557 flags.go:64] FLAG: --topology-manager-scope="container" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311552 2557 flags.go:64] FLAG: --v="2" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311558 2557 flags.go:64] FLAG: --version="false" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311564 2557 flags.go:64] FLAG: --vmodule="" Sep 11 07:59:23.321052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311569 2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 11 07:59:23.321679 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.311575 2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 11 07:59:23.321679 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311746 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 07:59:23.321679 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311753 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 07:59:23.321679 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311757 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 07:59:23.321679 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311761 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 07:59:23.321679 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311765 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 07:59:23.321679 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311772 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 07:59:23.321679 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311778 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 07:59:23.321679 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311783 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 07:59:23.321679 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311787 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 07:59:23.321679 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311792 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 07:59:23.321679 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311796 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 07:59:23.321679 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311800 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 07:59:23.321679 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311804 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 07:59:23.321679 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311809 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 07:59:23.321679 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311813 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 07:59:23.321679 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311817 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 07:59:23.321679 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311821 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 07:59:23.321679 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311825 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 07:59:23.322164 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311829 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 07:59:23.322164 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311834 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 07:59:23.322164 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311838 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 07:59:23.322164 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311842 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 07:59:23.322164 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311848 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 07:59:23.322164 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311852 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 07:59:23.322164 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311857 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 07:59:23.322164 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311862 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 07:59:23.322164 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311866 2557 feature_gate.go:328] unrecognized feature gate: Example Sep 11 07:59:23.322164 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311870 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 07:59:23.322164 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311875 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 07:59:23.322164 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311880 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 07:59:23.322164 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311886 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 07:59:23.322164 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311891 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 07:59:23.322164 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311913 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 07:59:23.322164 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311918 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 07:59:23.322164 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311922 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 07:59:23.322164 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311927 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 07:59:23.322164 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311931 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 07:59:23.322164 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311936 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 07:59:23.322656 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311940 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 07:59:23.322656 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311944 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 07:59:23.322656 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311948 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 07:59:23.322656 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311952 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 07:59:23.322656 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311956 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 07:59:23.322656 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311961 2557 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 07:59:23.322656 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311965 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 07:59:23.322656 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311969 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 07:59:23.322656 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311974 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 07:59:23.322656 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311978 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 07:59:23.322656 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311982 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 07:59:23.322656 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311987 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 07:59:23.322656 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311991 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 07:59:23.322656 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311995 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 07:59:23.322656 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.311999 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 07:59:23.322656 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312003 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 07:59:23.322656 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312009 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 07:59:23.322656 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312013 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 07:59:23.322656 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312017 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 07:59:23.322656 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312021 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 07:59:23.323167 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312027 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 07:59:23.323167 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312030 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 07:59:23.323167 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312034 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 07:59:23.323167 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312038 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 07:59:23.323167 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312043 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 07:59:23.323167 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312046 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 07:59:23.323167 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312053 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 07:59:23.323167 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312057 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 07:59:23.323167 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312062 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 07:59:23.323167 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312066 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 07:59:23.323167 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312070 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 07:59:23.323167 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312075 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 07:59:23.323167 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312079 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 07:59:23.323167 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312083 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 07:59:23.323167 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312087 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 07:59:23.323167 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312092 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 07:59:23.323167 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312096 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 07:59:23.323167 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312101 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 07:59:23.323167 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312105 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 07:59:23.323167 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312109 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 07:59:23.323668 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312113 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 07:59:23.323668 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312117 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 07:59:23.323668 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312121 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 07:59:23.323668 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312125 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 07:59:23.323668 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312130 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 07:59:23.323668 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312134 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 07:59:23.323668 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312138 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 07:59:23.323668 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312142 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 07:59:23.323668 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312148 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 07:59:23.323668 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312152 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 07:59:23.323668 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312158 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 07:59:23.323668 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312163 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 07:59:23.323668 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312170 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 07:59:23.323668 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312175 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 07:59:23.323668 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312180 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 07:59:23.323668 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312184 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 07:59:23.323668 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312188 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 07:59:23.323668 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312192 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 07:59:23.323668 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312197 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 07:59:23.324175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312204 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 07:59:23.324175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.312209 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 07:59:23.324175 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.312928 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 11 07:59:23.324175 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.321119 2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 11 07:59:23.324175 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.321220 2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 11 07:59:23.324175 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.322087 2557 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 11 07:59:23.324175 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.322095 2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 11 07:59:23.324175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322150 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 07:59:23.324175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322154 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 07:59:23.324175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322158 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 07:59:23.324175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322161 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 07:59:23.324175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322164 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 07:59:23.324175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322168 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 07:59:23.324175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322171 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 07:59:23.324175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322174 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 07:59:23.324553 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322177 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 07:59:23.324553 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322180 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 07:59:23.324553 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322183 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 07:59:23.324553 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322186 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 07:59:23.324553 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322189 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 07:59:23.324553 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322192 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 07:59:23.324553 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322194 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 07:59:23.324553 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322197 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 07:59:23.324553 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322200 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 07:59:23.324553 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322202 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 07:59:23.324553 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322205 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 07:59:23.324553 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322208 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 07:59:23.324553 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322212 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 07:59:23.324553 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322215 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 07:59:23.324553 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322218 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 07:59:23.324553 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322222 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 07:59:23.324553 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322226 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 07:59:23.324553 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322229 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 07:59:23.324553 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322231 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 07:59:23.325175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322234 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 07:59:23.325175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322237 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 07:59:23.325175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322240 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 07:59:23.325175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322243 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 07:59:23.325175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322246 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 07:59:23.325175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322250 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 07:59:23.325175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322253 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 07:59:23.325175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322257 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 07:59:23.325175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322259 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 07:59:23.325175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322263 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 07:59:23.325175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322266 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 07:59:23.325175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322268 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 07:59:23.325175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322271 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 07:59:23.325175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322274 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 07:59:23.325175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322277 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 07:59:23.325175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322279 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 07:59:23.325175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322282 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 07:59:23.325175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322284 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 07:59:23.325175 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322287 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 07:59:23.325631 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322289 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 07:59:23.325631 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322292 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 07:59:23.325631 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322295 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 07:59:23.325631 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322297 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 07:59:23.325631 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322300 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 07:59:23.325631 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322303 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 07:59:23.325631 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322305 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 07:59:23.325631 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322308 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 07:59:23.325631 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322311 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 07:59:23.325631 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322313 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 07:59:23.325631 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322316 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 07:59:23.325631 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322318 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 07:59:23.325631 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322321 2557 feature_gate.go:328] unrecognized feature gate: Example Sep 11 07:59:23.325631 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322323 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 07:59:23.325631 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322326 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 07:59:23.325631 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322329 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 07:59:23.325631 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322332 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 07:59:23.325631 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322335 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 07:59:23.325631 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322337 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 07:59:23.325631 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322340 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 07:59:23.326136 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322343 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 07:59:23.326136 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322345 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 07:59:23.326136 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322348 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 07:59:23.326136 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322351 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 07:59:23.326136 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322354 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 07:59:23.326136 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322356 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 07:59:23.326136 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322359 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 07:59:23.326136 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322361 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 07:59:23.326136 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322364 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 07:59:23.326136 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322367 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 07:59:23.326136 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322369 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 07:59:23.326136 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322372 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 07:59:23.326136 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322374 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 07:59:23.326136 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322378 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 07:59:23.326136 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322381 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 07:59:23.326136 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322383 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 07:59:23.326136 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322386 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 07:59:23.326136 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322389 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 07:59:23.326136 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322391 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 07:59:23.326136 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322394 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 07:59:23.326616 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322396 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 07:59:23.326616 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322399 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 07:59:23.326616 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322401 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 07:59:23.326616 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322404 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 07:59:23.326616 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322407 2557 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 07:59:23.326616 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322411 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 07:59:23.326616 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322414 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 07:59:23.326616 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322417 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 07:59:23.326616 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322420 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 07:59:23.326616 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322423 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 07:59:23.326616 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322426 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 07:59:23.326616 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322429 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 07:59:23.326616 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322431 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 07:59:23.326616 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.322436 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 11 07:59:23.326975 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322551 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 07:59:23.326975 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322557 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 07:59:23.326975 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322561 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 07:59:23.326975 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322564 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 07:59:23.326975 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322568 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 07:59:23.326975 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322571 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 07:59:23.326975 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322574 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 07:59:23.326975 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322576 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 07:59:23.326975 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322579 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 07:59:23.326975 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322582 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 07:59:23.326975 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322584 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 07:59:23.326975 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322587 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 07:59:23.326975 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322590 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 07:59:23.326975 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322593 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 07:59:23.326975 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322595 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 07:59:23.326975 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322598 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 07:59:23.326975 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322600 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 07:59:23.326975 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322603 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 07:59:23.326975 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322606 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 07:59:23.327448 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322608 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 07:59:23.327448 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322611 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 07:59:23.327448 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322613 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 07:59:23.327448 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322616 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 07:59:23.327448 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322619 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 07:59:23.327448 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322622 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 07:59:23.327448 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322624 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 07:59:23.327448 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322627 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 07:59:23.327448 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322629 2557 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 07:59:23.327448 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322632 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 07:59:23.327448 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322635 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 07:59:23.327448 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322638 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 07:59:23.327448 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322640 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 07:59:23.327448 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322643 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 07:59:23.327448 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322645 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 07:59:23.327448 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322648 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 07:59:23.327448 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322650 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 07:59:23.327448 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322653 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 07:59:23.327448 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322656 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 07:59:23.327448 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322658 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 07:59:23.327954 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322662 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 07:59:23.327954 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322664 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 07:59:23.327954 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322667 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 07:59:23.327954 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322670 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 07:59:23.327954 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322672 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 07:59:23.327954 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322675 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 07:59:23.327954 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322678 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 07:59:23.327954 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322682 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 07:59:23.327954 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322685 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 07:59:23.327954 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322688 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 07:59:23.327954 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322690 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 07:59:23.327954 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322693 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 07:59:23.327954 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322695 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 07:59:23.327954 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322698 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 07:59:23.327954 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322700 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 07:59:23.327954 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322703 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 07:59:23.327954 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322707 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 07:59:23.327954 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322710 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 07:59:23.327954 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322713 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 07:59:23.328411 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322716 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 07:59:23.328411 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322719 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 07:59:23.328411 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322722 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 07:59:23.328411 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322724 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 07:59:23.328411 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322727 2557 feature_gate.go:328] unrecognized feature gate: Example Sep 11 07:59:23.328411 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322730 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 07:59:23.328411 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322732 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 07:59:23.328411 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322735 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 07:59:23.328411 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322738 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 07:59:23.328411 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322740 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 07:59:23.328411 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322743 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 07:59:23.328411 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322746 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 07:59:23.328411 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322748 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 07:59:23.328411 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322751 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 07:59:23.328411 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322754 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 07:59:23.328411 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322757 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 07:59:23.328411 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322759 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 07:59:23.328411 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322762 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 07:59:23.328411 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322764 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 07:59:23.328411 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322767 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 07:59:23.328915 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322770 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 07:59:23.328915 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322773 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 07:59:23.328915 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322776 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 07:59:23.328915 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322778 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 07:59:23.328915 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322781 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 07:59:23.328915 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322783 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 07:59:23.328915 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322786 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 07:59:23.328915 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322788 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 07:59:23.328915 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322791 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 07:59:23.328915 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322793 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 07:59:23.328915 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322796 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 07:59:23.328915 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322798 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 07:59:23.328915 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322801 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 07:59:23.328915 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322804 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 07:59:23.328915 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322806 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 07:59:23.328915 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322809 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 07:59:23.328915 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322811 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 07:59:23.328915 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322814 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 07:59:23.328915 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322816 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 07:59:23.328915 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322819 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 07:59:23.329396 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:23.322822 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 07:59:23.329396 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.322826 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 11 07:59:23.329396 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.322934 2557 server.go:962] "Client rotation is on, will bootstrap in background" Sep 11 07:59:23.329396 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.327045 2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 11 07:59:23.329396 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.328091 2557 server.go:1019] "Starting client certificate rotation" Sep 11 07:59:23.329396 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.328200 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 11 07:59:23.329396 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.328236 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 11 07:59:23.331397 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.331382 2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 11 07:59:23.334440 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.334417 2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 11 07:59:23.349780 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.349760 2557 log.go:25] "Validated CRI v1 runtime API" Sep 11 07:59:23.354568 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.354554 2557 log.go:25] "Validated CRI v1 image API" Sep 11 07:59:23.356211 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.356184 2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 11 07:59:23.362005 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.361978 2557 fs.go:135] Filesystem UUIDs: map[3932b9c4-7994-45da-81e1-7ee5c7ed6d23:/dev/nvme0n1p4 7581e9b5-64c1-4f13-b1fa-b7bf42a0c672:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Sep 11 07:59:23.362063 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.362004 2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 11 07:59:23.366153 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.366049 2557 manager.go:216] Machine: {Timestamp:2026-09-11 07:59:23.36549993 +0000 UTC m=+0.371217580 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104245 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec282dc2596cd6c6423ddac82fdbb5a1 SystemUUID:ec282dc2-596c-d6c6-423d-dac82fdbb5a1 BootID:734d5337-e72d-459c-9af4-57d2a60d378a Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:08:1c:bb:28:f7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:08:1c:bb:28:f7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:37:25:9b:46:66 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 11 07:59:23.366153 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.366147 2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 11 07:59:23.366255 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.366232 2557 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 11 07:59:23.367373 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.367351 2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 11 07:59:23.367511 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.367376 2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-245.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 11 07:59:23.368307 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.368292 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 11 07:59:23.368923 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.368912 2557 topology_manager.go:138] "Creating topology manager with none policy" Sep 11 07:59:23.368960 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.368925 2557 container_manager_linux.go:309] "Creating device plugin manager" Sep 11 07:59:23.368960 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.368933 2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 11 07:59:23.369911 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.369890 2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 11 07:59:23.371102 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.371091 2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 11 07:59:23.375490 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.375478 2557 state_mem.go:36] "Initialized new in-memory state store" Sep 11 07:59:23.375599 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.375589 2557 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 11 07:59:23.375627 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.375621 2557 kubelet.go:486] "Attempting to sync node with API server" Sep 11 07:59:23.375654 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.375630 2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 11 07:59:23.375654 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.375643 2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 11 07:59:23.375654 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.375651 2557 kubelet.go:398] "Adding apiserver pod source" Sep 11 07:59:23.375780 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.375660 2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 11 07:59:23.375981 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.375970 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 11 07:59:23.376018 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.375992 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 11 07:59:23.378504 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.378491 2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 11 07:59:23.380698 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.380683 2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 11 07:59:23.380763 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.380714 2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 11 07:59:23.382693 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.382677 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 11 07:59:23.382693 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.382694 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 11 07:59:23.382768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.382700 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 11 07:59:23.382768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.382705 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 11 07:59:23.382768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.382713 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 11 07:59:23.382768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.382719 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 11 07:59:23.382768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.382731 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 11 07:59:23.382768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.382737 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 11 07:59:23.382768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.382743 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 11 07:59:23.382768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.382749 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 11 07:59:23.382768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.382757 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 11 07:59:23.382768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.382762 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 11 07:59:23.383738 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.383724 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 11 07:59:23.383781 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.383740 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 11 07:59:23.386475 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.386446 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-245.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 11 07:59:23.386529 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.386440 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 11 07:59:23.387517 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.387502 2557 server.go:1268] "Started kubelet" Sep 11 07:59:23.387590 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.387533 2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 11 07:59:23.387672 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.387630 2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 11 07:59:23.387725 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.387689 2557 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 11 07:59:23.388368 ip-10-0-136-245 systemd[1]: Started Kubernetes Kubelet. Sep 11 07:59:23.389847 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.389827 2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 11 07:59:23.389969 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.389870 2557 server.go:310] "Adding debug handlers to kubelet server" Sep 11 07:59:23.394213 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.394171 2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 11 07:59:23.396817 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.396800 2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-245.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 11 07:59:23.397386 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.397370 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 11 07:59:23.398035 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.396862 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-245.ec2.internal.18d4354e888af4ba default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-245.ec2.internal,UID:ip-10-0-136-245.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-245.ec2.internal,},FirstTimestamp:2026-09-11 07:59:23.387458746 +0000 UTC m=+0.393176395,LastTimestamp:2026-09-11 07:59:23.387458746 +0000 UTC m=+0.393176395,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-245.ec2.internal,}" Sep 11 07:59:23.398035 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.398024 2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 11 07:59:23.398389 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.398369 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-245.ec2.internal\" not found" Sep 11 07:59:23.398837 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.398824 2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 11 07:59:23.398837 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.398838 2557 factory.go:55] Registering systemd factory Sep 11 07:59:23.398939 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.398844 2557 factory.go:223] Registration of the systemd container factory successfully Sep 11 07:59:23.398939 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.398917 2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 11 07:59:23.398939 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.398922 2557 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 11 07:59:23.399067 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.398943 2557 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 11 07:59:23.399067 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.399041 2557 reconstruct.go:97] "Volume reconstruction finished" Sep 11 07:59:23.399067 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.399049 2557 reconciler.go:29] "Reconciler: start to sync state" Sep 11 07:59:23.399067 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.399061 2557 factory.go:153] Registering CRI-O factory Sep 11 07:59:23.399186 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.399073 2557 factory.go:223] Registration of the crio container factory successfully Sep 11 07:59:23.399186 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.399094 2557 factory.go:103] Registering Raw factory Sep 11 07:59:23.399186 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.399107 2557 manager.go:1196] Started watching for new ooms in manager Sep 11 07:59:23.399466 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.399453 2557 manager.go:349] Starting recovery of all containers Sep 11 07:59:23.405936 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.405890 2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-245.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 11 07:59:23.406020 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.405952 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 11 07:59:23.409819 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.409801 2557 manager.go:354] Recovery completed Sep 11 07:59:23.412659 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.411300 2557 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": readdirent /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Sep 11 07:59:23.414272 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.414259 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 07:59:23.416347 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.416329 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeHasSufficientMemory" Sep 11 07:59:23.416425 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.416361 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 07:59:23.416425 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.416372 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeHasSufficientPID" Sep 11 07:59:23.416938 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.416924 2557 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 11 07:59:23.416938 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.416931 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 07:59:23.416938 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.416936 2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 11 07:59:23.417133 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.416953 2557 state_mem.go:36] "Initialized new in-memory state store" Sep 11 07:59:23.417601 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.417586 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeHasSufficientMemory" Sep 11 07:59:23.417664 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.417611 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 07:59:23.417664 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.417620 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeHasSufficientPID" Sep 11 07:59:23.418961 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.418876 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-245.ec2.internal.18d4354e8a43c376 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-245.ec2.internal,UID:ip-10-0-136-245.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-245.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-245.ec2.internal,},FirstTimestamp:2026-09-11 07:59:23.41634751 +0000 UTC m=+0.422065159,LastTimestamp:2026-09-11 07:59:23.41634751 +0000 UTC m=+0.422065159,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-245.ec2.internal,}" Sep 11 07:59:23.419105 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.419090 2557 policy_none.go:49] "None policy: Start" Sep 11 07:59:23.419194 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.419109 2557 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 11 07:59:23.419194 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.419122 2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 11 07:59:23.421125 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.421111 2557 policy_none.go:47] "Start" Sep 11 07:59:23.431883 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.431810 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-245.ec2.internal.18d4354e8a440b3e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-245.ec2.internal,UID:ip-10-0-136-245.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-245.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-245.ec2.internal,},FirstTimestamp:2026-09-11 07:59:23.416365886 +0000 UTC m=+0.422083535,LastTimestamp:2026-09-11 07:59:23.416365886 +0000 UTC m=+0.422083535,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-245.ec2.internal,}" Sep 11 07:59:23.441570 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.441512 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-245.ec2.internal.18d4354e8a4436ae default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-245.ec2.internal,UID:ip-10-0-136-245.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-245.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-245.ec2.internal,},FirstTimestamp:2026-09-11 07:59:23.416377006 +0000 UTC m=+0.422094659,LastTimestamp:2026-09-11 07:59:23.416377006 +0000 UTC m=+0.422094659,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-245.ec2.internal,}" Sep 11 07:59:23.445758 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.445738 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cn5wx" Sep 11 07:59:23.451778 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.451713 2557 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-245.ec2.internal.18d4354e8a43c376\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-245.ec2.internal.18d4354e8a43c376 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-245.ec2.internal,UID:ip-10-0-136-245.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-245.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-245.ec2.internal,},FirstTimestamp:2026-09-11 07:59:23.41634751 +0000 UTC m=+0.422065159,LastTimestamp:2026-09-11 07:59:23.417600208 +0000 UTC m=+0.423317857,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-245.ec2.internal,}" Sep 11 07:59:23.477052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.458494 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cn5wx" Sep 11 07:59:23.477052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.458509 2557 manager.go:337] "Starting Device Plugin manager" Sep 11 07:59:23.477052 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.458555 2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 11 07:59:23.477052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.458568 2557 server.go:87] "Starting device plugin registration server" Sep 11 07:59:23.477052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.458873 2557 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 11 07:59:23.477052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.458883 2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 11 07:59:23.477052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.458955 2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 11 07:59:23.477052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.459038 2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 11 07:59:23.477052 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.459047 2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 11 07:59:23.477052 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.459538 2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 11 07:59:23.477052 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.459586 2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-245.ec2.internal\" not found" Sep 11 07:59:23.477052 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.464204 2557 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-245.ec2.internal.18d4354e8a440b3e\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-245.ec2.internal.18d4354e8a440b3e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-245.ec2.internal,UID:ip-10-0-136-245.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-245.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-245.ec2.internal,},FirstTimestamp:2026-09-11 07:59:23.416365886 +0000 UTC m=+0.422083535,LastTimestamp:2026-09-11 07:59:23.417615893 +0000 UTC m=+0.423333542,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-245.ec2.internal,}" Sep 11 07:59:23.495572 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.495554 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 11 07:59:23.496623 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.496607 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 11 07:59:23.496704 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.496662 2557 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 11 07:59:23.496704 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.496684 2557 kubelet.go:2448] "Starting kubelet main sync loop" Sep 11 07:59:23.496807 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.496725 2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 11 07:59:23.499510 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.499493 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 07:59:23.559872 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.559809 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 07:59:23.560688 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.560672 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeHasSufficientMemory" Sep 11 07:59:23.560754 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.560704 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 07:59:23.560754 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.560719 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeHasSufficientPID" Sep 11 07:59:23.560754 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.560744 2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-245.ec2.internal" Sep 11 07:59:23.569491 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.569475 2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-245.ec2.internal" Sep 11 07:59:23.569575 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.569499 2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-245.ec2.internal\": node \"ip-10-0-136-245.ec2.internal\" not found" Sep 11 07:59:23.596854 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.596836 2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-245.ec2.internal"] Sep 11 07:59:23.596936 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.596915 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 07:59:23.596985 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.596962 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-245.ec2.internal\" not found" Sep 11 07:59:23.597605 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.597592 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeHasSufficientMemory" Sep 11 07:59:23.597660 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.597613 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 07:59:23.597660 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.597626 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeHasSufficientPID" Sep 11 07:59:23.599820 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.599806 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 07:59:23.599973 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.599960 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal" Sep 11 07:59:23.600037 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.599993 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 07:59:23.600508 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.600488 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeHasSufficientMemory" Sep 11 07:59:23.600589 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.600514 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 07:59:23.600589 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.600528 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeHasSufficientPID" Sep 11 07:59:23.600589 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.600541 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeHasSufficientMemory" Sep 11 07:59:23.600589 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.600561 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 07:59:23.600589 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.600571 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeHasSufficientPID" Sep 11 07:59:23.602617 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.602604 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-245.ec2.internal" Sep 11 07:59:23.602668 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.602626 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 07:59:23.603294 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.603275 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeHasSufficientMemory" Sep 11 07:59:23.603294 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.603303 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 07:59:23.603448 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.603316 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeHasSufficientPID" Sep 11 07:59:23.624390 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.624373 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-245.ec2.internal\" not found" node="ip-10-0-136-245.ec2.internal" Sep 11 07:59:23.628559 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.628546 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-245.ec2.internal\" not found" node="ip-10-0-136-245.ec2.internal" Sep 11 07:59:23.697907 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.697875 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-245.ec2.internal\" not found" Sep 11 07:59:23.700094 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.700079 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/16b40a950e377baf3ad61bb98b577811-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal\" (UID: \"16b40a950e377baf3ad61bb98b577811\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal" Sep 11 07:59:23.700144 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.700101 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16b40a950e377baf3ad61bb98b577811-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal\" (UID: \"16b40a950e377baf3ad61bb98b577811\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal" Sep 11 07:59:23.700144 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.700116 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e11886251d79d45e61a827aee7470670-config\") pod \"kube-apiserver-proxy-ip-10-0-136-245.ec2.internal\" (UID: \"e11886251d79d45e61a827aee7470670\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-245.ec2.internal" Sep 11 07:59:23.798219 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.798200 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-245.ec2.internal\" not found" Sep 11 07:59:23.800431 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.800415 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16b40a950e377baf3ad61bb98b577811-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal\" (UID: \"16b40a950e377baf3ad61bb98b577811\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal" Sep 11 07:59:23.800481 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.800440 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e11886251d79d45e61a827aee7470670-config\") pod \"kube-apiserver-proxy-ip-10-0-136-245.ec2.internal\" (UID: \"e11886251d79d45e61a827aee7470670\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-245.ec2.internal" Sep 11 07:59:23.800481 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.800455 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/16b40a950e377baf3ad61bb98b577811-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal\" (UID: \"16b40a950e377baf3ad61bb98b577811\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal" Sep 11 07:59:23.800545 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.800514 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16b40a950e377baf3ad61bb98b577811-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal\" (UID: \"16b40a950e377baf3ad61bb98b577811\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal" Sep 11 07:59:23.800545 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.800531 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e11886251d79d45e61a827aee7470670-config\") pod \"kube-apiserver-proxy-ip-10-0-136-245.ec2.internal\" (UID: \"e11886251d79d45e61a827aee7470670\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-245.ec2.internal" Sep 11 07:59:23.800603 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.800516 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/16b40a950e377baf3ad61bb98b577811-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal\" (UID: \"16b40a950e377baf3ad61bb98b577811\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal" Sep 11 07:59:23.898837 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.898793 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-245.ec2.internal\" not found" Sep 11 07:59:23.926101 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.926085 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal" Sep 11 07:59:23.931275 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:23.931259 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-245.ec2.internal" Sep 11 07:59:23.999776 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:23.999753 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-245.ec2.internal\" not found" Sep 11 07:59:24.100169 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:24.100148 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-245.ec2.internal\" not found" Sep 11 07:59:24.200862 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:24.200841 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-245.ec2.internal\" not found" Sep 11 07:59:24.219288 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:24.219260 2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 07:59:24.301978 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:24.301953 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-245.ec2.internal\" not found" Sep 11 07:59:24.327453 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:24.327432 2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 11 07:59:24.327835 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:24.327562 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 11 07:59:24.327835 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:24.327565 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 11 07:59:24.402029 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:24.402006 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-245.ec2.internal\" not found" Sep 11 07:59:24.464929 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:24.464853 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 07:54:23 +0000 UTC" deadline="2028-05-10 17:13:40.495196319 +0000 UTC" Sep 11 07:59:24.464929 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:24.464880 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14577h14m16.030318861s" Sep 11 07:59:24.492869 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:24.492851 2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 07:59:24.498664 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:24.498630 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-245.ec2.internal" Sep 11 07:59:24.517617 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:24.517596 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 11 07:59:24.518573 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:24.518558 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal" Sep 11 07:59:24.553578 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:24.553559 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 11 07:59:24.686728 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:24.686705 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16b40a950e377baf3ad61bb98b577811.slice/crio-ce87eb78eb404945fa2fe0bccd2bd5d963cae992b2e1a5f0e8a492a0d96ddf82 WatchSource:0}: Error finding container ce87eb78eb404945fa2fe0bccd2bd5d963cae992b2e1a5f0e8a492a0d96ddf82: Status 404 returned error can't find the container with id ce87eb78eb404945fa2fe0bccd2bd5d963cae992b2e1a5f0e8a492a0d96ddf82 Sep 11 07:59:24.687238 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:24.687218 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode11886251d79d45e61a827aee7470670.slice/crio-0683d674db74ce77d5b38c9adf72da3a87e6ebf0bd79026b048e3902688c8c94 WatchSource:0}: Error finding container 0683d674db74ce77d5b38c9adf72da3a87e6ebf0bd79026b048e3902688c8c94: Status 404 returned error can't find the container with id 0683d674db74ce77d5b38c9adf72da3a87e6ebf0bd79026b048e3902688c8c94 Sep 11 07:59:24.691996 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:24.691971 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 11 07:59:24.992586 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:24.992561 2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 07:59:25.376727 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.376672 2557 apiserver.go:52] "Watching apiserver" Sep 11 07:59:25.382276 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.382247 2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 11 07:59:25.385209 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.383329 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal","openshift-network-diagnostics/network-check-target-bvr7m","openshift-network-operator/iptables-alerter-qfrhd","kube-system/global-pull-secret-syncer-f8ptc","kube-system/kube-apiserver-proxy-ip-10-0-136-245.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q","openshift-dns/node-resolver-vqjrt","openshift-multus/multus-797fj","openshift-multus/multus-additional-cni-plugins-8hfsw","openshift-multus/network-metrics-daemon-mkcn8","openshift-ovn-kubernetes/ovnkube-node-9j5fx","kube-system/konnectivity-agent-ddqh8","openshift-cluster-node-tuning-operator/tuned-hgrfs","openshift-image-registry/node-ca-6skrk"] Sep 11 07:59:25.386188 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.386164 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ddqh8" Sep 11 07:59:25.388413 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.388384 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:25.388732 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:25.388547 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvr7m" podUID="16110ca4-d538-4c5e-8def-a99a002c9735" Sep 11 07:59:25.388732 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.388559 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 11 07:59:25.388732 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.388659 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zjqfn\"" Sep 11 07:59:25.388732 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.388660 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 11 07:59:25.390941 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.390584 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qfrhd" Sep 11 07:59:25.392511 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.392481 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 11 07:59:25.392603 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.392555 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 11 07:59:25.392603 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.392566 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 11 07:59:25.392720 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.392555 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6g2qj\"" Sep 11 07:59:25.392864 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.392840 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6skrk" Sep 11 07:59:25.394538 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.394508 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 11 07:59:25.394640 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.394541 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 11 07:59:25.394640 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.394605 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 11 07:59:25.395012 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.394984 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.395454 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.395335 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-c9hzv\"" Sep 11 07:59:25.396654 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.396632 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 11 07:59:25.396804 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.396784 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gnqld\"" Sep 11 07:59:25.396886 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.396850 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 11 07:59:25.398233 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.398210 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.399251 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.398513 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 11 07:59:25.399678 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.399653 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 11 07:59:25.399768 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.399689 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 11 07:59:25.400289 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.400066 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 11 07:59:25.400289 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.400288 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-j29nr\"" Sep 11 07:59:25.400477 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.400455 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vqjrt" Sep 11 07:59:25.402041 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.402021 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-82rmv\"" Sep 11 07:59:25.402140 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.402071 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 11 07:59:25.402444 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.402423 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 11 07:59:25.403568 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.402891 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-797fj" Sep 11 07:59:25.404767 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.404739 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nmt5k\"" Sep 11 07:59:25.406286 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.405159 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 11 07:59:25.406286 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.405505 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 11 07:59:25.406286 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.405691 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 11 07:59:25.406286 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.405889 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.406286 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.405916 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 11 07:59:25.408336 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.407465 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 11 07:59:25.408336 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.407984 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:25.408336 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:25.408050 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f8ptc" podUID="98d2f5ae-5780-4110-935d-e3781cc271f7" Sep 11 07:59:25.408336 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408209 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 11 07:59:25.408336 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408270 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0d722217-e915-4d81-9033-6d5e1579f821-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vr95q\" (UID: \"0d722217-e915-4d81-9033-6d5e1579f821\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.408336 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408301 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mdgkb\"" Sep 11 07:59:25.408668 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408302 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-etc-tuned\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.408668 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408383 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/88bb78bf-db17-415d-b3e8-b88236c84350-serviceca\") pod \"node-ca-6skrk\" (UID: \"88bb78bf-db17-415d-b3e8-b88236c84350\") " pod="openshift-image-registry/node-ca-6skrk" Sep 11 07:59:25.408668 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408408 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7fm8\" (UniqueName: \"kubernetes.io/projected/88bb78bf-db17-415d-b3e8-b88236c84350-kube-api-access-k7fm8\") pod \"node-ca-6skrk\" (UID: \"88bb78bf-db17-415d-b3e8-b88236c84350\") " pod="openshift-image-registry/node-ca-6skrk" Sep 11 07:59:25.408668 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408431 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c61a8ab5-9ed9-4f3d-9e32-935de3901e01-hosts-file\") pod \"node-resolver-vqjrt\" (UID: \"c61a8ab5-9ed9-4f3d-9e32-935de3901e01\") " pod="openshift-dns/node-resolver-vqjrt" Sep 11 07:59:25.408668 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408455 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/967e46d4-81f0-49ce-a6ee-d8737cc7ad96-agent-certs\") pod \"konnectivity-agent-ddqh8\" (UID: \"967e46d4-81f0-49ce-a6ee-d8737cc7ad96\") " pod="kube-system/konnectivity-agent-ddqh8" Sep 11 07:59:25.408668 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408511 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-etc-sysctl-d\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.408668 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408534 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-lib-modules\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.408668 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408564 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0d722217-e915-4d81-9033-6d5e1579f821-registration-dir\") pod \"aws-ebs-csi-driver-node-vr95q\" (UID: \"0d722217-e915-4d81-9033-6d5e1579f821\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.408668 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408590 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/88bb78bf-db17-415d-b3e8-b88236c84350-host\") pod \"node-ca-6skrk\" (UID: \"88bb78bf-db17-415d-b3e8-b88236c84350\") " pod="openshift-image-registry/node-ca-6skrk" Sep 11 07:59:25.408668 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408633 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk5zp\" (UniqueName: \"kubernetes.io/projected/8a56f0ac-0efc-4afe-a83b-d44f0c11da3d-kube-api-access-bk5zp\") pod \"iptables-alerter-qfrhd\" (UID: \"8a56f0ac-0efc-4afe-a83b-d44f0c11da3d\") " pod="openshift-network-operator/iptables-alerter-qfrhd" Sep 11 07:59:25.408668 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408659 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0d722217-e915-4d81-9033-6d5e1579f821-etc-selinux\") pod \"aws-ebs-csi-driver-node-vr95q\" (UID: \"0d722217-e915-4d81-9033-6d5e1579f821\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.409693 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408677 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c61a8ab5-9ed9-4f3d-9e32-935de3901e01-tmp-dir\") pod \"node-resolver-vqjrt\" (UID: \"c61a8ab5-9ed9-4f3d-9e32-935de3901e01\") " pod="openshift-dns/node-resolver-vqjrt" Sep 11 07:59:25.409693 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408711 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-run\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.409693 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408745 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w67cx\" (UniqueName: \"kubernetes.io/projected/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-kube-api-access-w67cx\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.409693 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408771 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k2k2\" (UniqueName: \"kubernetes.io/projected/c61a8ab5-9ed9-4f3d-9e32-935de3901e01-kube-api-access-4k2k2\") pod \"node-resolver-vqjrt\" (UID: \"c61a8ab5-9ed9-4f3d-9e32-935de3901e01\") " pod="openshift-dns/node-resolver-vqjrt" Sep 11 07:59:25.409693 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408794 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8a56f0ac-0efc-4afe-a83b-d44f0c11da3d-iptables-alerter-script\") pod \"iptables-alerter-qfrhd\" (UID: \"8a56f0ac-0efc-4afe-a83b-d44f0c11da3d\") " pod="openshift-network-operator/iptables-alerter-qfrhd" Sep 11 07:59:25.409693 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408818 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-etc-modprobe-d\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.409693 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408860 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-etc-systemd\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.409693 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408913 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0d722217-e915-4d81-9033-6d5e1579f821-device-dir\") pod \"aws-ebs-csi-driver-node-vr95q\" (UID: \"0d722217-e915-4d81-9033-6d5e1579f821\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.409693 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408939 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a56f0ac-0efc-4afe-a83b-d44f0c11da3d-host-slash\") pod \"iptables-alerter-qfrhd\" (UID: \"8a56f0ac-0efc-4afe-a83b-d44f0c11da3d\") " pod="openshift-network-operator/iptables-alerter-qfrhd" Sep 11 07:59:25.409693 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408966 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-etc-sysctl-conf\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.409693 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.408989 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-host\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.409693 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.409025 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0d722217-e915-4d81-9033-6d5e1579f821-sys-fs\") pod \"aws-ebs-csi-driver-node-vr95q\" (UID: \"0d722217-e915-4d81-9033-6d5e1579f821\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.409693 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.409048 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-etc-sysconfig\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.409693 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.409071 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-etc-kubernetes\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.409693 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.409092 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-sys\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.409693 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.409112 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-tmp\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.410473 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.409168 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0d722217-e915-4d81-9033-6d5e1579f821-socket-dir\") pod \"aws-ebs-csi-driver-node-vr95q\" (UID: \"0d722217-e915-4d81-9033-6d5e1579f821\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.410473 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.409196 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7h8v\" (UniqueName: \"kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v\") pod \"network-check-target-bvr7m\" (UID: \"16110ca4-d538-4c5e-8def-a99a002c9735\") " pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:25.410473 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.409234 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/967e46d4-81f0-49ce-a6ee-d8737cc7ad96-konnectivity-ca\") pod \"konnectivity-agent-ddqh8\" (UID: \"967e46d4-81f0-49ce-a6ee-d8737cc7ad96\") " pod="kube-system/konnectivity-agent-ddqh8" Sep 11 07:59:25.410473 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.409257 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-var-lib-kubelet\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.410473 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.409279 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkt9h\" (UniqueName: \"kubernetes.io/projected/0d722217-e915-4d81-9033-6d5e1579f821-kube-api-access-gkt9h\") pod \"aws-ebs-csi-driver-node-vr95q\" (UID: \"0d722217-e915-4d81-9033-6d5e1579f821\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.411233 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.410892 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.411696 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.411521 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 11 07:59:25.412530 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.412506 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 11 07:59:25.412940 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.412919 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-n8nvm\"" Sep 11 07:59:25.413167 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.413142 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:25.413260 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:25.413210 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkcn8" podUID="c5234baf-5b8f-46ff-9fa3-f217c0087ded" Sep 11 07:59:25.413328 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.413283 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 11 07:59:25.413564 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.413542 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 11 07:59:25.413651 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.413575 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 11 07:59:25.413651 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.413542 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 11 07:59:25.413651 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.413642 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 11 07:59:25.439998 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.439978 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pr79f" Sep 11 07:59:25.449735 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.449707 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pr79f" Sep 11 07:59:25.501097 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.501079 2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 11 07:59:25.502070 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.502054 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 07:59:25.503738 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.503696 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-245.ec2.internal" event={"ID":"e11886251d79d45e61a827aee7470670","Type":"ContainerStarted","Data":"0683d674db74ce77d5b38c9adf72da3a87e6ebf0bd79026b048e3902688c8c94"} Sep 11 07:59:25.505592 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.505570 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal" event={"ID":"16b40a950e377baf3ad61bb98b577811","Type":"ContainerStarted","Data":"ce87eb78eb404945fa2fe0bccd2bd5d963cae992b2e1a5f0e8a492a0d96ddf82"} Sep 11 07:59:25.509867 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.509844 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-host-cni-netd\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.509974 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.509884 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0d722217-e915-4d81-9033-6d5e1579f821-etc-selinux\") pod \"aws-ebs-csi-driver-node-vr95q\" (UID: \"0d722217-e915-4d81-9033-6d5e1579f821\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.509974 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.509932 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2cb363cc-5b2b-47df-a9a4-29c33b6a67f5-cnibin\") pod \"multus-additional-cni-plugins-8hfsw\" (UID: \"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5\") " pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.509974 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.509955 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-system-cni-dir\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.510125 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.509976 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-hostroot\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.510125 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.509998 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-node-log\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.510125 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510024 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-host-run-ovn-kubernetes\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.510125 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510049 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8a56f0ac-0efc-4afe-a83b-d44f0c11da3d-iptables-alerter-script\") pod \"iptables-alerter-qfrhd\" (UID: \"8a56f0ac-0efc-4afe-a83b-d44f0c11da3d\") " pod="openshift-network-operator/iptables-alerter-qfrhd" Sep 11 07:59:25.510125 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510070 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/98d2f5ae-5780-4110-935d-e3781cc271f7-dbus\") pod \"global-pull-secret-syncer-f8ptc\" (UID: \"98d2f5ae-5780-4110-935d-e3781cc271f7\") " pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:25.510125 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510092 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-host-var-lib-cni-multus\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.510125 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510116 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06a3e034-7768-4e68-8ff4-4678f3cd69e7-multus-daemon-config\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.510445 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510137 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-ovnkube-config\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.510445 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510161 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-ovn-node-metrics-cert\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.510445 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510187 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-etc-sysctl-conf\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.510445 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510209 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-host\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.510445 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510234 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-cnibin\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.510445 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510256 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-etc-openvswitch\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.510445 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510283 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-tmp\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.510445 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510306 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0d722217-e915-4d81-9033-6d5e1579f821-socket-dir\") pod \"aws-ebs-csi-driver-node-vr95q\" (UID: \"0d722217-e915-4d81-9033-6d5e1579f821\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.510445 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510331 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7h8v\" (UniqueName: \"kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v\") pod \"network-check-target-bvr7m\" (UID: \"16110ca4-d538-4c5e-8def-a99a002c9735\") " pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:25.510445 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510349 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-multus-cni-dir\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.510445 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510368 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-host-run-netns\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.510445 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510385 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-host-slash\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.510445 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510401 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-env-overrides\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.510445 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510422 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-var-lib-kubelet\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.510445 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510446 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkt9h\" (UniqueName: \"kubernetes.io/projected/0d722217-e915-4d81-9033-6d5e1579f821-kube-api-access-gkt9h\") pod \"aws-ebs-csi-driver-node-vr95q\" (UID: \"0d722217-e915-4d81-9033-6d5e1579f821\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.511061 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510472 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2cb363cc-5b2b-47df-a9a4-29c33b6a67f5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8hfsw\" (UID: \"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5\") " pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.511061 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510496 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-host-var-lib-kubelet\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.511061 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510521 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0d722217-e915-4d81-9033-6d5e1579f821-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vr95q\" (UID: \"0d722217-e915-4d81-9033-6d5e1579f821\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.511061 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510544 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret\") pod \"global-pull-secret-syncer-f8ptc\" (UID: \"98d2f5ae-5780-4110-935d-e3781cc271f7\") " pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:25.511061 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510566 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-var-lib-openvswitch\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.511061 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510589 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-etc-tuned\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.511061 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510612 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c61a8ab5-9ed9-4f3d-9e32-935de3901e01-hosts-file\") pod \"node-resolver-vqjrt\" (UID: \"c61a8ab5-9ed9-4f3d-9e32-935de3901e01\") " pod="openshift-dns/node-resolver-vqjrt" Sep 11 07:59:25.511061 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510631 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-lib-modules\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.511061 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510649 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0d722217-e915-4d81-9033-6d5e1579f821-registration-dir\") pod \"aws-ebs-csi-driver-node-vr95q\" (UID: \"0d722217-e915-4d81-9033-6d5e1579f821\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.511061 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510664 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2cb363cc-5b2b-47df-a9a4-29c33b6a67f5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8hfsw\" (UID: \"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5\") " pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.511061 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510677 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-log-socket\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.511061 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510691 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.511061 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510724 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/88bb78bf-db17-415d-b3e8-b88236c84350-host\") pod \"node-ca-6skrk\" (UID: \"88bb78bf-db17-415d-b3e8-b88236c84350\") " pod="openshift-image-registry/node-ca-6skrk" Sep 11 07:59:25.511061 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510740 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk5zp\" (UniqueName: \"kubernetes.io/projected/8a56f0ac-0efc-4afe-a83b-d44f0c11da3d-kube-api-access-bk5zp\") pod \"iptables-alerter-qfrhd\" (UID: \"8a56f0ac-0efc-4afe-a83b-d44f0c11da3d\") " pod="openshift-network-operator/iptables-alerter-qfrhd" Sep 11 07:59:25.511061 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510755 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-host-run-k8s-cni-cncf-io\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.511061 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510769 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c61a8ab5-9ed9-4f3d-9e32-935de3901e01-tmp-dir\") pod \"node-resolver-vqjrt\" (UID: \"c61a8ab5-9ed9-4f3d-9e32-935de3901e01\") " pod="openshift-dns/node-resolver-vqjrt" Sep 11 07:59:25.511061 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510784 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-run\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.511793 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510799 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w67cx\" (UniqueName: \"kubernetes.io/projected/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-kube-api-access-w67cx\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.511793 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510813 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2cb363cc-5b2b-47df-a9a4-29c33b6a67f5-cni-binary-copy\") pod \"multus-additional-cni-plugins-8hfsw\" (UID: \"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5\") " pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.511793 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510830 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb9jp\" (UniqueName: \"kubernetes.io/projected/2cb363cc-5b2b-47df-a9a4-29c33b6a67f5-kube-api-access-nb9jp\") pod \"multus-additional-cni-plugins-8hfsw\" (UID: \"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5\") " pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.511793 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510853 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-multus-socket-dir-parent\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.511793 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510873 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-multus-conf-dir\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.511793 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510887 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-host-run-multus-certs\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.511793 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510922 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k2k2\" (UniqueName: \"kubernetes.io/projected/c61a8ab5-9ed9-4f3d-9e32-935de3901e01-kube-api-access-4k2k2\") pod \"node-resolver-vqjrt\" (UID: \"c61a8ab5-9ed9-4f3d-9e32-935de3901e01\") " pod="openshift-dns/node-resolver-vqjrt" Sep 11 07:59:25.511793 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510949 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-etc-modprobe-d\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.511793 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510985 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-var-lib-kubelet\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.511793 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.510985 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-etc-systemd\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.511793 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511037 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0d722217-e915-4d81-9033-6d5e1579f821-device-dir\") pod \"aws-ebs-csi-driver-node-vr95q\" (UID: \"0d722217-e915-4d81-9033-6d5e1579f821\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.511793 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511047 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-etc-systemd\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.511793 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511066 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs\") pod \"network-metrics-daemon-mkcn8\" (UID: \"c5234baf-5b8f-46ff-9fa3-f217c0087ded\") " pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:25.511793 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511091 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06a3e034-7768-4e68-8ff4-4678f3cd69e7-cni-binary-copy\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.511793 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511116 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-etc-kubernetes\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.511793 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511143 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-run-openvswitch\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.511793 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511169 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a56f0ac-0efc-4afe-a83b-d44f0c11da3d-host-slash\") pod \"iptables-alerter-qfrhd\" (UID: \"8a56f0ac-0efc-4afe-a83b-d44f0c11da3d\") " pod="openshift-network-operator/iptables-alerter-qfrhd" Sep 11 07:59:25.512598 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511193 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0d722217-e915-4d81-9033-6d5e1579f821-sys-fs\") pod \"aws-ebs-csi-driver-node-vr95q\" (UID: \"0d722217-e915-4d81-9033-6d5e1579f821\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.512598 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511216 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-host-run-netns\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.512598 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511241 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-host-cni-bin\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.512598 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511266 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-etc-sysconfig\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.512598 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511294 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-etc-kubernetes\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.512598 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511319 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-sys\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.512598 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511339 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0d722217-e915-4d81-9033-6d5e1579f821-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vr95q\" (UID: \"0d722217-e915-4d81-9033-6d5e1579f821\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.512598 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511344 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-host-var-lib-cni-bin\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.512598 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511369 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56grd\" (UniqueName: \"kubernetes.io/projected/06a3e034-7768-4e68-8ff4-4678f3cd69e7-kube-api-access-56grd\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.512598 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511394 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-systemd-units\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.512598 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511420 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxntn\" (UniqueName: \"kubernetes.io/projected/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-kube-api-access-jxntn\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.512598 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511447 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/967e46d4-81f0-49ce-a6ee-d8737cc7ad96-konnectivity-ca\") pod \"konnectivity-agent-ddqh8\" (UID: \"967e46d4-81f0-49ce-a6ee-d8737cc7ad96\") " pod="kube-system/konnectivity-agent-ddqh8" Sep 11 07:59:25.512598 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511471 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2cb363cc-5b2b-47df-a9a4-29c33b6a67f5-system-cni-dir\") pod \"multus-additional-cni-plugins-8hfsw\" (UID: \"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5\") " pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.512598 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511496 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2cb363cc-5b2b-47df-a9a4-29c33b6a67f5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8hfsw\" (UID: \"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5\") " pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.512598 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511535 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-host-kubelet\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.512598 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511558 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-run-systemd\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.512598 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511582 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-ovnkube-script-lib\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.513389 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511608 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k9zb\" (UniqueName: \"kubernetes.io/projected/c5234baf-5b8f-46ff-9fa3-f217c0087ded-kube-api-access-9k9zb\") pod \"network-metrics-daemon-mkcn8\" (UID: \"c5234baf-5b8f-46ff-9fa3-f217c0087ded\") " pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:25.513389 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511627 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-run-ovn\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.513389 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511647 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/88bb78bf-db17-415d-b3e8-b88236c84350-serviceca\") pod \"node-ca-6skrk\" (UID: \"88bb78bf-db17-415d-b3e8-b88236c84350\") " pod="openshift-image-registry/node-ca-6skrk" Sep 11 07:59:25.513389 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511670 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7fm8\" (UniqueName: \"kubernetes.io/projected/88bb78bf-db17-415d-b3e8-b88236c84350-kube-api-access-k7fm8\") pod \"node-ca-6skrk\" (UID: \"88bb78bf-db17-415d-b3e8-b88236c84350\") " pod="openshift-image-registry/node-ca-6skrk" Sep 11 07:59:25.513389 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511690 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/967e46d4-81f0-49ce-a6ee-d8737cc7ad96-agent-certs\") pod \"konnectivity-agent-ddqh8\" (UID: \"967e46d4-81f0-49ce-a6ee-d8737cc7ad96\") " pod="kube-system/konnectivity-agent-ddqh8" Sep 11 07:59:25.513389 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511710 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-etc-sysctl-d\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.513389 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511732 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/98d2f5ae-5780-4110-935d-e3781cc271f7-kubelet-config\") pod \"global-pull-secret-syncer-f8ptc\" (UID: \"98d2f5ae-5780-4110-935d-e3781cc271f7\") " pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:25.513389 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511872 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0d722217-e915-4d81-9033-6d5e1579f821-etc-selinux\") pod \"aws-ebs-csi-driver-node-vr95q\" (UID: \"0d722217-e915-4d81-9033-6d5e1579f821\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.513389 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511884 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c61a8ab5-9ed9-4f3d-9e32-935de3901e01-hosts-file\") pod \"node-resolver-vqjrt\" (UID: \"c61a8ab5-9ed9-4f3d-9e32-935de3901e01\") " pod="openshift-dns/node-resolver-vqjrt" Sep 11 07:59:25.513389 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.511966 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0d722217-e915-4d81-9033-6d5e1579f821-registration-dir\") pod \"aws-ebs-csi-driver-node-vr95q\" (UID: \"0d722217-e915-4d81-9033-6d5e1579f821\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.513389 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.512017 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-host\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.513389 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.512189 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0d722217-e915-4d81-9033-6d5e1579f821-socket-dir\") pod \"aws-ebs-csi-driver-node-vr95q\" (UID: \"0d722217-e915-4d81-9033-6d5e1579f821\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.513389 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.512234 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/88bb78bf-db17-415d-b3e8-b88236c84350-host\") pod \"node-ca-6skrk\" (UID: \"88bb78bf-db17-415d-b3e8-b88236c84350\") " pod="openshift-image-registry/node-ca-6skrk" Sep 11 07:59:25.513389 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.512276 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0d722217-e915-4d81-9033-6d5e1579f821-device-dir\") pod \"aws-ebs-csi-driver-node-vr95q\" (UID: \"0d722217-e915-4d81-9033-6d5e1579f821\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.513389 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.512337 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a56f0ac-0efc-4afe-a83b-d44f0c11da3d-host-slash\") pod \"iptables-alerter-qfrhd\" (UID: \"8a56f0ac-0efc-4afe-a83b-d44f0c11da3d\") " pod="openshift-network-operator/iptables-alerter-qfrhd" Sep 11 07:59:25.513389 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.512386 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0d722217-e915-4d81-9033-6d5e1579f821-sys-fs\") pod \"aws-ebs-csi-driver-node-vr95q\" (UID: \"0d722217-e915-4d81-9033-6d5e1579f821\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.513389 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.512502 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-etc-kubernetes\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.513389 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.512512 2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 11 07:59:25.514156 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.512552 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-sys\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.514156 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.512662 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c61a8ab5-9ed9-4f3d-9e32-935de3901e01-tmp-dir\") pod \"node-resolver-vqjrt\" (UID: \"c61a8ab5-9ed9-4f3d-9e32-935de3901e01\") " pod="openshift-dns/node-resolver-vqjrt" Sep 11 07:59:25.514156 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.512756 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-etc-sysconfig\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.514156 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.512853 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-run\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.514156 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.513099 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/967e46d4-81f0-49ce-a6ee-d8737cc7ad96-konnectivity-ca\") pod \"konnectivity-agent-ddqh8\" (UID: \"967e46d4-81f0-49ce-a6ee-d8737cc7ad96\") " pod="kube-system/konnectivity-agent-ddqh8" Sep 11 07:59:25.514156 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.513138 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-etc-modprobe-d\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.514156 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.513285 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-lib-modules\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.514156 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.513396 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-etc-sysctl-d\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.514156 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.513523 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/88bb78bf-db17-415d-b3e8-b88236c84350-serviceca\") pod \"node-ca-6skrk\" (UID: \"88bb78bf-db17-415d-b3e8-b88236c84350\") " pod="openshift-image-registry/node-ca-6skrk" Sep 11 07:59:25.514156 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.513673 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-etc-sysctl-conf\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.514156 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.513864 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8a56f0ac-0efc-4afe-a83b-d44f0c11da3d-iptables-alerter-script\") pod \"iptables-alerter-qfrhd\" (UID: \"8a56f0ac-0efc-4afe-a83b-d44f0c11da3d\") " pod="openshift-network-operator/iptables-alerter-qfrhd" Sep 11 07:59:25.518855 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.518046 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-etc-tuned\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.521247 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:25.519281 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 07:59:25.521247 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:25.519299 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 07:59:25.521247 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:25.519312 2557 projected.go:196] Error preparing data for projected volume kube-api-access-r7h8v for pod openshift-network-diagnostics/network-check-target-bvr7m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 07:59:25.521247 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:25.519368 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v podName:16110ca4-d538-4c5e-8def-a99a002c9735 nodeName:}" failed. No retries permitted until 2026-09-11 07:59:26.019349948 +0000 UTC m=+3.025067602 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-r7h8v" (UniqueName: "kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v") pod "network-check-target-bvr7m" (UID: "16110ca4-d538-4c5e-8def-a99a002c9735") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 07:59:25.521247 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.520192 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-tmp\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.522986 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.522964 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/967e46d4-81f0-49ce-a6ee-d8737cc7ad96-agent-certs\") pod \"konnectivity-agent-ddqh8\" (UID: \"967e46d4-81f0-49ce-a6ee-d8737cc7ad96\") " pod="kube-system/konnectivity-agent-ddqh8" Sep 11 07:59:25.524146 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.523772 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w67cx\" (UniqueName: \"kubernetes.io/projected/097541bd-1aa0-4851-8b2c-8a5452ecd3fc-kube-api-access-w67cx\") pod \"tuned-hgrfs\" (UID: \"097541bd-1aa0-4851-8b2c-8a5452ecd3fc\") " pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.524234 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.524191 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k2k2\" (UniqueName: \"kubernetes.io/projected/c61a8ab5-9ed9-4f3d-9e32-935de3901e01-kube-api-access-4k2k2\") pod \"node-resolver-vqjrt\" (UID: \"c61a8ab5-9ed9-4f3d-9e32-935de3901e01\") " pod="openshift-dns/node-resolver-vqjrt" Sep 11 07:59:25.524590 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.524570 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkt9h\" (UniqueName: \"kubernetes.io/projected/0d722217-e915-4d81-9033-6d5e1579f821-kube-api-access-gkt9h\") pod \"aws-ebs-csi-driver-node-vr95q\" (UID: \"0d722217-e915-4d81-9033-6d5e1579f821\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.526045 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.525958 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk5zp\" (UniqueName: \"kubernetes.io/projected/8a56f0ac-0efc-4afe-a83b-d44f0c11da3d-kube-api-access-bk5zp\") pod \"iptables-alerter-qfrhd\" (UID: \"8a56f0ac-0efc-4afe-a83b-d44f0c11da3d\") " pod="openshift-network-operator/iptables-alerter-qfrhd" Sep 11 07:59:25.526138 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.526107 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7fm8\" (UniqueName: \"kubernetes.io/projected/88bb78bf-db17-415d-b3e8-b88236c84350-kube-api-access-k7fm8\") pod \"node-ca-6skrk\" (UID: \"88bb78bf-db17-415d-b3e8-b88236c84350\") " pod="openshift-image-registry/node-ca-6skrk" Sep 11 07:59:25.612555 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.612518 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/98d2f5ae-5780-4110-935d-e3781cc271f7-dbus\") pod \"global-pull-secret-syncer-f8ptc\" (UID: \"98d2f5ae-5780-4110-935d-e3781cc271f7\") " pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:25.612555 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.612556 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-host-var-lib-cni-multus\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.612743 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.612612 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-host-var-lib-cni-multus\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.612743 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.612635 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/98d2f5ae-5780-4110-935d-e3781cc271f7-dbus\") pod \"global-pull-secret-syncer-f8ptc\" (UID: \"98d2f5ae-5780-4110-935d-e3781cc271f7\") " pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:25.612743 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.612680 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06a3e034-7768-4e68-8ff4-4678f3cd69e7-multus-daemon-config\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.612743 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.612707 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-ovnkube-config\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.612743 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.612733 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-ovn-node-metrics-cert\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.613004 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.612760 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-cnibin\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.613004 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.612783 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-etc-openvswitch\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.613004 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.612825 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-etc-openvswitch\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.613004 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.612883 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-cnibin\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.613004 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.612932 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-multus-cni-dir\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.613004 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.612947 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-host-run-netns\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.613004 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.612962 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-host-slash\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.613004 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.612979 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-env-overrides\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.613345 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613006 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2cb363cc-5b2b-47df-a9a4-29c33b6a67f5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8hfsw\" (UID: \"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5\") " pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.613345 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613027 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-host-var-lib-kubelet\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.613345 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613048 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret\") pod \"global-pull-secret-syncer-f8ptc\" (UID: \"98d2f5ae-5780-4110-935d-e3781cc271f7\") " pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:25.613345 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613065 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-var-lib-openvswitch\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.613345 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613083 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2cb363cc-5b2b-47df-a9a4-29c33b6a67f5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8hfsw\" (UID: \"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5\") " pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.613345 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613097 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-log-socket\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.613345 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613114 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.613345 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613141 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-host-run-k8s-cni-cncf-io\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.613345 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613166 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2cb363cc-5b2b-47df-a9a4-29c33b6a67f5-cni-binary-copy\") pod \"multus-additional-cni-plugins-8hfsw\" (UID: \"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5\") " pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.613345 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613189 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb9jp\" (UniqueName: \"kubernetes.io/projected/2cb363cc-5b2b-47df-a9a4-29c33b6a67f5-kube-api-access-nb9jp\") pod \"multus-additional-cni-plugins-8hfsw\" (UID: \"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5\") " pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.613345 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613216 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-multus-socket-dir-parent\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.613345 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613239 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-multus-conf-dir\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.613345 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613261 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-host-run-multus-certs\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.613345 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613286 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs\") pod \"network-metrics-daemon-mkcn8\" (UID: \"c5234baf-5b8f-46ff-9fa3-f217c0087ded\") " pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:25.613345 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613301 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06a3e034-7768-4e68-8ff4-4678f3cd69e7-multus-daemon-config\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.613345 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613309 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06a3e034-7768-4e68-8ff4-4678f3cd69e7-cni-binary-copy\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.613345 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613321 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-ovnkube-config\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.613928 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613334 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-etc-kubernetes\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.613928 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613379 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-host-run-k8s-cni-cncf-io\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.613928 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613395 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-var-lib-openvswitch\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.613928 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613414 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-host-var-lib-kubelet\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.613928 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613477 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-log-socket\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.613928 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613525 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.613928 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613568 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-host-run-netns\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.613928 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613624 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-multus-cni-dir\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.613928 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613660 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-run-openvswitch\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.613928 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613654 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2cb363cc-5b2b-47df-a9a4-29c33b6a67f5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8hfsw\" (UID: \"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5\") " pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.613928 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613702 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-host-run-netns\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.613928 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613714 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-run-openvswitch\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.613928 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613726 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-host-cni-bin\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.613928 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613727 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-host-slash\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.613928 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613758 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-host-var-lib-cni-bin\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.613928 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613780 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-etc-kubernetes\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.613928 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613786 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56grd\" (UniqueName: \"kubernetes.io/projected/06a3e034-7768-4e68-8ff4-4678f3cd69e7-kube-api-access-56grd\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.613928 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613820 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-host-cni-bin\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.614480 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613837 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2cb363cc-5b2b-47df-a9a4-29c33b6a67f5-cni-binary-copy\") pod \"multus-additional-cni-plugins-8hfsw\" (UID: \"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5\") " pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.614480 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613847 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-systemd-units\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.614480 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613876 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-systemd-units\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.614480 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613876 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2cb363cc-5b2b-47df-a9a4-29c33b6a67f5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8hfsw\" (UID: \"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5\") " pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.614480 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613885 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxntn\" (UniqueName: \"kubernetes.io/projected/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-kube-api-access-jxntn\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.614480 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613929 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-host-run-netns\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.614480 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613933 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2cb363cc-5b2b-47df-a9a4-29c33b6a67f5-system-cni-dir\") pod \"multus-additional-cni-plugins-8hfsw\" (UID: \"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5\") " pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.614480 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613950 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-host-var-lib-cni-bin\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.614480 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613963 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2cb363cc-5b2b-47df-a9a4-29c33b6a67f5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8hfsw\" (UID: \"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5\") " pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.614480 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.613995 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-host-kubelet\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.614480 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:25.614017 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 07:59:25.614480 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614045 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-host-kubelet\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.614480 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614037 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-multus-conf-dir\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.614480 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:25.614086 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs podName:c5234baf-5b8f-46ff-9fa3-f217c0087ded nodeName:}" failed. No retries permitted until 2026-09-11 07:59:26.114068099 +0000 UTC m=+3.119785752 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs") pod "network-metrics-daemon-mkcn8" (UID: "c5234baf-5b8f-46ff-9fa3-f217c0087ded") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 07:59:25.614480 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614125 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-host-run-multus-certs\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.614480 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614017 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-multus-socket-dir-parent\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.614480 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:25.614146 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 07:59:25.614480 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614190 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-env-overrides\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.615129 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:25.614225 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret podName:98d2f5ae-5780-4110-935d-e3781cc271f7 nodeName:}" failed. No retries permitted until 2026-09-11 07:59:26.114192292 +0000 UTC m=+3.119909935 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret") pod "global-pull-secret-syncer-f8ptc" (UID: "98d2f5ae-5780-4110-935d-e3781cc271f7") : object "kube-system"/"original-pull-secret" not registered Sep 11 07:59:25.615129 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614251 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-run-systemd\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.615129 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614276 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-ovnkube-script-lib\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.615129 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614310 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-run-systemd\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.615129 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614322 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k9zb\" (UniqueName: \"kubernetes.io/projected/c5234baf-5b8f-46ff-9fa3-f217c0087ded-kube-api-access-9k9zb\") pod \"network-metrics-daemon-mkcn8\" (UID: \"c5234baf-5b8f-46ff-9fa3-f217c0087ded\") " pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:25.615129 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614379 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2cb363cc-5b2b-47df-a9a4-29c33b6a67f5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8hfsw\" (UID: \"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5\") " pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.615129 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614427 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-run-ovn\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.615129 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614383 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-run-ovn\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.615129 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614478 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/98d2f5ae-5780-4110-935d-e3781cc271f7-kubelet-config\") pod \"global-pull-secret-syncer-f8ptc\" (UID: \"98d2f5ae-5780-4110-935d-e3781cc271f7\") " pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:25.615129 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614508 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-host-cni-netd\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.615129 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614540 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2cb363cc-5b2b-47df-a9a4-29c33b6a67f5-cnibin\") pod \"multus-additional-cni-plugins-8hfsw\" (UID: \"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5\") " pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.615129 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614565 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-system-cni-dir\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.615129 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614592 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-hostroot\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.615129 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614619 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-node-log\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.615129 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614646 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-host-run-ovn-kubernetes\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.615129 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614680 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06a3e034-7768-4e68-8ff4-4678f3cd69e7-cni-binary-copy\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.615129 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614723 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2cb363cc-5b2b-47df-a9a4-29c33b6a67f5-system-cni-dir\") pod \"multus-additional-cni-plugins-8hfsw\" (UID: \"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5\") " pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.615975 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614735 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/98d2f5ae-5780-4110-935d-e3781cc271f7-kubelet-config\") pod \"global-pull-secret-syncer-f8ptc\" (UID: \"98d2f5ae-5780-4110-935d-e3781cc271f7\") " pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:25.615975 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614760 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-host-run-ovn-kubernetes\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.615975 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614774 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-host-cni-netd\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.615975 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614808 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-hostroot\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.615975 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614832 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-node-log\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.615975 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614840 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06a3e034-7768-4e68-8ff4-4678f3cd69e7-system-cni-dir\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.615975 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.614857 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2cb363cc-5b2b-47df-a9a4-29c33b6a67f5-cnibin\") pod \"multus-additional-cni-plugins-8hfsw\" (UID: \"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5\") " pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.615975 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.615026 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-ovnkube-script-lib\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.617532 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.616730 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-ovn-node-metrics-cert\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.626005 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.625893 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxntn\" (UniqueName: \"kubernetes.io/projected/0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08-kube-api-access-jxntn\") pod \"ovnkube-node-9j5fx\" (UID: \"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08\") " pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.626822 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.626761 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56grd\" (UniqueName: \"kubernetes.io/projected/06a3e034-7768-4e68-8ff4-4678f3cd69e7-kube-api-access-56grd\") pod \"multus-797fj\" (UID: \"06a3e034-7768-4e68-8ff4-4678f3cd69e7\") " pod="openshift-multus/multus-797fj" Sep 11 07:59:25.627267 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.627221 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb9jp\" (UniqueName: \"kubernetes.io/projected/2cb363cc-5b2b-47df-a9a4-29c33b6a67f5-kube-api-access-nb9jp\") pod \"multus-additional-cni-plugins-8hfsw\" (UID: \"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5\") " pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.627355 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.627267 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k9zb\" (UniqueName: \"kubernetes.io/projected/c5234baf-5b8f-46ff-9fa3-f217c0087ded-kube-api-access-9k9zb\") pod \"network-metrics-daemon-mkcn8\" (UID: \"c5234baf-5b8f-46ff-9fa3-f217c0087ded\") " pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:25.699910 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.699859 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ddqh8" Sep 11 07:59:25.711357 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:25.711330 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod967e46d4_81f0_49ce_a6ee_d8737cc7ad96.slice/crio-5cb5167bc9e519844fcd354338544e8053c6c11360cc886c9489b7fe374fe3e5 WatchSource:0}: Error finding container 5cb5167bc9e519844fcd354338544e8053c6c11360cc886c9489b7fe374fe3e5: Status 404 returned error can't find the container with id 5cb5167bc9e519844fcd354338544e8053c6c11360cc886c9489b7fe374fe3e5 Sep 11 07:59:25.712318 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.712298 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qfrhd" Sep 11 07:59:25.722528 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.722507 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6skrk" Sep 11 07:59:25.732252 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:25.732220 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88bb78bf_db17_415d_b3e8_b88236c84350.slice/crio-cff86bb2fbb9afd817fabc6c68a28e1084ea2446ea615530682d0a6e6a94c9cc WatchSource:0}: Error finding container cff86bb2fbb9afd817fabc6c68a28e1084ea2446ea615530682d0a6e6a94c9cc: Status 404 returned error can't find the container with id cff86bb2fbb9afd817fabc6c68a28e1084ea2446ea615530682d0a6e6a94c9cc Sep 11 07:59:25.733098 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.733081 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" Sep 11 07:59:25.740477 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.740453 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" Sep 11 07:59:25.742637 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:25.742616 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod097541bd_1aa0_4851_8b2c_8a5452ecd3fc.slice/crio-345ec3f822fb46804f8d6a4d59ae3a1aa3b935b3019f1b5f6958c356cf97dd42 WatchSource:0}: Error finding container 345ec3f822fb46804f8d6a4d59ae3a1aa3b935b3019f1b5f6958c356cf97dd42: Status 404 returned error can't find the container with id 345ec3f822fb46804f8d6a4d59ae3a1aa3b935b3019f1b5f6958c356cf97dd42 Sep 11 07:59:25.749423 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.749402 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vqjrt" Sep 11 07:59:25.750618 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:25.750592 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d722217_e915_4d81_9033_6d5e1579f821.slice/crio-3b89bb6422e36b13591f8a80bf3aa0e96c14a07ddc7055a6237811dfc5b1d57b WatchSource:0}: Error finding container 3b89bb6422e36b13591f8a80bf3aa0e96c14a07ddc7055a6237811dfc5b1d57b: Status 404 returned error can't find the container with id 3b89bb6422e36b13591f8a80bf3aa0e96c14a07ddc7055a6237811dfc5b1d57b Sep 11 07:59:25.757991 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.757829 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-797fj" Sep 11 07:59:25.759505 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:25.759473 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc61a8ab5_9ed9_4f3d_9e32_935de3901e01.slice/crio-6aedd436ac7913318f13adbd09f3eee1e92900f86a2aad0fd6670b4a5f92b1df WatchSource:0}: Error finding container 6aedd436ac7913318f13adbd09f3eee1e92900f86a2aad0fd6670b4a5f92b1df: Status 404 returned error can't find the container with id 6aedd436ac7913318f13adbd09f3eee1e92900f86a2aad0fd6670b4a5f92b1df Sep 11 07:59:25.766554 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.766391 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8hfsw" Sep 11 07:59:25.766820 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:25.766790 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06a3e034_7768_4e68_8ff4_4678f3cd69e7.slice/crio-868c9a038ea1c48640c2507988c3b00769dcf14d189faec84b887371d23e2cf9 WatchSource:0}: Error finding container 868c9a038ea1c48640c2507988c3b00769dcf14d189faec84b887371d23e2cf9: Status 404 returned error can't find the container with id 868c9a038ea1c48640c2507988c3b00769dcf14d189faec84b887371d23e2cf9 Sep 11 07:59:25.773297 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:25.773255 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:25.776268 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:25.776248 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cb363cc_5b2b_47df_a9a4_29c33b6a67f5.slice/crio-f087d98371463f2bf935518b0082e11a7d3654c656502ebc039da9e6ae72599f WatchSource:0}: Error finding container f087d98371463f2bf935518b0082e11a7d3654c656502ebc039da9e6ae72599f: Status 404 returned error can't find the container with id f087d98371463f2bf935518b0082e11a7d3654c656502ebc039da9e6ae72599f Sep 11 07:59:25.783843 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:25.783810 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c06f0f2_b5b9_40c2_bd00_438a9e0c0f08.slice/crio-2bdac0c70c5477c5cd03efe466e1bde286b9122e2524177137e5c0ea950a186a WatchSource:0}: Error finding container 2bdac0c70c5477c5cd03efe466e1bde286b9122e2524177137e5c0ea950a186a: Status 404 returned error can't find the container with id 2bdac0c70c5477c5cd03efe466e1bde286b9122e2524177137e5c0ea950a186a Sep 11 07:59:26.117265 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:26.117188 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs\") pod \"network-metrics-daemon-mkcn8\" (UID: \"c5234baf-5b8f-46ff-9fa3-f217c0087ded\") " pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:26.117265 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:26.117247 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7h8v\" (UniqueName: \"kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v\") pod \"network-check-target-bvr7m\" (UID: \"16110ca4-d538-4c5e-8def-a99a002c9735\") " pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:26.117449 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:26.117278 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret\") pod \"global-pull-secret-syncer-f8ptc\" (UID: \"98d2f5ae-5780-4110-935d-e3781cc271f7\") " pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:26.117449 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:26.117342 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 07:59:26.117449 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:26.117366 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 07:59:26.117449 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:26.117407 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 07:59:26.117449 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:26.117420 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs podName:c5234baf-5b8f-46ff-9fa3-f217c0087ded nodeName:}" failed. No retries permitted until 2026-09-11 07:59:27.117399527 +0000 UTC m=+4.123117183 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs") pod "network-metrics-daemon-mkcn8" (UID: "c5234baf-5b8f-46ff-9fa3-f217c0087ded") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 07:59:26.117449 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:26.117427 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 07:59:26.117449 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:26.117435 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret podName:98d2f5ae-5780-4110-935d-e3781cc271f7 nodeName:}" failed. No retries permitted until 2026-09-11 07:59:27.117429246 +0000 UTC m=+4.123146882 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret") pod "global-pull-secret-syncer-f8ptc" (UID: "98d2f5ae-5780-4110-935d-e3781cc271f7") : object "kube-system"/"original-pull-secret" not registered Sep 11 07:59:26.117449 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:26.117442 2557 projected.go:196] Error preparing data for projected volume kube-api-access-r7h8v for pod openshift-network-diagnostics/network-check-target-bvr7m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 07:59:26.117732 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:26.117486 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v podName:16110ca4-d538-4c5e-8def-a99a002c9735 nodeName:}" failed. No retries permitted until 2026-09-11 07:59:27.117470887 +0000 UTC m=+4.123188541 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7h8v" (UniqueName: "kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v") pod "network-check-target-bvr7m" (UID: "16110ca4-d538-4c5e-8def-a99a002c9735") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 07:59:26.453252 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:26.453151 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 07:54:25 +0000 UTC" deadline="2028-02-20 19:16:19.025145477 +0000 UTC" Sep 11 07:59:26.453252 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:26.453196 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12659h16m52.571953939s" Sep 11 07:59:26.510699 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:26.510663 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" event={"ID":"0d722217-e915-4d81-9033-6d5e1579f821","Type":"ContainerStarted","Data":"3b89bb6422e36b13591f8a80bf3aa0e96c14a07ddc7055a6237811dfc5b1d57b"} Sep 11 07:59:26.513770 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:26.513735 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qfrhd" event={"ID":"8a56f0ac-0efc-4afe-a83b-d44f0c11da3d","Type":"ContainerStarted","Data":"7ab0d6130a5a569a7a3971e68ae0bb7d974da0f32a62e71487b2001c1a4bfb2f"} Sep 11 07:59:26.515701 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:26.515678 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" event={"ID":"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08","Type":"ContainerStarted","Data":"2bdac0c70c5477c5cd03efe466e1bde286b9122e2524177137e5c0ea950a186a"} Sep 11 07:59:26.517620 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:26.517556 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hfsw" event={"ID":"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5","Type":"ContainerStarted","Data":"f087d98371463f2bf935518b0082e11a7d3654c656502ebc039da9e6ae72599f"} Sep 11 07:59:26.519159 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:26.519126 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-797fj" event={"ID":"06a3e034-7768-4e68-8ff4-4678f3cd69e7","Type":"ContainerStarted","Data":"868c9a038ea1c48640c2507988c3b00769dcf14d189faec84b887371d23e2cf9"} Sep 11 07:59:26.520820 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:26.520769 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vqjrt" event={"ID":"c61a8ab5-9ed9-4f3d-9e32-935de3901e01","Type":"ContainerStarted","Data":"6aedd436ac7913318f13adbd09f3eee1e92900f86a2aad0fd6670b4a5f92b1df"} Sep 11 07:59:26.522418 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:26.522395 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" event={"ID":"097541bd-1aa0-4851-8b2c-8a5452ecd3fc","Type":"ContainerStarted","Data":"345ec3f822fb46804f8d6a4d59ae3a1aa3b935b3019f1b5f6958c356cf97dd42"} Sep 11 07:59:26.524387 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:26.524360 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6skrk" event={"ID":"88bb78bf-db17-415d-b3e8-b88236c84350","Type":"ContainerStarted","Data":"cff86bb2fbb9afd817fabc6c68a28e1084ea2446ea615530682d0a6e6a94c9cc"} Sep 11 07:59:26.525892 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:26.525870 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ddqh8" event={"ID":"967e46d4-81f0-49ce-a6ee-d8737cc7ad96","Type":"ContainerStarted","Data":"5cb5167bc9e519844fcd354338544e8053c6c11360cc886c9489b7fe374fe3e5"} Sep 11 07:59:26.959599 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:26.959530 2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 07:59:27.125946 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:27.125910 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7h8v\" (UniqueName: \"kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v\") pod \"network-check-target-bvr7m\" (UID: \"16110ca4-d538-4c5e-8def-a99a002c9735\") " pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:27.126106 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:27.125970 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret\") pod \"global-pull-secret-syncer-f8ptc\" (UID: \"98d2f5ae-5780-4110-935d-e3781cc271f7\") " pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:27.126106 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:27.126013 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs\") pod \"network-metrics-daemon-mkcn8\" (UID: \"c5234baf-5b8f-46ff-9fa3-f217c0087ded\") " pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:27.126106 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:27.126048 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 07:59:27.126106 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:27.126076 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 07:59:27.126106 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:27.126089 2557 projected.go:196] Error preparing data for projected volume kube-api-access-r7h8v for pod openshift-network-diagnostics/network-check-target-bvr7m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 07:59:27.126371 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:27.126130 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 07:59:27.126371 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:27.126144 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v podName:16110ca4-d538-4c5e-8def-a99a002c9735 nodeName:}" failed. No retries permitted until 2026-09-11 07:59:29.126126414 +0000 UTC m=+6.131844073 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7h8v" (UniqueName: "kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v") pod "network-check-target-bvr7m" (UID: "16110ca4-d538-4c5e-8def-a99a002c9735") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 07:59:27.126371 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:27.126179 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs podName:c5234baf-5b8f-46ff-9fa3-f217c0087ded nodeName:}" failed. No retries permitted until 2026-09-11 07:59:29.126164621 +0000 UTC m=+6.131882275 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs") pod "network-metrics-daemon-mkcn8" (UID: "c5234baf-5b8f-46ff-9fa3-f217c0087ded") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 07:59:27.126371 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:27.126234 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 07:59:27.126371 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:27.126266 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret podName:98d2f5ae-5780-4110-935d-e3781cc271f7 nodeName:}" failed. No retries permitted until 2026-09-11 07:59:29.126255275 +0000 UTC m=+6.131972926 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret") pod "global-pull-secret-syncer-f8ptc" (UID: "98d2f5ae-5780-4110-935d-e3781cc271f7") : object "kube-system"/"original-pull-secret" not registered Sep 11 07:59:27.454013 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:27.453964 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 07:54:25 +0000 UTC" deadline="2028-06-03 18:03:47.760106472 +0000 UTC" Sep 11 07:59:27.454013 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:27.454000 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15154h4m20.30611125s" Sep 11 07:59:27.499453 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:27.499382 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:27.499580 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:27.499508 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvr7m" podUID="16110ca4-d538-4c5e-8def-a99a002c9735" Sep 11 07:59:27.499931 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:27.499891 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:27.500045 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:27.500011 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f8ptc" podUID="98d2f5ae-5780-4110-935d-e3781cc271f7" Sep 11 07:59:27.500104 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:27.500081 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:27.500195 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:27.500168 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkcn8" podUID="c5234baf-5b8f-46ff-9fa3-f217c0087ded" Sep 11 07:59:27.546045 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:27.546017 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-245.ec2.internal" event={"ID":"e11886251d79d45e61a827aee7470670","Type":"ContainerStarted","Data":"469d29c13119b336be0d9c451a8c88f9bf957a88beead7a115247e4ca620b6fc"} Sep 11 07:59:28.565997 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:28.565956 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal" event={"ID":"16b40a950e377baf3ad61bb98b577811","Type":"ContainerStarted","Data":"d13e157017c2b70637c85387441bb0efb2cc46ab7972e47d86fd4c9d1fac17a0"} Sep 11 07:59:28.585096 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:28.584329 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-245.ec2.internal" podStartSLOduration=4.584309254 podStartE2EDuration="4.584309254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 07:59:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 07:59:27.56716793 +0000 UTC m=+4.572885590" watchObservedRunningTime="2026-09-11 07:59:28.584309254 +0000 UTC m=+5.590026914" Sep 11 07:59:29.143194 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:29.143159 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs\") pod \"network-metrics-daemon-mkcn8\" (UID: \"c5234baf-5b8f-46ff-9fa3-f217c0087ded\") " pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:29.143440 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:29.143227 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7h8v\" (UniqueName: \"kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v\") pod \"network-check-target-bvr7m\" (UID: \"16110ca4-d538-4c5e-8def-a99a002c9735\") " pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:29.143440 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:29.143257 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret\") pod \"global-pull-secret-syncer-f8ptc\" (UID: \"98d2f5ae-5780-4110-935d-e3781cc271f7\") " pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:29.143440 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:29.143373 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 07:59:29.143440 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:29.143436 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret podName:98d2f5ae-5780-4110-935d-e3781cc271f7 nodeName:}" failed. No retries permitted until 2026-09-11 07:59:33.143417844 +0000 UTC m=+10.149135490 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret") pod "global-pull-secret-syncer-f8ptc" (UID: "98d2f5ae-5780-4110-935d-e3781cc271f7") : object "kube-system"/"original-pull-secret" not registered Sep 11 07:59:29.143689 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:29.143504 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 07:59:29.143689 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:29.143539 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs podName:c5234baf-5b8f-46ff-9fa3-f217c0087ded nodeName:}" failed. No retries permitted until 2026-09-11 07:59:33.143526649 +0000 UTC m=+10.149244288 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs") pod "network-metrics-daemon-mkcn8" (UID: "c5234baf-5b8f-46ff-9fa3-f217c0087ded") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 07:59:29.143689 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:29.143615 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 07:59:29.143689 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:29.143632 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 07:59:29.143689 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:29.143644 2557 projected.go:196] Error preparing data for projected volume kube-api-access-r7h8v for pod openshift-network-diagnostics/network-check-target-bvr7m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 07:59:29.143689 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:29.143676 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v podName:16110ca4-d538-4c5e-8def-a99a002c9735 nodeName:}" failed. No retries permitted until 2026-09-11 07:59:33.143665704 +0000 UTC m=+10.149383348 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7h8v" (UniqueName: "kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v") pod "network-check-target-bvr7m" (UID: "16110ca4-d538-4c5e-8def-a99a002c9735") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 07:59:29.498107 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:29.498017 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:29.498107 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:29.498061 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:29.498312 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:29.498135 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvr7m" podUID="16110ca4-d538-4c5e-8def-a99a002c9735" Sep 11 07:59:29.498312 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:29.498210 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:29.498383 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:29.498308 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f8ptc" podUID="98d2f5ae-5780-4110-935d-e3781cc271f7" Sep 11 07:59:29.498426 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:29.498399 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkcn8" podUID="c5234baf-5b8f-46ff-9fa3-f217c0087ded" Sep 11 07:59:31.497662 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:31.497619 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:31.497662 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:31.497661 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:31.498165 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:31.497749 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvr7m" podUID="16110ca4-d538-4c5e-8def-a99a002c9735" Sep 11 07:59:31.498165 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:31.497623 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:31.498165 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:31.497884 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f8ptc" podUID="98d2f5ae-5780-4110-935d-e3781cc271f7" Sep 11 07:59:31.498165 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:31.497996 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkcn8" podUID="c5234baf-5b8f-46ff-9fa3-f217c0087ded" Sep 11 07:59:31.574637 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:31.574378 2557 generic.go:356] "Generic (PLEG): container finished" podID="16b40a950e377baf3ad61bb98b577811" containerID="d13e157017c2b70637c85387441bb0efb2cc46ab7972e47d86fd4c9d1fac17a0" exitCode=0 Sep 11 07:59:31.574637 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:31.574454 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal" event={"ID":"16b40a950e377baf3ad61bb98b577811","Type":"ContainerDied","Data":"d13e157017c2b70637c85387441bb0efb2cc46ab7972e47d86fd4c9d1fac17a0"} Sep 11 07:59:33.176706 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:33.176653 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7h8v\" (UniqueName: \"kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v\") pod \"network-check-target-bvr7m\" (UID: \"16110ca4-d538-4c5e-8def-a99a002c9735\") " pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:33.176706 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:33.176702 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret\") pod \"global-pull-secret-syncer-f8ptc\" (UID: \"98d2f5ae-5780-4110-935d-e3781cc271f7\") " pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:33.177206 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:33.176747 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs\") pod \"network-metrics-daemon-mkcn8\" (UID: \"c5234baf-5b8f-46ff-9fa3-f217c0087ded\") " pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:33.177206 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:33.176812 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 07:59:33.177206 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:33.176832 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 07:59:33.177206 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:33.176842 2557 projected.go:196] Error preparing data for projected volume kube-api-access-r7h8v for pod openshift-network-diagnostics/network-check-target-bvr7m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 07:59:33.177206 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:33.176865 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 07:59:33.177206 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:33.176885 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v podName:16110ca4-d538-4c5e-8def-a99a002c9735 nodeName:}" failed. No retries permitted until 2026-09-11 07:59:41.176871226 +0000 UTC m=+18.182588861 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7h8v" (UniqueName: "kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v") pod "network-check-target-bvr7m" (UID: "16110ca4-d538-4c5e-8def-a99a002c9735") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 07:59:33.177206 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:33.176934 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs podName:c5234baf-5b8f-46ff-9fa3-f217c0087ded nodeName:}" failed. No retries permitted until 2026-09-11 07:59:41.176917193 +0000 UTC m=+18.182634828 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs") pod "network-metrics-daemon-mkcn8" (UID: "c5234baf-5b8f-46ff-9fa3-f217c0087ded") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 07:59:33.177206 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:33.176981 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 07:59:33.177206 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:33.177025 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret podName:98d2f5ae-5780-4110-935d-e3781cc271f7 nodeName:}" failed. No retries permitted until 2026-09-11 07:59:41.177011204 +0000 UTC m=+18.182728864 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret") pod "global-pull-secret-syncer-f8ptc" (UID: "98d2f5ae-5780-4110-935d-e3781cc271f7") : object "kube-system"/"original-pull-secret" not registered Sep 11 07:59:33.498067 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:33.497991 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:33.498210 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:33.498090 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvr7m" podUID="16110ca4-d538-4c5e-8def-a99a002c9735" Sep 11 07:59:33.498266 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:33.498229 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:33.498338 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:33.498310 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f8ptc" podUID="98d2f5ae-5780-4110-935d-e3781cc271f7" Sep 11 07:59:33.498460 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:33.498371 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:33.498582 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:33.498455 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkcn8" podUID="c5234baf-5b8f-46ff-9fa3-f217c0087ded" Sep 11 07:59:35.496867 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:35.496832 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:35.497304 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:35.496838 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:35.497304 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:35.496979 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvr7m" podUID="16110ca4-d538-4c5e-8def-a99a002c9735" Sep 11 07:59:35.497304 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:35.496846 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:35.497304 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:35.497061 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f8ptc" podUID="98d2f5ae-5780-4110-935d-e3781cc271f7" Sep 11 07:59:35.497304 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:35.497108 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkcn8" podUID="c5234baf-5b8f-46ff-9fa3-f217c0087ded" Sep 11 07:59:37.497085 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:37.497049 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:37.497085 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:37.497062 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:37.497555 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:37.497182 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:37.497555 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:37.497187 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f8ptc" podUID="98d2f5ae-5780-4110-935d-e3781cc271f7" Sep 11 07:59:37.497555 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:37.497280 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkcn8" podUID="c5234baf-5b8f-46ff-9fa3-f217c0087ded" Sep 11 07:59:37.497555 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:37.497347 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvr7m" podUID="16110ca4-d538-4c5e-8def-a99a002c9735" Sep 11 07:59:39.497117 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:39.497081 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:39.497478 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:39.497093 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:39.497478 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:39.497179 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f8ptc" podUID="98d2f5ae-5780-4110-935d-e3781cc271f7" Sep 11 07:59:39.497478 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:39.497204 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:39.497478 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:39.497346 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkcn8" podUID="c5234baf-5b8f-46ff-9fa3-f217c0087ded" Sep 11 07:59:39.497478 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:39.497452 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvr7m" podUID="16110ca4-d538-4c5e-8def-a99a002c9735" Sep 11 07:59:41.235717 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:41.235681 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret\") pod \"global-pull-secret-syncer-f8ptc\" (UID: \"98d2f5ae-5780-4110-935d-e3781cc271f7\") " pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:41.236187 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:41.235724 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs\") pod \"network-metrics-daemon-mkcn8\" (UID: \"c5234baf-5b8f-46ff-9fa3-f217c0087ded\") " pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:41.236187 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:41.235777 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7h8v\" (UniqueName: \"kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v\") pod \"network-check-target-bvr7m\" (UID: \"16110ca4-d538-4c5e-8def-a99a002c9735\") " pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:41.236187 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:41.235833 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 07:59:41.236187 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:41.235874 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 07:59:41.236187 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:41.235890 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 07:59:41.236187 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:41.235913 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret podName:98d2f5ae-5780-4110-935d-e3781cc271f7 nodeName:}" failed. No retries permitted until 2026-09-11 07:59:57.235880977 +0000 UTC m=+34.241598820 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret") pod "global-pull-secret-syncer-f8ptc" (UID: "98d2f5ae-5780-4110-935d-e3781cc271f7") : object "kube-system"/"original-pull-secret" not registered Sep 11 07:59:41.236187 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:41.235916 2557 projected.go:196] Error preparing data for projected volume kube-api-access-r7h8v for pod openshift-network-diagnostics/network-check-target-bvr7m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 07:59:41.236187 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:41.235913 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 07:59:41.236187 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:41.235954 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v podName:16110ca4-d538-4c5e-8def-a99a002c9735 nodeName:}" failed. No retries permitted until 2026-09-11 07:59:57.235943249 +0000 UTC m=+34.241660890 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7h8v" (UniqueName: "kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v") pod "network-check-target-bvr7m" (UID: "16110ca4-d538-4c5e-8def-a99a002c9735") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 07:59:41.236187 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:41.236005 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs podName:c5234baf-5b8f-46ff-9fa3-f217c0087ded nodeName:}" failed. No retries permitted until 2026-09-11 07:59:57.23598624 +0000 UTC m=+34.241703880 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs") pod "network-metrics-daemon-mkcn8" (UID: "c5234baf-5b8f-46ff-9fa3-f217c0087ded") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 07:59:41.497216 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:41.497112 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:41.497216 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:41.497144 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:41.497216 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:41.497112 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:41.497476 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:41.497232 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f8ptc" podUID="98d2f5ae-5780-4110-935d-e3781cc271f7" Sep 11 07:59:41.497476 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:41.497359 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkcn8" podUID="c5234baf-5b8f-46ff-9fa3-f217c0087ded" Sep 11 07:59:41.497476 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:41.497450 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvr7m" podUID="16110ca4-d538-4c5e-8def-a99a002c9735" Sep 11 07:59:43.498705 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:43.498674 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:43.499073 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:43.498768 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:43.499073 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:43.498772 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f8ptc" podUID="98d2f5ae-5780-4110-935d-e3781cc271f7" Sep 11 07:59:43.499073 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:43.498879 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkcn8" podUID="c5234baf-5b8f-46ff-9fa3-f217c0087ded" Sep 11 07:59:43.499073 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:43.498938 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:43.499073 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:43.499011 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvr7m" podUID="16110ca4-d538-4c5e-8def-a99a002c9735" Sep 11 07:59:44.597749 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.597527 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" event={"ID":"097541bd-1aa0-4851-8b2c-8a5452ecd3fc","Type":"ContainerStarted","Data":"9da05dbcf23a094f3dc9ff23f4fcb1dd2f08a881b91aead24fb34ab8bf992ebf"} Sep 11 07:59:44.599248 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.599222 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6skrk" event={"ID":"88bb78bf-db17-415d-b3e8-b88236c84350","Type":"ContainerStarted","Data":"359c45be20ad4d230a50add4692fa8bac7b05192d87f10ba8a164631e8e0af43"} Sep 11 07:59:44.600915 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.600859 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ddqh8" event={"ID":"967e46d4-81f0-49ce-a6ee-d8737cc7ad96","Type":"ContainerStarted","Data":"bc69ca4a285134af905f132034a7919f193fe6da4b4d55de7feed23f450765c1"} Sep 11 07:59:44.602999 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.602972 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal" event={"ID":"16b40a950e377baf3ad61bb98b577811","Type":"ContainerStarted","Data":"473431cb5749d8ff22faf9a11455160ad41e82672ed7ed1c5ddadcdc6df1f79a"} Sep 11 07:59:44.604699 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.604676 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" event={"ID":"0d722217-e915-4d81-9033-6d5e1579f821","Type":"ContainerStarted","Data":"1dd5b11b658f2a63f272135135985b1303df0e367c7630282757bef89b561d35"} Sep 11 07:59:44.608835 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.608799 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" event={"ID":"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08","Type":"ContainerStarted","Data":"516cbc3e501df483e299a67046003f17caa3e603af323b4647bf12048c01a363"} Sep 11 07:59:44.608954 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.608868 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" event={"ID":"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08","Type":"ContainerStarted","Data":"56b5030ea1e599084238761fb4459a7c6d4aaffffb54ef98027cbed1b7b7f1dc"} Sep 11 07:59:44.608954 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.608937 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" event={"ID":"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08","Type":"ContainerStarted","Data":"8e1c44af67e4298ec7c40d8fc9320d437fd4b571bb989dfc983ac5a3af13a319"} Sep 11 07:59:44.609074 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.608986 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" event={"ID":"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08","Type":"ContainerStarted","Data":"c5ea1320c6d36a6a36be7a2109c28b9c1ff3f637b6582e5af415bc0faac8d1d6"} Sep 11 07:59:44.609074 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.609034 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" event={"ID":"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08","Type":"ContainerStarted","Data":"77f5cee500e8824f7b923e0a2dce5b49774a911d0f00beec191154c23e448078"} Sep 11 07:59:44.609174 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.609083 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" event={"ID":"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08","Type":"ContainerStarted","Data":"84f1604f744a745f009d388bb050a22744508500c978d385764a3a3fd9c8084d"} Sep 11 07:59:44.610487 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.610460 2557 generic.go:356] "Generic (PLEG): container finished" podID="2cb363cc-5b2b-47df-a9a4-29c33b6a67f5" containerID="a7f740246ced8d19e0f90d92cda73745f9973e18502953a5ced0f3c8c9a86d4d" exitCode=0 Sep 11 07:59:44.610571 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.610539 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hfsw" event={"ID":"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5","Type":"ContainerDied","Data":"a7f740246ced8d19e0f90d92cda73745f9973e18502953a5ced0f3c8c9a86d4d"} Sep 11 07:59:44.612392 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.612367 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-797fj" event={"ID":"06a3e034-7768-4e68-8ff4-4678f3cd69e7","Type":"ContainerStarted","Data":"c23b6610da124007c486ed785bd7e46b0293de706f236820f6bd7c4234b67385"} Sep 11 07:59:44.613945 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.613926 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vqjrt" event={"ID":"c61a8ab5-9ed9-4f3d-9e32-935de3901e01","Type":"ContainerStarted","Data":"da50055b24085157a28e11a3ae1651a1d5f70c753fda858dd3895a734cbbe0bf"} Sep 11 07:59:44.616254 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.616220 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hgrfs" podStartSLOduration=3.740515624 podStartE2EDuration="21.616207766s" totalImagesPullingTime="17.875692142s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 07:59:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 07:59:44.615740779 +0000 UTC m=+21.621458438" watchObservedRunningTime="2026-09-11 07:59:44.616207766 +0000 UTC m=+21.621925423" Sep 11 07:59:44.654312 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.654270 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ddqh8" podStartSLOduration=3.771815697 podStartE2EDuration="21.654253738s" totalImagesPullingTime="17.882438041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 07:59:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 07:59:44.654210016 +0000 UTC m=+21.659927678" watchObservedRunningTime="2026-09-11 07:59:44.654253738 +0000 UTC m=+21.659971411" Sep 11 07:59:44.654693 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.654664 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-797fj" podStartSLOduration=3.632763407 podStartE2EDuration="21.654657567s" totalImagesPullingTime="18.02189416s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 07:59:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 07:59:44.633623492 +0000 UTC m=+21.639341151" watchObservedRunningTime="2026-09-11 07:59:44.654657567 +0000 UTC m=+21.660375225" Sep 11 07:59:44.678554 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.678511 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6skrk" podStartSLOduration=3.816966512 podStartE2EDuration="21.678494666s" totalImagesPullingTime="17.861528154s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 07:59:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 07:59:44.677825986 +0000 UTC m=+21.683543643" watchObservedRunningTime="2026-09-11 07:59:44.678494666 +0000 UTC m=+21.684212326" Sep 11 07:59:44.715343 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.715308 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vqjrt" podStartSLOduration=3.856250683 podStartE2EDuration="21.715295676s" totalImagesPullingTime="17.859044993s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 07:59:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 07:59:44.692605271 +0000 UTC m=+21.698322929" watchObservedRunningTime="2026-09-11 07:59:44.715295676 +0000 UTC m=+21.721013334" Sep 11 07:59:44.742618 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.742572 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-245.ec2.internal" podStartSLOduration=20.74255573 podStartE2EDuration="20.74255573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 07:59:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 07:59:44.742222897 +0000 UTC m=+21.747940555" watchObservedRunningTime="2026-09-11 07:59:44.74255573 +0000 UTC m=+21.748273406" Sep 11 07:59:44.819441 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:44.819422 2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 11 07:59:45.479033 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:45.478933 2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T07:59:44.819438427Z","UUID":"491ccf8d-656f-4250-96e0-a9bccdb005ca","Handler":null,"Name":"","Endpoint":""} Sep 11 07:59:45.480423 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:45.480404 2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 11 07:59:45.480530 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:45.480431 2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 11 07:59:45.497472 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:45.497446 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:45.497472 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:45.497468 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:45.497638 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:45.497446 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:45.497638 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:45.497548 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f8ptc" podUID="98d2f5ae-5780-4110-935d-e3781cc271f7" Sep 11 07:59:45.497638 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:45.497609 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvr7m" podUID="16110ca4-d538-4c5e-8def-a99a002c9735" Sep 11 07:59:45.497762 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:45.497678 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkcn8" podUID="c5234baf-5b8f-46ff-9fa3-f217c0087ded" Sep 11 07:59:45.618827 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:45.618794 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" event={"ID":"0d722217-e915-4d81-9033-6d5e1579f821","Type":"ContainerStarted","Data":"445b10ed29f4c9fdce7ea7526d4ea1da7d337f73c69df7515c295c3f110cebe2"} Sep 11 07:59:45.620676 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:45.620636 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qfrhd" event={"ID":"8a56f0ac-0efc-4afe-a83b-d44f0c11da3d","Type":"ContainerStarted","Data":"8af6d3d7298b11ff260f67ef4b1f5b7ef9ff6f8a1ab6ad1a976c8d28d796e792"} Sep 11 07:59:45.638136 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:45.638097 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qfrhd" podStartSLOduration=4.770407231 podStartE2EDuration="22.638079478s" totalImagesPullingTime="17.867672247s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 07:59:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 07:59:45.637541432 +0000 UTC m=+22.643259091" watchObservedRunningTime="2026-09-11 07:59:45.638079478 +0000 UTC m=+22.643797138" Sep 11 07:59:46.170635 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:46.170567 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ddqh8" Sep 11 07:59:46.171070 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:46.171054 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ddqh8" Sep 11 07:59:46.625173 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:46.625090 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" event={"ID":"0d722217-e915-4d81-9033-6d5e1579f821","Type":"ContainerStarted","Data":"37ae57921475bd7f132f9e411f2d1f5d7f992eead4733ffdc9624a7d6a008049"} Sep 11 07:59:46.628939 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:46.628893 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" event={"ID":"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08","Type":"ContainerStarted","Data":"e756b7ee82db60700730a6fb61ab8a212e97f1d018e792a79986402dfd8ef0a8"} Sep 11 07:59:47.497328 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:47.497299 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:47.497328 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:47.497313 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:47.497328 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:47.497317 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:47.497672 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:47.497400 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvr7m" podUID="16110ca4-d538-4c5e-8def-a99a002c9735" Sep 11 07:59:47.497672 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:47.497467 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f8ptc" podUID="98d2f5ae-5780-4110-935d-e3781cc271f7" Sep 11 07:59:47.497672 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:47.497539 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkcn8" podUID="c5234baf-5b8f-46ff-9fa3-f217c0087ded" Sep 11 07:59:47.630912 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:47.630876 2557 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 11 07:59:49.497875 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:49.497634 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:49.498442 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:49.497638 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:49.498442 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:49.497991 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f8ptc" podUID="98d2f5ae-5780-4110-935d-e3781cc271f7" Sep 11 07:59:49.498442 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:49.497707 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:49.498442 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:49.498053 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvr7m" podUID="16110ca4-d538-4c5e-8def-a99a002c9735" Sep 11 07:59:49.498442 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:49.498128 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkcn8" podUID="c5234baf-5b8f-46ff-9fa3-f217c0087ded" Sep 11 07:59:50.641062 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:50.640986 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" event={"ID":"0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08","Type":"ContainerStarted","Data":"f4cf1bdae09f1a8ff3044dd00ccfc470b52ce86bfb859f9d8b0101fd56cdc27c"} Sep 11 07:59:50.641755 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:50.641221 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:50.642825 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:50.642804 2557 generic.go:356] "Generic (PLEG): container finished" podID="2cb363cc-5b2b-47df-a9a4-29c33b6a67f5" containerID="4632d66d5bb8011e9308ef74270e14d18d9748b47425ddc182b27fe012b714a6" exitCode=0 Sep 11 07:59:50.642931 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:50.642842 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hfsw" event={"ID":"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5","Type":"ContainerDied","Data":"4632d66d5bb8011e9308ef74270e14d18d9748b47425ddc182b27fe012b714a6"} Sep 11 07:59:50.656185 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:50.656168 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:50.667942 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:50.667887 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vr95q" podStartSLOduration=8.13528859 podStartE2EDuration="27.667875562s" totalImagesPullingTime="19.532586972s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 07:59:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 07:59:46.64664011 +0000 UTC m=+23.652357767" watchObservedRunningTime="2026-09-11 07:59:50.667875562 +0000 UTC m=+27.673593221" Sep 11 07:59:50.668193 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:50.668174 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" podStartSLOduration=8.821552651 podStartE2EDuration="26.668168555s" totalImagesPullingTime="17.846615904s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 07:59:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 07:59:50.667342025 +0000 UTC m=+27.673059686" watchObservedRunningTime="2026-09-11 07:59:50.668168555 +0000 UTC m=+27.673886213" Sep 11 07:59:51.497051 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:51.496879 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:51.497224 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:51.496880 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:51.497224 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:51.497129 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f8ptc" podUID="98d2f5ae-5780-4110-935d-e3781cc271f7" Sep 11 07:59:51.497224 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:51.496886 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:51.497379 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:51.497211 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvr7m" podUID="16110ca4-d538-4c5e-8def-a99a002c9735" Sep 11 07:59:51.497379 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:51.497347 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkcn8" podUID="c5234baf-5b8f-46ff-9fa3-f217c0087ded" Sep 11 07:59:51.645513 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:51.645484 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:51.645848 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:51.645570 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:51.659132 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:51.659106 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 07:59:52.205952 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:52.205864 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f8ptc"] Sep 11 07:59:52.206091 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:52.205988 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:52.206091 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:52.206074 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f8ptc" podUID="98d2f5ae-5780-4110-935d-e3781cc271f7" Sep 11 07:59:52.208429 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:52.208404 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bvr7m"] Sep 11 07:59:52.208554 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:52.208484 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:52.208627 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:52.208562 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvr7m" podUID="16110ca4-d538-4c5e-8def-a99a002c9735" Sep 11 07:59:52.214176 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:52.214157 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mkcn8"] Sep 11 07:59:52.214279 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:52.214230 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:52.214343 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:52.214302 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkcn8" podUID="c5234baf-5b8f-46ff-9fa3-f217c0087ded" Sep 11 07:59:52.649212 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:52.649186 2557 generic.go:356] "Generic (PLEG): container finished" podID="2cb363cc-5b2b-47df-a9a4-29c33b6a67f5" containerID="66e4952e880c157919e7baf195b13f6ec868c0052c6d356f4305f84c52156ce2" exitCode=0 Sep 11 07:59:52.649531 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:52.649270 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hfsw" event={"ID":"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5","Type":"ContainerDied","Data":"66e4952e880c157919e7baf195b13f6ec868c0052c6d356f4305f84c52156ce2"} Sep 11 07:59:53.498175 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:53.498145 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:53.498373 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:53.498177 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:53.498373 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:53.498253 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:53.498373 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:53.498252 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvr7m" podUID="16110ca4-d538-4c5e-8def-a99a002c9735" Sep 11 07:59:53.498373 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:53.498339 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkcn8" podUID="c5234baf-5b8f-46ff-9fa3-f217c0087ded" Sep 11 07:59:53.498575 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:53.498432 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f8ptc" podUID="98d2f5ae-5780-4110-935d-e3781cc271f7" Sep 11 07:59:54.655319 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:54.655291 2557 generic.go:356] "Generic (PLEG): container finished" podID="2cb363cc-5b2b-47df-a9a4-29c33b6a67f5" containerID="3b1283803df56c8f3da058fba01d6371de469b8db99302a3b14632fcbcc3e99b" exitCode=0 Sep 11 07:59:54.655657 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:54.655336 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hfsw" event={"ID":"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5","Type":"ContainerDied","Data":"3b1283803df56c8f3da058fba01d6371de469b8db99302a3b14632fcbcc3e99b"} Sep 11 07:59:54.899354 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:54.899288 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ddqh8" Sep 11 07:59:54.899489 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:54.899400 2557 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 11 07:59:54.899842 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:54.899824 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ddqh8" Sep 11 07:59:55.497948 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:55.497770 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:55.498099 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:55.498010 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvr7m" podUID="16110ca4-d538-4c5e-8def-a99a002c9735" Sep 11 07:59:55.498099 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:55.497788 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:55.498186 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:55.497813 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:55.498186 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:55.498133 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f8ptc" podUID="98d2f5ae-5780-4110-935d-e3781cc271f7" Sep 11 07:59:55.498258 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:55.498187 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkcn8" podUID="c5234baf-5b8f-46ff-9fa3-f217c0087ded" Sep 11 07:59:56.272305 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.272280 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-245.ec2.internal" event="NodeReady" Sep 11 07:59:56.272685 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.272427 2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 11 07:59:56.310575 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.310549 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6777dc96c4-2qdpb"] Sep 11 07:59:56.314687 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.314666 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.316573 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.316546 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 11 07:59:56.316891 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.316869 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 11 07:59:56.316992 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.316962 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-47pmh\"" Sep 11 07:59:56.316992 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.316975 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 11 07:59:56.322608 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.322589 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 11 07:59:56.328996 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.328973 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6777dc96c4-2qdpb"] Sep 11 07:59:56.336124 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.336106 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xl9mm"] Sep 11 07:59:56.339656 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.339640 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qwg5f"] Sep 11 07:59:56.339759 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.339744 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xl9mm" Sep 11 07:59:56.342688 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.342670 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 11 07:59:56.342688 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.342678 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g75rl\"" Sep 11 07:59:56.343146 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.343104 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qwg5f" Sep 11 07:59:56.343601 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.343547 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 11 07:59:56.344614 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.344594 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 11 07:59:56.345022 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.344994 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 11 07:59:56.345140 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.345020 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6jgk6\"" Sep 11 07:59:56.345270 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.345256 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 11 07:59:56.350459 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.350439 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qwg5f"] Sep 11 07:59:56.355032 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.354999 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xl9mm"] Sep 11 07:59:56.445317 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.445285 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78dsr\" (UniqueName: \"kubernetes.io/projected/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-kube-api-access-78dsr\") pod \"dns-default-xl9mm\" (UID: \"022e9b3f-8f4d-40c1-989a-140d1d00f5d0\") " pod="openshift-dns/dns-default-xl9mm" Sep 11 07:59:56.445317 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.445321 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8fd7bec-f0a5-43a7-b8f5-59401860b502-trusted-ca\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.445520 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.445340 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8fd7bec-f0a5-43a7-b8f5-59401860b502-installation-pull-secrets\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.445520 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.445357 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.445520 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.445430 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8fd7bec-f0a5-43a7-b8f5-59401860b502-image-registry-private-configuration\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.445520 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.445480 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fc4eb55-74fd-4d56-bc7f-6812a50165a4-cert\") pod \"ingress-canary-qwg5f\" (UID: \"8fc4eb55-74fd-4d56-bc7f-6812a50165a4\") " pod="openshift-ingress-canary/ingress-canary-qwg5f" Sep 11 07:59:56.445520 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.445516 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-config-volume\") pod \"dns-default-xl9mm\" (UID: \"022e9b3f-8f4d-40c1-989a-140d1d00f5d0\") " pod="openshift-dns/dns-default-xl9mm" Sep 11 07:59:56.445772 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.445543 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-certificates\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.445772 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.445593 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8fd7bec-f0a5-43a7-b8f5-59401860b502-ca-trust-extracted\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.445772 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.445616 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clrfv\" (UniqueName: \"kubernetes.io/projected/8fc4eb55-74fd-4d56-bc7f-6812a50165a4-kube-api-access-clrfv\") pod \"ingress-canary-qwg5f\" (UID: \"8fc4eb55-74fd-4d56-bc7f-6812a50165a4\") " pod="openshift-ingress-canary/ingress-canary-qwg5f" Sep 11 07:59:56.445772 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.445681 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-bound-sa-token\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.445772 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.445706 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls\") pod \"dns-default-xl9mm\" (UID: \"022e9b3f-8f4d-40c1-989a-140d1d00f5d0\") " pod="openshift-dns/dns-default-xl9mm" Sep 11 07:59:56.445772 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.445739 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-tmp-dir\") pod \"dns-default-xl9mm\" (UID: \"022e9b3f-8f4d-40c1-989a-140d1d00f5d0\") " pod="openshift-dns/dns-default-xl9mm" Sep 11 07:59:56.445772 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.445753 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm9qd\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-kube-api-access-lm9qd\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.546422 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.546367 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8fd7bec-f0a5-43a7-b8f5-59401860b502-ca-trust-extracted\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.546422 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.546415 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clrfv\" (UniqueName: \"kubernetes.io/projected/8fc4eb55-74fd-4d56-bc7f-6812a50165a4-kube-api-access-clrfv\") pod \"ingress-canary-qwg5f\" (UID: \"8fc4eb55-74fd-4d56-bc7f-6812a50165a4\") " pod="openshift-ingress-canary/ingress-canary-qwg5f" Sep 11 07:59:56.546584 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.546458 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-bound-sa-token\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.546584 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.546478 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls\") pod \"dns-default-xl9mm\" (UID: \"022e9b3f-8f4d-40c1-989a-140d1d00f5d0\") " pod="openshift-dns/dns-default-xl9mm" Sep 11 07:59:56.546584 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.546520 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-tmp-dir\") pod \"dns-default-xl9mm\" (UID: \"022e9b3f-8f4d-40c1-989a-140d1d00f5d0\") " pod="openshift-dns/dns-default-xl9mm" Sep 11 07:59:56.546584 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.546544 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm9qd\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-kube-api-access-lm9qd\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.546584 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.546575 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78dsr\" (UniqueName: \"kubernetes.io/projected/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-kube-api-access-78dsr\") pod \"dns-default-xl9mm\" (UID: \"022e9b3f-8f4d-40c1-989a-140d1d00f5d0\") " pod="openshift-dns/dns-default-xl9mm" Sep 11 07:59:56.546812 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.546600 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8fd7bec-f0a5-43a7-b8f5-59401860b502-trusted-ca\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.546812 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.546623 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8fd7bec-f0a5-43a7-b8f5-59401860b502-installation-pull-secrets\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.546812 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.546651 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.546812 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.546678 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8fd7bec-f0a5-43a7-b8f5-59401860b502-image-registry-private-configuration\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.546812 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.546700 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fc4eb55-74fd-4d56-bc7f-6812a50165a4-cert\") pod \"ingress-canary-qwg5f\" (UID: \"8fc4eb55-74fd-4d56-bc7f-6812a50165a4\") " pod="openshift-ingress-canary/ingress-canary-qwg5f" Sep 11 07:59:56.546812 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.546722 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-config-volume\") pod \"dns-default-xl9mm\" (UID: \"022e9b3f-8f4d-40c1-989a-140d1d00f5d0\") " pod="openshift-dns/dns-default-xl9mm" Sep 11 07:59:56.546812 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.546744 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-certificates\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.546812 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.546746 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8fd7bec-f0a5-43a7-b8f5-59401860b502-ca-trust-extracted\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.546812 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:56.546805 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 07:59:56.547276 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:56.546866 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls podName:022e9b3f-8f4d-40c1-989a-140d1d00f5d0 nodeName:}" failed. No retries permitted until 2026-09-11 07:59:57.04684564 +0000 UTC m=+34.052563297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls") pod "dns-default-xl9mm" (UID: "022e9b3f-8f4d-40c1-989a-140d1d00f5d0") : secret "dns-default-metrics-tls" not found Sep 11 07:59:56.547276 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.546971 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-tmp-dir\") pod \"dns-default-xl9mm\" (UID: \"022e9b3f-8f4d-40c1-989a-140d1d00f5d0\") " pod="openshift-dns/dns-default-xl9mm" Sep 11 07:59:56.547276 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:56.547067 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 07:59:56.547276 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:56.547080 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6777dc96c4-2qdpb: secret "image-registry-tls" not found Sep 11 07:59:56.547276 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:56.547133 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls podName:f8fd7bec-f0a5-43a7-b8f5-59401860b502 nodeName:}" failed. No retries permitted until 2026-09-11 07:59:57.047119315 +0000 UTC m=+34.052836965 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls") pod "image-registry-6777dc96c4-2qdpb" (UID: "f8fd7bec-f0a5-43a7-b8f5-59401860b502") : secret "image-registry-tls" not found Sep 11 07:59:56.547559 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.547346 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-config-volume\") pod \"dns-default-xl9mm\" (UID: \"022e9b3f-8f4d-40c1-989a-140d1d00f5d0\") " pod="openshift-dns/dns-default-xl9mm" Sep 11 07:59:56.547559 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.547482 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-certificates\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.547665 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.547645 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8fd7bec-f0a5-43a7-b8f5-59401860b502-trusted-ca\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.551333 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.551311 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8fd7bec-f0a5-43a7-b8f5-59401860b502-installation-pull-secrets\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.551453 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.551334 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8fd7bec-f0a5-43a7-b8f5-59401860b502-image-registry-private-configuration\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.551453 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.551401 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fc4eb55-74fd-4d56-bc7f-6812a50165a4-cert\") pod \"ingress-canary-qwg5f\" (UID: \"8fc4eb55-74fd-4d56-bc7f-6812a50165a4\") " pod="openshift-ingress-canary/ingress-canary-qwg5f" Sep 11 07:59:56.565573 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.565552 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78dsr\" (UniqueName: \"kubernetes.io/projected/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-kube-api-access-78dsr\") pod \"dns-default-xl9mm\" (UID: \"022e9b3f-8f4d-40c1-989a-140d1d00f5d0\") " pod="openshift-dns/dns-default-xl9mm" Sep 11 07:59:56.565730 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.565707 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-bound-sa-token\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.566107 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.566065 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm9qd\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-kube-api-access-lm9qd\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:56.566409 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.566389 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clrfv\" (UniqueName: \"kubernetes.io/projected/8fc4eb55-74fd-4d56-bc7f-6812a50165a4-kube-api-access-clrfv\") pod \"ingress-canary-qwg5f\" (UID: \"8fc4eb55-74fd-4d56-bc7f-6812a50165a4\") " pod="openshift-ingress-canary/ingress-canary-qwg5f" Sep 11 07:59:56.653597 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.653575 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qwg5f" Sep 11 07:59:56.936673 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:56.936637 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qwg5f"] Sep 11 07:59:56.940010 ip-10-0-136-245 kubenswrapper[2557]: W0911 07:59:56.939983 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fc4eb55_74fd_4d56_bc7f_6812a50165a4.slice/crio-a482f80cc98527e6a398d4d768550a30f69e9b89bf43e2f643261bd8f4beba83 WatchSource:0}: Error finding container a482f80cc98527e6a398d4d768550a30f69e9b89bf43e2f643261bd8f4beba83: Status 404 returned error can't find the container with id a482f80cc98527e6a398d4d768550a30f69e9b89bf43e2f643261bd8f4beba83 Sep 11 07:59:57.049930 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:57.049891 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls\") pod \"dns-default-xl9mm\" (UID: \"022e9b3f-8f4d-40c1-989a-140d1d00f5d0\") " pod="openshift-dns/dns-default-xl9mm" Sep 11 07:59:57.050019 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:57.049953 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:57.050070 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:57.050033 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 07:59:57.050070 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:57.050034 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 07:59:57.050140 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:57.050090 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls podName:022e9b3f-8f4d-40c1-989a-140d1d00f5d0 nodeName:}" failed. No retries permitted until 2026-09-11 07:59:58.05007521 +0000 UTC m=+35.055792850 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls") pod "dns-default-xl9mm" (UID: "022e9b3f-8f4d-40c1-989a-140d1d00f5d0") : secret "dns-default-metrics-tls" not found Sep 11 07:59:57.050140 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:57.050042 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6777dc96c4-2qdpb: secret "image-registry-tls" not found Sep 11 07:59:57.050213 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:57.050166 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls podName:f8fd7bec-f0a5-43a7-b8f5-59401860b502 nodeName:}" failed. No retries permitted until 2026-09-11 07:59:58.05015308 +0000 UTC m=+35.055870716 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls") pod "image-registry-6777dc96c4-2qdpb" (UID: "f8fd7bec-f0a5-43a7-b8f5-59401860b502") : secret "image-registry-tls" not found Sep 11 07:59:57.250783 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:57.250720 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7h8v\" (UniqueName: \"kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v\") pod \"network-check-target-bvr7m\" (UID: \"16110ca4-d538-4c5e-8def-a99a002c9735\") " pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:57.250783 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:57.250755 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret\") pod \"global-pull-secret-syncer-f8ptc\" (UID: \"98d2f5ae-5780-4110-935d-e3781cc271f7\") " pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:57.251026 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:57.250797 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs\") pod \"network-metrics-daemon-mkcn8\" (UID: \"c5234baf-5b8f-46ff-9fa3-f217c0087ded\") " pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:57.251026 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:57.250887 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 07:59:57.251026 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:57.250910 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 07:59:57.251026 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:57.250894 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 07:59:57.251026 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:57.250953 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 07:59:57.251026 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:57.250959 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret podName:98d2f5ae-5780-4110-935d-e3781cc271f7 nodeName:}" failed. No retries permitted until 2026-09-11 08:00:29.250941379 +0000 UTC m=+66.256659030 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret") pod "global-pull-secret-syncer-f8ptc" (UID: "98d2f5ae-5780-4110-935d-e3781cc271f7") : object "kube-system"/"original-pull-secret" not registered Sep 11 07:59:57.251026 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:57.250968 2557 projected.go:196] Error preparing data for projected volume kube-api-access-r7h8v for pod openshift-network-diagnostics/network-check-target-bvr7m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 07:59:57.251026 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:57.250980 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs podName:c5234baf-5b8f-46ff-9fa3-f217c0087ded nodeName:}" failed. No retries permitted until 2026-09-11 08:00:29.250968227 +0000 UTC m=+66.256685878 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs") pod "network-metrics-daemon-mkcn8" (UID: "c5234baf-5b8f-46ff-9fa3-f217c0087ded") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 07:59:57.251026 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:57.251000 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v podName:16110ca4-d538-4c5e-8def-a99a002c9735 nodeName:}" failed. No retries permitted until 2026-09-11 08:00:29.250990266 +0000 UTC m=+66.256707903 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7h8v" (UniqueName: "kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v") pod "network-check-target-bvr7m" (UID: "16110ca4-d538-4c5e-8def-a99a002c9735") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 07:59:57.497703 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:57.497516 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 07:59:57.497703 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:57.497556 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 07:59:57.497703 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:57.497516 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 07:59:57.500311 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:57.499926 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s4cpg\"" Sep 11 07:59:57.500311 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:57.499942 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 11 07:59:57.500560 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:57.500544 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 11 07:59:57.501217 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:57.500842 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 11 07:59:57.501217 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:57.500984 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mdzhs\"" Sep 11 07:59:57.501217 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:57.501093 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 11 07:59:57.664332 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:57.664299 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qwg5f" event={"ID":"8fc4eb55-74fd-4d56-bc7f-6812a50165a4","Type":"ContainerStarted","Data":"a482f80cc98527e6a398d4d768550a30f69e9b89bf43e2f643261bd8f4beba83"} Sep 11 07:59:58.057540 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:58.057508 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 07:59:58.057698 ip-10-0-136-245 kubenswrapper[2557]: I0911 07:59:58.057591 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls\") pod \"dns-default-xl9mm\" (UID: \"022e9b3f-8f4d-40c1-989a-140d1d00f5d0\") " pod="openshift-dns/dns-default-xl9mm" Sep 11 07:59:58.057698 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:58.057676 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 07:59:58.057817 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:58.057760 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls podName:022e9b3f-8f4d-40c1-989a-140d1d00f5d0 nodeName:}" failed. No retries permitted until 2026-09-11 08:00:00.05773898 +0000 UTC m=+37.063456641 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls") pod "dns-default-xl9mm" (UID: "022e9b3f-8f4d-40c1-989a-140d1d00f5d0") : secret "dns-default-metrics-tls" not found Sep 11 07:59:58.057817 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:58.057683 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 07:59:58.057817 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:58.057784 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6777dc96c4-2qdpb: secret "image-registry-tls" not found Sep 11 07:59:58.057976 ip-10-0-136-245 kubenswrapper[2557]: E0911 07:59:58.057875 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls podName:f8fd7bec-f0a5-43a7-b8f5-59401860b502 nodeName:}" failed. No retries permitted until 2026-09-11 08:00:00.057857737 +0000 UTC m=+37.063575380 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls") pod "image-registry-6777dc96c4-2qdpb" (UID: "f8fd7bec-f0a5-43a7-b8f5-59401860b502") : secret "image-registry-tls" not found Sep 11 08:00:00.073200 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:00.073002 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 08:00:00.073640 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:00.073247 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls\") pod \"dns-default-xl9mm\" (UID: \"022e9b3f-8f4d-40c1-989a-140d1d00f5d0\") " pod="openshift-dns/dns-default-xl9mm" Sep 11 08:00:00.073640 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:00.073165 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 08:00:00.073640 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:00.073300 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6777dc96c4-2qdpb: secret "image-registry-tls" not found Sep 11 08:00:00.073640 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:00.073365 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 08:00:00.073640 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:00.073392 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls podName:f8fd7bec-f0a5-43a7-b8f5-59401860b502 nodeName:}" failed. No retries permitted until 2026-09-11 08:00:04.073356068 +0000 UTC m=+41.079073709 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls") pod "image-registry-6777dc96c4-2qdpb" (UID: "f8fd7bec-f0a5-43a7-b8f5-59401860b502") : secret "image-registry-tls" not found Sep 11 08:00:00.073640 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:00.073411 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls podName:022e9b3f-8f4d-40c1-989a-140d1d00f5d0 nodeName:}" failed. No retries permitted until 2026-09-11 08:00:04.073401449 +0000 UTC m=+41.079119091 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls") pod "dns-default-xl9mm" (UID: "022e9b3f-8f4d-40c1-989a-140d1d00f5d0") : secret "dns-default-metrics-tls" not found Sep 11 08:00:01.676617 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:01.676586 2557 generic.go:356] "Generic (PLEG): container finished" podID="2cb363cc-5b2b-47df-a9a4-29c33b6a67f5" containerID="4f3c04b4a9d0f19eae35e7de64dc973cb1313b8ee133ecd66459b25570f4ef57" exitCode=0 Sep 11 08:00:01.677016 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:01.676664 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hfsw" event={"ID":"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5","Type":"ContainerDied","Data":"4f3c04b4a9d0f19eae35e7de64dc973cb1313b8ee133ecd66459b25570f4ef57"} Sep 11 08:00:01.678490 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:01.678468 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qwg5f" event={"ID":"8fc4eb55-74fd-4d56-bc7f-6812a50165a4","Type":"ContainerStarted","Data":"ed1dd1f5b0ef61d961e54e1752ddd349a04d5f624f23d705eda0e5bac789b657"} Sep 11 08:00:01.718206 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:01.717933 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qwg5f" podStartSLOduration=2.052058887 podStartE2EDuration="5.717882999s" totalImagesPullingTime="3.665824112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 07:59:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:00:01.716995604 +0000 UTC m=+38.722713263" watchObservedRunningTime="2026-09-11 08:00:01.717882999 +0000 UTC m=+38.723600670" Sep 11 08:00:02.683874 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:02.683841 2557 generic.go:356] "Generic (PLEG): container finished" podID="2cb363cc-5b2b-47df-a9a4-29c33b6a67f5" containerID="3e8ca84bfa2cb30c0bc92ed615de3ce1b63bdcbea9c2f95480268976f5514268" exitCode=0 Sep 11 08:00:02.684236 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:02.683930 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hfsw" event={"ID":"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5","Type":"ContainerDied","Data":"3e8ca84bfa2cb30c0bc92ed615de3ce1b63bdcbea9c2f95480268976f5514268"} Sep 11 08:00:03.689456 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:03.689420 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hfsw" event={"ID":"2cb363cc-5b2b-47df-a9a4-29c33b6a67f5","Type":"ContainerStarted","Data":"d0377c5baeb5589a0b6f9242ab1ef5bc4aeb440e77299a79e01b5876e065fdd6"} Sep 11 08:00:03.744302 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:03.744245 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8hfsw" podStartSLOduration=9.170239204 podStartE2EDuration="40.744231677s" totalImagesPullingTime="31.573992473s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 07:59:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:00:03.74397537 +0000 UTC m=+40.749693028" watchObservedRunningTime="2026-09-11 08:00:03.744231677 +0000 UTC m=+40.749949336" Sep 11 08:00:04.102017 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:04.101943 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls\") pod \"dns-default-xl9mm\" (UID: \"022e9b3f-8f4d-40c1-989a-140d1d00f5d0\") " pod="openshift-dns/dns-default-xl9mm" Sep 11 08:00:04.102017 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:04.102003 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 08:00:04.102184 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:04.102098 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 08:00:04.102184 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:04.102109 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6777dc96c4-2qdpb: secret "image-registry-tls" not found Sep 11 08:00:04.102184 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:04.102164 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls podName:f8fd7bec-f0a5-43a7-b8f5-59401860b502 nodeName:}" failed. No retries permitted until 2026-09-11 08:00:12.10215076 +0000 UTC m=+49.107868410 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls") pod "image-registry-6777dc96c4-2qdpb" (UID: "f8fd7bec-f0a5-43a7-b8f5-59401860b502") : secret "image-registry-tls" not found Sep 11 08:00:04.102184 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:04.102098 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 08:00:04.102322 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:04.102226 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls podName:022e9b3f-8f4d-40c1-989a-140d1d00f5d0 nodeName:}" failed. No retries permitted until 2026-09-11 08:00:12.102210345 +0000 UTC m=+49.107927982 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls") pod "dns-default-xl9mm" (UID: "022e9b3f-8f4d-40c1-989a-140d1d00f5d0") : secret "dns-default-metrics-tls" not found Sep 11 08:00:12.152646 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:12.152607 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls\") pod \"dns-default-xl9mm\" (UID: \"022e9b3f-8f4d-40c1-989a-140d1d00f5d0\") " pod="openshift-dns/dns-default-xl9mm" Sep 11 08:00:12.153198 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:12.152673 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 08:00:12.153198 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:12.152832 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 08:00:12.153198 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:12.152830 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 08:00:12.153198 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:12.152951 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6777dc96c4-2qdpb: secret "image-registry-tls" not found Sep 11 08:00:12.153198 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:12.152925 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls podName:022e9b3f-8f4d-40c1-989a-140d1d00f5d0 nodeName:}" failed. No retries permitted until 2026-09-11 08:00:28.152882069 +0000 UTC m=+65.158599709 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls") pod "dns-default-xl9mm" (UID: "022e9b3f-8f4d-40c1-989a-140d1d00f5d0") : secret "dns-default-metrics-tls" not found Sep 11 08:00:12.153198 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:12.153062 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls podName:f8fd7bec-f0a5-43a7-b8f5-59401860b502 nodeName:}" failed. No retries permitted until 2026-09-11 08:00:28.15303393 +0000 UTC m=+65.158751569 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls") pod "image-registry-6777dc96c4-2qdpb" (UID: "f8fd7bec-f0a5-43a7-b8f5-59401860b502") : secret "image-registry-tls" not found Sep 11 08:00:23.660401 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:23.660375 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9j5fx" Sep 11 08:00:28.157344 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:28.157303 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls\") pod \"dns-default-xl9mm\" (UID: \"022e9b3f-8f4d-40c1-989a-140d1d00f5d0\") " pod="openshift-dns/dns-default-xl9mm" Sep 11 08:00:28.157853 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:28.157363 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 08:00:28.157853 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:28.157450 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 08:00:28.157853 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:28.157472 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 08:00:28.157853 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:28.157490 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6777dc96c4-2qdpb: secret "image-registry-tls" not found Sep 11 08:00:28.157853 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:28.157517 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls podName:022e9b3f-8f4d-40c1-989a-140d1d00f5d0 nodeName:}" failed. No retries permitted until 2026-09-11 08:01:00.157501619 +0000 UTC m=+97.163219255 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls") pod "dns-default-xl9mm" (UID: "022e9b3f-8f4d-40c1-989a-140d1d00f5d0") : secret "dns-default-metrics-tls" not found Sep 11 08:00:28.157853 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:28.157544 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls podName:f8fd7bec-f0a5-43a7-b8f5-59401860b502 nodeName:}" failed. No retries permitted until 2026-09-11 08:01:00.15752867 +0000 UTC m=+97.163246383 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls") pod "image-registry-6777dc96c4-2qdpb" (UID: "f8fd7bec-f0a5-43a7-b8f5-59401860b502") : secret "image-registry-tls" not found Sep 11 08:00:29.265486 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:29.265448 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs\") pod \"network-metrics-daemon-mkcn8\" (UID: \"c5234baf-5b8f-46ff-9fa3-f217c0087ded\") " pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 08:00:29.266000 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:29.265509 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7h8v\" (UniqueName: \"kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v\") pod \"network-check-target-bvr7m\" (UID: \"16110ca4-d538-4c5e-8def-a99a002c9735\") " pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 08:00:29.266000 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:29.265532 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret\") pod \"global-pull-secret-syncer-f8ptc\" (UID: \"98d2f5ae-5780-4110-935d-e3781cc271f7\") " pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 08:00:29.267886 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:29.267864 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 11 08:00:29.268025 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:29.267936 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 11 08:00:29.268139 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:29.268122 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 11 08:00:29.276757 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:29.276731 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 11 08:00:29.276843 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:29.276829 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs podName:c5234baf-5b8f-46ff-9fa3-f217c0087ded nodeName:}" failed. No retries permitted until 2026-09-11 08:01:33.276805879 +0000 UTC m=+130.282523534 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs") pod "network-metrics-daemon-mkcn8" (UID: "c5234baf-5b8f-46ff-9fa3-f217c0087ded") : secret "metrics-daemon-secret" not found Sep 11 08:00:29.278552 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:29.278535 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 11 08:00:29.280485 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:29.280461 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/98d2f5ae-5780-4110-935d-e3781cc271f7-original-pull-secret\") pod \"global-pull-secret-syncer-f8ptc\" (UID: \"98d2f5ae-5780-4110-935d-e3781cc271f7\") " pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 08:00:29.289202 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:29.289179 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7h8v\" (UniqueName: \"kubernetes.io/projected/16110ca4-d538-4c5e-8def-a99a002c9735-kube-api-access-r7h8v\") pod \"network-check-target-bvr7m\" (UID: \"16110ca4-d538-4c5e-8def-a99a002c9735\") " pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 08:00:29.320174 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:29.320152 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f8ptc" Sep 11 08:00:29.327711 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:29.327688 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s4cpg\"" Sep 11 08:00:29.336511 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:29.336492 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 08:00:29.610957 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:29.610885 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f8ptc"] Sep 11 08:00:29.614039 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:00:29.614015 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98d2f5ae_5780_4110_935d_e3781cc271f7.slice/crio-e687286ad1da4cc8e3d3633c7ab7ebb7bd069795f606609316ea249a490ea075 WatchSource:0}: Error finding container e687286ad1da4cc8e3d3633c7ab7ebb7bd069795f606609316ea249a490ea075: Status 404 returned error can't find the container with id e687286ad1da4cc8e3d3633c7ab7ebb7bd069795f606609316ea249a490ea075 Sep 11 08:00:29.632871 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:29.632855 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bvr7m"] Sep 11 08:00:29.634706 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:00:29.634684 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16110ca4_d538_4c5e_8def_a99a002c9735.slice/crio-2ea0b22a3bca347862d73834dc49190d43cabe59160b9d2a51a903d5bdff85a4 WatchSource:0}: Error finding container 2ea0b22a3bca347862d73834dc49190d43cabe59160b9d2a51a903d5bdff85a4: Status 404 returned error can't find the container with id 2ea0b22a3bca347862d73834dc49190d43cabe59160b9d2a51a903d5bdff85a4 Sep 11 08:00:29.753430 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:29.753402 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f8ptc" event={"ID":"98d2f5ae-5780-4110-935d-e3781cc271f7","Type":"ContainerStarted","Data":"e687286ad1da4cc8e3d3633c7ab7ebb7bd069795f606609316ea249a490ea075"} Sep 11 08:00:29.754489 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:29.754467 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bvr7m" event={"ID":"16110ca4-d538-4c5e-8def-a99a002c9735","Type":"ContainerStarted","Data":"2ea0b22a3bca347862d73834dc49190d43cabe59160b9d2a51a903d5bdff85a4"} Sep 11 08:00:35.151323 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.151293 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5f8bf9f44-82dnf"] Sep 11 08:00:35.155060 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.155044 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:35.159213 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.159187 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 11 08:00:35.159685 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.159506 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-7lvkz\"" Sep 11 08:00:35.159685 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.159536 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 11 08:00:35.159685 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.159549 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 11 08:00:35.159685 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.159536 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 11 08:00:35.159964 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.159770 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 11 08:00:35.160087 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.160072 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 11 08:00:35.171596 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.171566 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5f8bf9f44-82dnf"] Sep 11 08:00:35.206766 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.206748 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47a69b8b-6b79-426e-bf18-963fe1046241-service-ca-bundle\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:35.206851 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.206784 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-stats-auth\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:35.206851 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.206802 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-default-certificate\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:35.206851 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.206827 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnbhp\" (UniqueName: \"kubernetes.io/projected/47a69b8b-6b79-426e-bf18-963fe1046241-kube-api-access-xnbhp\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:35.206970 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.206874 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-metrics-certs\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:35.307072 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.307049 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-default-certificate\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:35.307156 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.307076 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnbhp\" (UniqueName: \"kubernetes.io/projected/47a69b8b-6b79-426e-bf18-963fe1046241-kube-api-access-xnbhp\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:35.307156 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.307098 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-metrics-certs\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:35.307156 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.307147 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47a69b8b-6b79-426e-bf18-963fe1046241-service-ca-bundle\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:35.307303 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.307176 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-stats-auth\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:35.307355 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:35.307324 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 11 08:00:35.307355 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:35.307339 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/47a69b8b-6b79-426e-bf18-963fe1046241-service-ca-bundle podName:47a69b8b-6b79-426e-bf18-963fe1046241 nodeName:}" failed. No retries permitted until 2026-09-11 08:00:35.807320081 +0000 UTC m=+72.813037736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/47a69b8b-6b79-426e-bf18-963fe1046241-service-ca-bundle") pod "router-default-5f8bf9f44-82dnf" (UID: "47a69b8b-6b79-426e-bf18-963fe1046241") : configmap references non-existent config key: service-ca.crt Sep 11 08:00:35.307476 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:35.307383 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-metrics-certs podName:47a69b8b-6b79-426e-bf18-963fe1046241 nodeName:}" failed. No retries permitted until 2026-09-11 08:00:35.807365816 +0000 UTC m=+72.813083461 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-metrics-certs") pod "router-default-5f8bf9f44-82dnf" (UID: "47a69b8b-6b79-426e-bf18-963fe1046241") : secret "router-metrics-certs-default" not found Sep 11 08:00:35.309340 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.309322 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-default-certificate\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:35.309426 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.309410 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-stats-auth\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:35.324915 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.324867 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnbhp\" (UniqueName: \"kubernetes.io/projected/47a69b8b-6b79-426e-bf18-963fe1046241-kube-api-access-xnbhp\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:35.771443 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.771400 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bvr7m" event={"ID":"16110ca4-d538-4c5e-8def-a99a002c9735","Type":"ContainerStarted","Data":"a5a79b272700677fd1a97162b246eada54d8b1d4c303c342aab5e32d38b8b0ea"} Sep 11 08:00:35.771628 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.771609 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 08:00:35.772823 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.772802 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f8ptc" event={"ID":"98d2f5ae-5780-4110-935d-e3781cc271f7","Type":"ContainerStarted","Data":"cdb570babc4bbc64bb57c17711dc112f70e019cde7e789d4e57c431afeae24ff"} Sep 11 08:00:35.790998 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.790956 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bvr7m" podStartSLOduration=67.721553056 podStartE2EDuration="1m12.790943462s" totalImagesPullingTime="5.069390406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 07:59:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:00:35.790545823 +0000 UTC m=+72.796263483" watchObservedRunningTime="2026-09-11 08:00:35.790943462 +0000 UTC m=+72.796661134" Sep 11 08:00:35.806998 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.806963 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-f8ptc" podStartSLOduration=66.705969941 podStartE2EDuration="1m11.806951148s" totalImagesPullingTime="5.100981207s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 07:59:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:00:35.806541404 +0000 UTC m=+72.812259074" watchObservedRunningTime="2026-09-11 08:00:35.806951148 +0000 UTC m=+72.812668806" Sep 11 08:00:35.810905 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.810875 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-metrics-certs\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:35.811009 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:35.810992 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47a69b8b-6b79-426e-bf18-963fe1046241-service-ca-bundle\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:35.811066 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:35.811015 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 11 08:00:35.811111 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:35.811096 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-metrics-certs podName:47a69b8b-6b79-426e-bf18-963fe1046241 nodeName:}" failed. No retries permitted until 2026-09-11 08:00:36.811079957 +0000 UTC m=+73.816797610 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-metrics-certs") pod "router-default-5f8bf9f44-82dnf" (UID: "47a69b8b-6b79-426e-bf18-963fe1046241") : secret "router-metrics-certs-default" not found Sep 11 08:00:35.811161 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:35.811122 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/47a69b8b-6b79-426e-bf18-963fe1046241-service-ca-bundle podName:47a69b8b-6b79-426e-bf18-963fe1046241 nodeName:}" failed. No retries permitted until 2026-09-11 08:00:36.811112129 +0000 UTC m=+73.816829779 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/47a69b8b-6b79-426e-bf18-963fe1046241-service-ca-bundle") pod "router-default-5f8bf9f44-82dnf" (UID: "47a69b8b-6b79-426e-bf18-963fe1046241") : configmap references non-existent config key: service-ca.crt Sep 11 08:00:36.817549 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:36.817521 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47a69b8b-6b79-426e-bf18-963fe1046241-service-ca-bundle\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:36.817967 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:36.817573 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-metrics-certs\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:36.817967 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:36.817683 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 11 08:00:36.817967 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:36.817724 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/47a69b8b-6b79-426e-bf18-963fe1046241-service-ca-bundle podName:47a69b8b-6b79-426e-bf18-963fe1046241 nodeName:}" failed. No retries permitted until 2026-09-11 08:00:38.817702524 +0000 UTC m=+75.823420180 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/47a69b8b-6b79-426e-bf18-963fe1046241-service-ca-bundle") pod "router-default-5f8bf9f44-82dnf" (UID: "47a69b8b-6b79-426e-bf18-963fe1046241") : configmap references non-existent config key: service-ca.crt Sep 11 08:00:36.817967 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:36.817757 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-metrics-certs podName:47a69b8b-6b79-426e-bf18-963fe1046241 nodeName:}" failed. No retries permitted until 2026-09-11 08:00:38.817746467 +0000 UTC m=+75.823464103 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-metrics-certs") pod "router-default-5f8bf9f44-82dnf" (UID: "47a69b8b-6b79-426e-bf18-963fe1046241") : secret "router-metrics-certs-default" not found Sep 11 08:00:38.830561 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:38.830528 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-metrics-certs\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:38.830960 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:38.830588 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47a69b8b-6b79-426e-bf18-963fe1046241-service-ca-bundle\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:38.830960 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:38.830667 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 11 08:00:38.830960 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:38.830718 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/47a69b8b-6b79-426e-bf18-963fe1046241-service-ca-bundle podName:47a69b8b-6b79-426e-bf18-963fe1046241 nodeName:}" failed. No retries permitted until 2026-09-11 08:00:42.830700696 +0000 UTC m=+79.836418351 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/47a69b8b-6b79-426e-bf18-963fe1046241-service-ca-bundle") pod "router-default-5f8bf9f44-82dnf" (UID: "47a69b8b-6b79-426e-bf18-963fe1046241") : configmap references non-existent config key: service-ca.crt Sep 11 08:00:38.830960 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:38.830735 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-metrics-certs podName:47a69b8b-6b79-426e-bf18-963fe1046241 nodeName:}" failed. No retries permitted until 2026-09-11 08:00:42.830727577 +0000 UTC m=+79.836445213 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-metrics-certs") pod "router-default-5f8bf9f44-82dnf" (UID: "47a69b8b-6b79-426e-bf18-963fe1046241") : secret "router-metrics-certs-default" not found Sep 11 08:00:41.043098 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.043066 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fssdv"] Sep 11 08:00:41.046296 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.046262 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:41.049703 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.049674 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 11 08:00:41.049842 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.049760 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 11 08:00:41.049842 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.049777 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 11 08:00:41.049842 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.049764 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 11 08:00:41.049842 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.049826 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4cbcr\"" Sep 11 08:00:41.067105 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.067078 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fssdv"] Sep 11 08:00:41.147170 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.147148 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0aad543f-d277-4552-8909-de30a3820f5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:41.147277 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.147178 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnvhq\" (UniqueName: \"kubernetes.io/projected/0aad543f-d277-4552-8909-de30a3820f5d-kube-api-access-fnvhq\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:41.147277 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.147242 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0aad543f-d277-4552-8909-de30a3820f5d-crio-socket\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:41.147353 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.147319 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0aad543f-d277-4552-8909-de30a3820f5d-data-volume\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:41.147353 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.147343 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0aad543f-d277-4552-8909-de30a3820f5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:41.248404 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.248379 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0aad543f-d277-4552-8909-de30a3820f5d-data-volume\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:41.248495 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.248411 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0aad543f-d277-4552-8909-de30a3820f5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:41.248495 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.248450 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0aad543f-d277-4552-8909-de30a3820f5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:41.248495 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.248474 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnvhq\" (UniqueName: \"kubernetes.io/projected/0aad543f-d277-4552-8909-de30a3820f5d-kube-api-access-fnvhq\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:41.248640 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:41.248502 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 11 08:00:41.248640 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.248516 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0aad543f-d277-4552-8909-de30a3820f5d-crio-socket\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:41.248640 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:41.248567 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0aad543f-d277-4552-8909-de30a3820f5d-kube-rbac-proxy-cm podName:0aad543f-d277-4552-8909-de30a3820f5d nodeName:}" failed. No retries permitted until 2026-09-11 08:00:41.748548012 +0000 UTC m=+78.754265665 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0aad543f-d277-4552-8909-de30a3820f5d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fssdv" (UID: "0aad543f-d277-4552-8909-de30a3820f5d") : configmap "kube-rbac-proxy" not found Sep 11 08:00:41.248640 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.248614 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0aad543f-d277-4552-8909-de30a3820f5d-crio-socket\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:41.248843 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:41.248639 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 08:00:41.248843 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:41.248705 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0aad543f-d277-4552-8909-de30a3820f5d-insights-runtime-extractor-tls podName:0aad543f-d277-4552-8909-de30a3820f5d nodeName:}" failed. No retries permitted until 2026-09-11 08:00:41.748686541 +0000 UTC m=+78.754404192 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0aad543f-d277-4552-8909-de30a3820f5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fssdv" (UID: "0aad543f-d277-4552-8909-de30a3820f5d") : secret "insights-runtime-extractor-tls" not found Sep 11 08:00:41.248843 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.248743 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0aad543f-d277-4552-8909-de30a3820f5d-data-volume\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:41.257890 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.257863 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnvhq\" (UniqueName: \"kubernetes.io/projected/0aad543f-d277-4552-8909-de30a3820f5d-kube-api-access-fnvhq\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:41.753714 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.753675 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0aad543f-d277-4552-8909-de30a3820f5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:41.753866 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:41.753736 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0aad543f-d277-4552-8909-de30a3820f5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:41.753866 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:41.753796 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 11 08:00:41.753866 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:41.753837 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 08:00:41.753985 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:41.753880 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0aad543f-d277-4552-8909-de30a3820f5d-kube-rbac-proxy-cm podName:0aad543f-d277-4552-8909-de30a3820f5d nodeName:}" failed. No retries permitted until 2026-09-11 08:00:42.753862706 +0000 UTC m=+79.759580348 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0aad543f-d277-4552-8909-de30a3820f5d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fssdv" (UID: "0aad543f-d277-4552-8909-de30a3820f5d") : configmap "kube-rbac-proxy" not found Sep 11 08:00:41.753985 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:41.753913 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0aad543f-d277-4552-8909-de30a3820f5d-insights-runtime-extractor-tls podName:0aad543f-d277-4552-8909-de30a3820f5d nodeName:}" failed. No retries permitted until 2026-09-11 08:00:42.753890256 +0000 UTC m=+79.759607891 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0aad543f-d277-4552-8909-de30a3820f5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fssdv" (UID: "0aad543f-d277-4552-8909-de30a3820f5d") : secret "insights-runtime-extractor-tls" not found Sep 11 08:00:42.760837 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:42.760808 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0aad543f-d277-4552-8909-de30a3820f5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:42.761211 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:42.760850 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0aad543f-d277-4552-8909-de30a3820f5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:42.761211 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:42.760944 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 11 08:00:42.761211 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:42.760963 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 08:00:42.761211 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:42.761009 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0aad543f-d277-4552-8909-de30a3820f5d-insights-runtime-extractor-tls podName:0aad543f-d277-4552-8909-de30a3820f5d nodeName:}" failed. No retries permitted until 2026-09-11 08:00:44.760997241 +0000 UTC m=+81.766714891 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0aad543f-d277-4552-8909-de30a3820f5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fssdv" (UID: "0aad543f-d277-4552-8909-de30a3820f5d") : secret "insights-runtime-extractor-tls" not found Sep 11 08:00:42.761211 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:42.761032 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0aad543f-d277-4552-8909-de30a3820f5d-kube-rbac-proxy-cm podName:0aad543f-d277-4552-8909-de30a3820f5d nodeName:}" failed. No retries permitted until 2026-09-11 08:00:44.761015379 +0000 UTC m=+81.766733015 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0aad543f-d277-4552-8909-de30a3820f5d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fssdv" (UID: "0aad543f-d277-4552-8909-de30a3820f5d") : configmap "kube-rbac-proxy" not found Sep 11 08:00:42.862036 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:42.862013 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-metrics-certs\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:42.862159 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:42.862063 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47a69b8b-6b79-426e-bf18-963fe1046241-service-ca-bundle\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:42.862159 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:42.862137 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 11 08:00:42.862159 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:42.862145 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/47a69b8b-6b79-426e-bf18-963fe1046241-service-ca-bundle podName:47a69b8b-6b79-426e-bf18-963fe1046241 nodeName:}" failed. No retries permitted until 2026-09-11 08:00:50.862134695 +0000 UTC m=+87.867852344 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/47a69b8b-6b79-426e-bf18-963fe1046241-service-ca-bundle") pod "router-default-5f8bf9f44-82dnf" (UID: "47a69b8b-6b79-426e-bf18-963fe1046241") : configmap references non-existent config key: service-ca.crt Sep 11 08:00:42.862267 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:42.862175 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-metrics-certs podName:47a69b8b-6b79-426e-bf18-963fe1046241 nodeName:}" failed. No retries permitted until 2026-09-11 08:00:50.862163909 +0000 UTC m=+87.867881545 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-metrics-certs") pod "router-default-5f8bf9f44-82dnf" (UID: "47a69b8b-6b79-426e-bf18-963fe1046241") : secret "router-metrics-certs-default" not found Sep 11 08:00:43.173316 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:43.173254 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vqjrt_c61a8ab5-9ed9-4f3d-9e32-935de3901e01/dns-node-resolver/0.log" Sep 11 08:00:43.773541 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:43.773519 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6skrk_88bb78bf-db17-415d-b3e8-b88236c84350/node-ca/0.log" Sep 11 08:00:44.773710 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:44.773672 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qwg5f_8fc4eb55-74fd-4d56-bc7f-6812a50165a4/serve-healthcheck-canary/0.log" Sep 11 08:00:44.775803 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:44.775781 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0aad543f-d277-4552-8909-de30a3820f5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:44.775858 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:44.775828 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0aad543f-d277-4552-8909-de30a3820f5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:44.775914 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:44.775889 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 11 08:00:44.775972 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:44.775962 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0aad543f-d277-4552-8909-de30a3820f5d-kube-rbac-proxy-cm podName:0aad543f-d277-4552-8909-de30a3820f5d nodeName:}" failed. No retries permitted until 2026-09-11 08:00:48.775944643 +0000 UTC m=+85.781662290 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0aad543f-d277-4552-8909-de30a3820f5d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fssdv" (UID: "0aad543f-d277-4552-8909-de30a3820f5d") : configmap "kube-rbac-proxy" not found Sep 11 08:00:44.776016 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:44.775991 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 08:00:44.776048 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:44.776036 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0aad543f-d277-4552-8909-de30a3820f5d-insights-runtime-extractor-tls podName:0aad543f-d277-4552-8909-de30a3820f5d nodeName:}" failed. No retries permitted until 2026-09-11 08:00:48.776021622 +0000 UTC m=+85.781739273 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0aad543f-d277-4552-8909-de30a3820f5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fssdv" (UID: "0aad543f-d277-4552-8909-de30a3820f5d") : secret "insights-runtime-extractor-tls" not found Sep 11 08:00:48.801558 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:48.801413 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0aad543f-d277-4552-8909-de30a3820f5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:48.801558 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:48.801470 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0aad543f-d277-4552-8909-de30a3820f5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:48.802081 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:48.801561 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 11 08:00:48.802081 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:48.801601 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 08:00:48.802081 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:48.801651 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0aad543f-d277-4552-8909-de30a3820f5d-kube-rbac-proxy-cm podName:0aad543f-d277-4552-8909-de30a3820f5d nodeName:}" failed. No retries permitted until 2026-09-11 08:00:56.801632878 +0000 UTC m=+93.807350529 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0aad543f-d277-4552-8909-de30a3820f5d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fssdv" (UID: "0aad543f-d277-4552-8909-de30a3820f5d") : configmap "kube-rbac-proxy" not found Sep 11 08:00:48.802081 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:48.801673 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0aad543f-d277-4552-8909-de30a3820f5d-insights-runtime-extractor-tls podName:0aad543f-d277-4552-8909-de30a3820f5d nodeName:}" failed. No retries permitted until 2026-09-11 08:00:56.801661186 +0000 UTC m=+93.807378857 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0aad543f-d277-4552-8909-de30a3820f5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fssdv" (UID: "0aad543f-d277-4552-8909-de30a3820f5d") : secret "insights-runtime-extractor-tls" not found Sep 11 08:00:50.914405 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:50.914370 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47a69b8b-6b79-426e-bf18-963fe1046241-service-ca-bundle\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:50.914736 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:50.914439 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-metrics-certs\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:00:50.914736 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:50.914527 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/47a69b8b-6b79-426e-bf18-963fe1046241-service-ca-bundle podName:47a69b8b-6b79-426e-bf18-963fe1046241 nodeName:}" failed. No retries permitted until 2026-09-11 08:01:06.914511123 +0000 UTC m=+103.920228759 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/47a69b8b-6b79-426e-bf18-963fe1046241-service-ca-bundle") pod "router-default-5f8bf9f44-82dnf" (UID: "47a69b8b-6b79-426e-bf18-963fe1046241") : configmap references non-existent config key: service-ca.crt Sep 11 08:00:50.914736 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:50.914535 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 11 08:00:50.914736 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:50.914591 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-metrics-certs podName:47a69b8b-6b79-426e-bf18-963fe1046241 nodeName:}" failed. No retries permitted until 2026-09-11 08:01:06.914575124 +0000 UTC m=+103.920292763 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-metrics-certs") pod "router-default-5f8bf9f44-82dnf" (UID: "47a69b8b-6b79-426e-bf18-963fe1046241") : secret "router-metrics-certs-default" not found Sep 11 08:00:56.862156 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:56.862115 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0aad543f-d277-4552-8909-de30a3820f5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:56.862156 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:56.862172 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0aad543f-d277-4552-8909-de30a3820f5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:00:56.862673 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:56.862417 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 08:00:56.862673 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:00:56.862479 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0aad543f-d277-4552-8909-de30a3820f5d-insights-runtime-extractor-tls podName:0aad543f-d277-4552-8909-de30a3820f5d nodeName:}" failed. No retries permitted until 2026-09-11 08:01:12.862460049 +0000 UTC m=+109.868177687 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0aad543f-d277-4552-8909-de30a3820f5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fssdv" (UID: "0aad543f-d277-4552-8909-de30a3820f5d") : secret "insights-runtime-extractor-tls" not found Sep 11 08:00:56.862673 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:00:56.862638 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0aad543f-d277-4552-8909-de30a3820f5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:01:00.187659 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:00.187611 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls\") pod \"dns-default-xl9mm\" (UID: \"022e9b3f-8f4d-40c1-989a-140d1d00f5d0\") " pod="openshift-dns/dns-default-xl9mm" Sep 11 08:01:00.188091 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:00.187688 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 08:01:00.190362 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:00.190325 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/022e9b3f-8f4d-40c1-989a-140d1d00f5d0-metrics-tls\") pod \"dns-default-xl9mm\" (UID: \"022e9b3f-8f4d-40c1-989a-140d1d00f5d0\") " pod="openshift-dns/dns-default-xl9mm" Sep 11 08:01:00.190513 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:00.190371 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls\") pod \"image-registry-6777dc96c4-2qdpb\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 08:01:00.232236 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:00.232217 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-47pmh\"" Sep 11 08:01:00.236291 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:00.236278 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 08:01:00.252144 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:00.252120 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g75rl\"" Sep 11 08:01:00.259720 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:00.259696 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xl9mm" Sep 11 08:01:00.516833 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:00.516769 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6777dc96c4-2qdpb"] Sep 11 08:01:00.519812 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:01:00.519789 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8fd7bec_f0a5_43a7_b8f5_59401860b502.slice/crio-01bf483041a389998dd057db069c0fbc5f310b1466b022cf620254bad39ea74b WatchSource:0}: Error finding container 01bf483041a389998dd057db069c0fbc5f310b1466b022cf620254bad39ea74b: Status 404 returned error can't find the container with id 01bf483041a389998dd057db069c0fbc5f310b1466b022cf620254bad39ea74b Sep 11 08:01:00.536247 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:00.536221 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xl9mm"] Sep 11 08:01:00.538143 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:01:00.538122 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod022e9b3f_8f4d_40c1_989a_140d1d00f5d0.slice/crio-21c4ed98886a632fb852ad440ed06a6d1c5b9f33adb5c4bb2d4017bef7bdaf3e WatchSource:0}: Error finding container 21c4ed98886a632fb852ad440ed06a6d1c5b9f33adb5c4bb2d4017bef7bdaf3e: Status 404 returned error can't find the container with id 21c4ed98886a632fb852ad440ed06a6d1c5b9f33adb5c4bb2d4017bef7bdaf3e Sep 11 08:01:00.833049 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:00.832975 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" event={"ID":"f8fd7bec-f0a5-43a7-b8f5-59401860b502","Type":"ContainerStarted","Data":"7a18806c45930a6758aec89c436dfb922cf838637994af16f243d705eca9974a"} Sep 11 08:01:00.833197 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:00.833161 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 08:01:00.833268 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:00.833204 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" event={"ID":"f8fd7bec-f0a5-43a7-b8f5-59401860b502","Type":"ContainerStarted","Data":"01bf483041a389998dd057db069c0fbc5f310b1466b022cf620254bad39ea74b"} Sep 11 08:01:00.834170 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:00.834149 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xl9mm" event={"ID":"022e9b3f-8f4d-40c1-989a-140d1d00f5d0","Type":"ContainerStarted","Data":"21c4ed98886a632fb852ad440ed06a6d1c5b9f33adb5c4bb2d4017bef7bdaf3e"} Sep 11 08:01:00.857092 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:00.857046 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" podStartSLOduration=96.857032202 podStartE2EDuration="1m36.857032202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 07:59:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:01:00.856449801 +0000 UTC m=+97.862167460" watchObservedRunningTime="2026-09-11 08:01:00.857032202 +0000 UTC m=+97.862749890" Sep 11 08:01:02.843762 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:02.843698 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xl9mm" event={"ID":"022e9b3f-8f4d-40c1-989a-140d1d00f5d0","Type":"ContainerStarted","Data":"06d7011a9d9ae71ed4fdf40022dda252575c34b4ec8fae58f0da66d01eafb685"} Sep 11 08:01:02.844122 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:02.843772 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xl9mm" event={"ID":"022e9b3f-8f4d-40c1-989a-140d1d00f5d0","Type":"ContainerStarted","Data":"02470cfc457f45095961ac8b6517c733219032bd3f315fcd56f12113350058d4"} Sep 11 08:01:02.844122 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:02.843835 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xl9mm" Sep 11 08:01:05.253132 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.253074 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xl9mm" podStartSLOduration=67.265143541 podStartE2EDuration="1m9.2530499s" totalImagesPullingTime="1.987906359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 07:59:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:01:02.872138086 +0000 UTC m=+99.877855744" watchObservedRunningTime="2026-09-11 08:01:05.2530499 +0000 UTC m=+102.258767557" Sep 11 08:01:05.253894 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.253874 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-gczkp"] Sep 11 08:01:05.255757 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.255737 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gczkp" Sep 11 08:01:05.258222 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.258193 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-548bc9cc64-mdlgc"] Sep 11 08:01:05.259980 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.259957 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 11 08:01:05.260236 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.260216 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548bc9cc64-mdlgc" Sep 11 08:01:05.270997 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.270974 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-547nb\"" Sep 11 08:01:05.272536 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.272506 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 11 08:01:05.274814 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.274773 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-gczkp"] Sep 11 08:01:05.290020 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.289998 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 11 08:01:05.298745 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.298722 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-548bc9cc64-mdlgc"] Sep 11 08:01:05.299821 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.299805 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 11 08:01:05.304988 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.304965 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 11 08:01:05.305088 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.305037 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 11 08:01:05.328080 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.328053 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6eafaa84-dc03-4fef-a4c6-ecb80dd2172e-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-gczkp\" (UID: \"6eafaa84-dc03-4fef-a4c6-ecb80dd2172e\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gczkp" Sep 11 08:01:05.328186 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.328084 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5fa7abe1-a3bc-4429-9500-b6217772b22e-klusterlet-config\") pod \"klusterlet-addon-workmgr-548bc9cc64-mdlgc\" (UID: \"5fa7abe1-a3bc-4429-9500-b6217772b22e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548bc9cc64-mdlgc" Sep 11 08:01:05.328186 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.328166 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6eafaa84-dc03-4fef-a4c6-ecb80dd2172e-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-gczkp\" (UID: \"6eafaa84-dc03-4fef-a4c6-ecb80dd2172e\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gczkp" Sep 11 08:01:05.328302 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.328247 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf66q\" (UniqueName: \"kubernetes.io/projected/5fa7abe1-a3bc-4429-9500-b6217772b22e-kube-api-access-zf66q\") pod \"klusterlet-addon-workmgr-548bc9cc64-mdlgc\" (UID: \"5fa7abe1-a3bc-4429-9500-b6217772b22e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548bc9cc64-mdlgc" Sep 11 08:01:05.328302 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.328288 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fa7abe1-a3bc-4429-9500-b6217772b22e-tmp\") pod \"klusterlet-addon-workmgr-548bc9cc64-mdlgc\" (UID: \"5fa7abe1-a3bc-4429-9500-b6217772b22e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548bc9cc64-mdlgc" Sep 11 08:01:05.429545 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.429523 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6eafaa84-dc03-4fef-a4c6-ecb80dd2172e-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-gczkp\" (UID: \"6eafaa84-dc03-4fef-a4c6-ecb80dd2172e\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gczkp" Sep 11 08:01:05.429622 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.429571 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf66q\" (UniqueName: \"kubernetes.io/projected/5fa7abe1-a3bc-4429-9500-b6217772b22e-kube-api-access-zf66q\") pod \"klusterlet-addon-workmgr-548bc9cc64-mdlgc\" (UID: \"5fa7abe1-a3bc-4429-9500-b6217772b22e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548bc9cc64-mdlgc" Sep 11 08:01:05.429622 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.429591 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fa7abe1-a3bc-4429-9500-b6217772b22e-tmp\") pod \"klusterlet-addon-workmgr-548bc9cc64-mdlgc\" (UID: \"5fa7abe1-a3bc-4429-9500-b6217772b22e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548bc9cc64-mdlgc" Sep 11 08:01:05.429700 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.429621 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6eafaa84-dc03-4fef-a4c6-ecb80dd2172e-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-gczkp\" (UID: \"6eafaa84-dc03-4fef-a4c6-ecb80dd2172e\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gczkp" Sep 11 08:01:05.429700 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.429642 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5fa7abe1-a3bc-4429-9500-b6217772b22e-klusterlet-config\") pod \"klusterlet-addon-workmgr-548bc9cc64-mdlgc\" (UID: \"5fa7abe1-a3bc-4429-9500-b6217772b22e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548bc9cc64-mdlgc" Sep 11 08:01:05.430084 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.430063 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fa7abe1-a3bc-4429-9500-b6217772b22e-tmp\") pod \"klusterlet-addon-workmgr-548bc9cc64-mdlgc\" (UID: \"5fa7abe1-a3bc-4429-9500-b6217772b22e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548bc9cc64-mdlgc" Sep 11 08:01:05.430307 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.430289 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6eafaa84-dc03-4fef-a4c6-ecb80dd2172e-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-gczkp\" (UID: \"6eafaa84-dc03-4fef-a4c6-ecb80dd2172e\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gczkp" Sep 11 08:01:05.432066 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.432047 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6eafaa84-dc03-4fef-a4c6-ecb80dd2172e-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-gczkp\" (UID: \"6eafaa84-dc03-4fef-a4c6-ecb80dd2172e\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gczkp" Sep 11 08:01:05.432250 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.432234 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5fa7abe1-a3bc-4429-9500-b6217772b22e-klusterlet-config\") pod \"klusterlet-addon-workmgr-548bc9cc64-mdlgc\" (UID: \"5fa7abe1-a3bc-4429-9500-b6217772b22e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548bc9cc64-mdlgc" Sep 11 08:01:05.446318 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.446290 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf66q\" (UniqueName: \"kubernetes.io/projected/5fa7abe1-a3bc-4429-9500-b6217772b22e-kube-api-access-zf66q\") pod \"klusterlet-addon-workmgr-548bc9cc64-mdlgc\" (UID: \"5fa7abe1-a3bc-4429-9500-b6217772b22e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548bc9cc64-mdlgc" Sep 11 08:01:05.566674 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.566615 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gczkp" Sep 11 08:01:05.573512 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.573496 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548bc9cc64-mdlgc" Sep 11 08:01:05.856444 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.856382 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-gczkp"] Sep 11 08:01:05.859063 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:01:05.859040 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eafaa84_dc03_4fef_a4c6_ecb80dd2172e.slice/crio-68c768304fe2ee17f7a45ae48e1e4b21a342164ee90fdb7bb287e4fdf56b0838 WatchSource:0}: Error finding container 68c768304fe2ee17f7a45ae48e1e4b21a342164ee90fdb7bb287e4fdf56b0838: Status 404 returned error can't find the container with id 68c768304fe2ee17f7a45ae48e1e4b21a342164ee90fdb7bb287e4fdf56b0838 Sep 11 08:01:05.903805 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:05.903779 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-548bc9cc64-mdlgc"] Sep 11 08:01:05.906213 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:01:05.906194 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fa7abe1_a3bc_4429_9500_b6217772b22e.slice/crio-11d046a724f556a1e6365acf2b7635d8cad3e86ec0ac69f5e38ec0f8da96af7c WatchSource:0}: Error finding container 11d046a724f556a1e6365acf2b7635d8cad3e86ec0ac69f5e38ec0f8da96af7c: Status 404 returned error can't find the container with id 11d046a724f556a1e6365acf2b7635d8cad3e86ec0ac69f5e38ec0f8da96af7c Sep 11 08:01:06.778405 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:06.778380 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bvr7m" Sep 11 08:01:06.857430 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:06.857403 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548bc9cc64-mdlgc" event={"ID":"5fa7abe1-a3bc-4429-9500-b6217772b22e","Type":"ContainerStarted","Data":"11d046a724f556a1e6365acf2b7635d8cad3e86ec0ac69f5e38ec0f8da96af7c"} Sep 11 08:01:06.858585 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:06.858562 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gczkp" event={"ID":"6eafaa84-dc03-4fef-a4c6-ecb80dd2172e","Type":"ContainerStarted","Data":"68c768304fe2ee17f7a45ae48e1e4b21a342164ee90fdb7bb287e4fdf56b0838"} Sep 11 08:01:06.940214 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:06.940185 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47a69b8b-6b79-426e-bf18-963fe1046241-service-ca-bundle\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:01:06.940322 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:06.940265 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-metrics-certs\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:01:06.940977 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:06.940931 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47a69b8b-6b79-426e-bf18-963fe1046241-service-ca-bundle\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:01:06.943215 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:06.943191 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47a69b8b-6b79-426e-bf18-963fe1046241-metrics-certs\") pod \"router-default-5f8bf9f44-82dnf\" (UID: \"47a69b8b-6b79-426e-bf18-963fe1046241\") " pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:01:06.964044 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:06.963999 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:01:07.270737 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:07.270663 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5f8bf9f44-82dnf"] Sep 11 08:01:07.722668 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:01:07.722367 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47a69b8b_6b79_426e_bf18_963fe1046241.slice/crio-f69f835a0892817444626ebbe7e6f2b3672820e24c29e0df2e263e0bbfd93d36 WatchSource:0}: Error finding container f69f835a0892817444626ebbe7e6f2b3672820e24c29e0df2e263e0bbfd93d36: Status 404 returned error can't find the container with id f69f835a0892817444626ebbe7e6f2b3672820e24c29e0df2e263e0bbfd93d36 Sep 11 08:01:07.865737 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:07.865178 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5f8bf9f44-82dnf" event={"ID":"47a69b8b-6b79-426e-bf18-963fe1046241","Type":"ContainerStarted","Data":"cb55ed5cfe896ace34f59106c6c1e06a1461122983b22407c27e8f071b605fe0"} Sep 11 08:01:07.865737 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:07.865253 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5f8bf9f44-82dnf" event={"ID":"47a69b8b-6b79-426e-bf18-963fe1046241","Type":"ContainerStarted","Data":"f69f835a0892817444626ebbe7e6f2b3672820e24c29e0df2e263e0bbfd93d36"} Sep 11 08:01:07.888039 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:07.887984 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5f8bf9f44-82dnf" podStartSLOduration=32.887963958 podStartE2EDuration="32.887963958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:00:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:01:07.887027888 +0000 UTC m=+104.892745563" watchObservedRunningTime="2026-09-11 08:01:07.887963958 +0000 UTC m=+104.893681633" Sep 11 08:01:07.964351 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:07.964317 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:01:07.967109 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:07.967066 2557 patch_prober.go:28] interesting pod/router-default-5f8bf9f44-82dnf container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Sep 11 08:01:07.967109 ip-10-0-136-245 kubenswrapper[2557]: [+]has-synced ok Sep 11 08:01:07.967109 ip-10-0-136-245 kubenswrapper[2557]: [+]process-running ok Sep 11 08:01:07.967109 ip-10-0-136-245 kubenswrapper[2557]: healthz check failed Sep 11 08:01:07.967384 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:07.967134 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5f8bf9f44-82dnf" podUID="47a69b8b-6b79-426e-bf18-963fe1046241" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 11 08:01:08.966655 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:08.966628 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:01:09.871665 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:09.871626 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:01:09.875749 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:09.873644 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5f8bf9f44-82dnf" Sep 11 08:01:10.877618 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:10.877579 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gczkp" event={"ID":"6eafaa84-dc03-4fef-a4c6-ecb80dd2172e","Type":"ContainerStarted","Data":"dd26b846be2974030dea42a0184bee734a1cb5630c9ec2c50bb790b3a6783a40"} Sep 11 08:01:10.878990 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:10.878956 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548bc9cc64-mdlgc" event={"ID":"5fa7abe1-a3bc-4429-9500-b6217772b22e","Type":"ContainerStarted","Data":"d8493ea7c4b77ff73d03f3cad42eef5f4ab7601ae3902785a5d31dec282b10a1"} Sep 11 08:01:10.894831 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:10.894788 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gczkp" podStartSLOduration=1.739309928 podStartE2EDuration="5.894774234s" totalImagesPullingTime="4.155464306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:01:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:01:10.894217531 +0000 UTC m=+107.899935186" watchObservedRunningTime="2026-09-11 08:01:10.894774234 +0000 UTC m=+107.900491891" Sep 11 08:01:10.931486 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:10.931405 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548bc9cc64-mdlgc" podStartSLOduration=1.78444856 podStartE2EDuration="5.93139181s" totalImagesPullingTime="4.14694325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:01:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:01:10.930961757 +0000 UTC m=+107.936679440" watchObservedRunningTime="2026-09-11 08:01:10.93139181 +0000 UTC m=+107.937109479" Sep 11 08:01:11.647747 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:11.647722 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-d5jvv"] Sep 11 08:01:11.649711 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:11.649691 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-d5jvv" Sep 11 08:01:11.651999 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:11.651978 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 11 08:01:11.652205 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:11.652189 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-d29j6\"" Sep 11 08:01:11.663125 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:11.663105 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-d5jvv"] Sep 11 08:01:11.777085 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:11.777062 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/931723ec-ab59-48c7-8d99-881c5144da26-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-d5jvv\" (UID: \"931723ec-ab59-48c7-8d99-881c5144da26\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-d5jvv" Sep 11 08:01:11.878300 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:11.878276 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/931723ec-ab59-48c7-8d99-881c5144da26-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-d5jvv\" (UID: \"931723ec-ab59-48c7-8d99-881c5144da26\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-d5jvv" Sep 11 08:01:11.880602 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:11.880583 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/931723ec-ab59-48c7-8d99-881c5144da26-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-d5jvv\" (UID: \"931723ec-ab59-48c7-8d99-881c5144da26\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-d5jvv" Sep 11 08:01:11.882505 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:11.882485 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548bc9cc64-mdlgc" Sep 11 08:01:11.883888 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:11.883872 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548bc9cc64-mdlgc" Sep 11 08:01:11.958950 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:11.958888 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-d5jvv" Sep 11 08:01:12.237974 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:12.237949 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-d5jvv"] Sep 11 08:01:12.240128 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:01:12.240099 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod931723ec_ab59_48c7_8d99_881c5144da26.slice/crio-af2e50ff4ccc2d4c7ccec584500221528aae0e09849aa106ed0c41aeec36be54 WatchSource:0}: Error finding container af2e50ff4ccc2d4c7ccec584500221528aae0e09849aa106ed0c41aeec36be54: Status 404 returned error can't find the container with id af2e50ff4ccc2d4c7ccec584500221528aae0e09849aa106ed0c41aeec36be54 Sep 11 08:01:12.849446 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:12.849424 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xl9mm" Sep 11 08:01:12.890016 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:12.886455 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0aad543f-d277-4552-8909-de30a3820f5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:01:12.890016 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:12.888390 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-d5jvv" event={"ID":"931723ec-ab59-48c7-8d99-881c5144da26","Type":"ContainerStarted","Data":"af2e50ff4ccc2d4c7ccec584500221528aae0e09849aa106ed0c41aeec36be54"} Sep 11 08:01:12.890016 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:12.889349 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0aad543f-d277-4552-8909-de30a3820f5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fssdv\" (UID: \"0aad543f-d277-4552-8909-de30a3820f5d\") " pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:01:12.936283 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:12.936255 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-ccb84d448-fxzbx"] Sep 11 08:01:12.939144 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:12.939125 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:12.944686 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:12.944666 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 11 08:01:12.944795 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:12.944701 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 11 08:01:12.945018 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:12.944998 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 11 08:01:12.945609 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:12.945354 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 11 08:01:12.946492 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:12.946471 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-pzhvc\"" Sep 11 08:01:12.946603 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:12.946531 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 11 08:01:12.947705 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:12.947690 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 11 08:01:12.947794 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:12.947710 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 11 08:01:12.969564 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:12.969463 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-ccb84d448-fxzbx"] Sep 11 08:01:13.088735 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.088703 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-oauth-serving-cert\") pod \"console-ccb84d448-fxzbx\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:13.088735 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.088738 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-console-serving-cert\") pod \"console-ccb84d448-fxzbx\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:13.088857 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.088755 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-console-oauth-config\") pod \"console-ccb84d448-fxzbx\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:13.088857 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.088809 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-console-config\") pod \"console-ccb84d448-fxzbx\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:13.088857 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.088838 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rrdc\" (UniqueName: \"kubernetes.io/projected/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-kube-api-access-5rrdc\") pod \"console-ccb84d448-fxzbx\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:13.088982 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.088881 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-service-ca\") pod \"console-ccb84d448-fxzbx\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:13.155987 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.155965 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fssdv" Sep 11 08:01:13.189914 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.189867 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-console-config\") pod \"console-ccb84d448-fxzbx\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:13.190035 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.189930 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rrdc\" (UniqueName: \"kubernetes.io/projected/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-kube-api-access-5rrdc\") pod \"console-ccb84d448-fxzbx\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:13.190035 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.189966 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-service-ca\") pod \"console-ccb84d448-fxzbx\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:13.190203 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.190174 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-oauth-serving-cert\") pod \"console-ccb84d448-fxzbx\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:13.190273 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.190227 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-console-serving-cert\") pod \"console-ccb84d448-fxzbx\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:13.190273 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.190256 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-console-oauth-config\") pod \"console-ccb84d448-fxzbx\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:13.190784 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.190750 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-service-ca\") pod \"console-ccb84d448-fxzbx\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:13.190883 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.190811 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-oauth-serving-cert\") pod \"console-ccb84d448-fxzbx\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:13.190883 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.190848 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-console-config\") pod \"console-ccb84d448-fxzbx\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:13.192552 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.192531 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-console-oauth-config\") pod \"console-ccb84d448-fxzbx\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:13.193017 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.193000 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-console-serving-cert\") pod \"console-ccb84d448-fxzbx\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:13.204209 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.204147 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rrdc\" (UniqueName: \"kubernetes.io/projected/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-kube-api-access-5rrdc\") pod \"console-ccb84d448-fxzbx\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:13.248340 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.248320 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:13.437444 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.437379 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fssdv"] Sep 11 08:01:13.440601 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:01:13.440575 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aad543f_d277_4552_8909_de30a3820f5d.slice/crio-2d3c1f180c17e89845ebef2f87846049cb861f12f9f7d0e4cede5e4e1f753f94 WatchSource:0}: Error finding container 2d3c1f180c17e89845ebef2f87846049cb861f12f9f7d0e4cede5e4e1f753f94: Status 404 returned error can't find the container with id 2d3c1f180c17e89845ebef2f87846049cb861f12f9f7d0e4cede5e4e1f753f94 Sep 11 08:01:13.542853 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.542828 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-ccb84d448-fxzbx"] Sep 11 08:01:13.545128 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:01:13.545103 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc17f4ec2_c74a_4227_ac55_a4dfc6b290e1.slice/crio-fb7268b4df421b8f771bc6a68e97cdc5d2c7ac951b1b962b333c4590aa2cd910 WatchSource:0}: Error finding container fb7268b4df421b8f771bc6a68e97cdc5d2c7ac951b1b962b333c4590aa2cd910: Status 404 returned error can't find the container with id fb7268b4df421b8f771bc6a68e97cdc5d2c7ac951b1b962b333c4590aa2cd910 Sep 11 08:01:13.892985 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.892953 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fssdv" event={"ID":"0aad543f-d277-4552-8909-de30a3820f5d","Type":"ContainerStarted","Data":"8a1c350874ddaa09232858bfee6304dfb16409eab852cdcdf23471c0f1fd5321"} Sep 11 08:01:13.893408 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.893016 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fssdv" event={"ID":"0aad543f-d277-4552-8909-de30a3820f5d","Type":"ContainerStarted","Data":"2d3c1f180c17e89845ebef2f87846049cb861f12f9f7d0e4cede5e4e1f753f94"} Sep 11 08:01:13.894429 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:13.894385 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ccb84d448-fxzbx" event={"ID":"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1","Type":"ContainerStarted","Data":"fb7268b4df421b8f771bc6a68e97cdc5d2c7ac951b1b962b333c4590aa2cd910"} Sep 11 08:01:14.899716 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:14.899682 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-d5jvv" event={"ID":"931723ec-ab59-48c7-8d99-881c5144da26","Type":"ContainerStarted","Data":"abb8d1e9f103b753a3e885bd5925dc37e14fa9d116c451ecc5afa649395d527a"} Sep 11 08:01:14.900158 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:14.899916 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-d5jvv" Sep 11 08:01:14.901873 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:14.901844 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fssdv" event={"ID":"0aad543f-d277-4552-8909-de30a3820f5d","Type":"ContainerStarted","Data":"cacb63e32614221ccc63f0b3893757e80f39e8597095452319fcc788841322df"} Sep 11 08:01:14.905494 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:14.905471 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-d5jvv" Sep 11 08:01:14.922841 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:14.922794 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-d5jvv" podStartSLOduration=2.212212661 podStartE2EDuration="3.922777799s" totalImagesPullingTime="1.710565138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:01:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:01:14.921356953 +0000 UTC m=+111.927074611" watchObservedRunningTime="2026-09-11 08:01:14.922777799 +0000 UTC m=+111.928495458" Sep 11 08:01:15.522783 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.522742 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5844bd5f5-xk6lm"] Sep 11 08:01:15.525943 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.525917 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.534370 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.534175 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 11 08:01:15.560326 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.560299 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5844bd5f5-xk6lm"] Sep 11 08:01:15.610030 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.609880 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/562bcd6d-2e72-4be9-9d98-893487decb92-console-serving-cert\") pod \"console-5844bd5f5-xk6lm\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.610030 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.609945 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/562bcd6d-2e72-4be9-9d98-893487decb92-console-oauth-config\") pod \"console-5844bd5f5-xk6lm\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.610030 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.609982 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-oauth-serving-cert\") pod \"console-5844bd5f5-xk6lm\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.610263 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.610072 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-service-ca\") pod \"console-5844bd5f5-xk6lm\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.610263 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.610105 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t5fc\" (UniqueName: \"kubernetes.io/projected/562bcd6d-2e72-4be9-9d98-893487decb92-kube-api-access-6t5fc\") pod \"console-5844bd5f5-xk6lm\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.610263 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.610140 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-console-config\") pod \"console-5844bd5f5-xk6lm\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.610263 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.610220 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-trusted-ca-bundle\") pod \"console-5844bd5f5-xk6lm\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.711548 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.711514 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/562bcd6d-2e72-4be9-9d98-893487decb92-console-serving-cert\") pod \"console-5844bd5f5-xk6lm\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.711548 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.711552 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/562bcd6d-2e72-4be9-9d98-893487decb92-console-oauth-config\") pod \"console-5844bd5f5-xk6lm\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.711757 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.711576 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-oauth-serving-cert\") pod \"console-5844bd5f5-xk6lm\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.711757 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.711627 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-service-ca\") pod \"console-5844bd5f5-xk6lm\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.711757 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.711653 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t5fc\" (UniqueName: \"kubernetes.io/projected/562bcd6d-2e72-4be9-9d98-893487decb92-kube-api-access-6t5fc\") pod \"console-5844bd5f5-xk6lm\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.711757 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.711688 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-console-config\") pod \"console-5844bd5f5-xk6lm\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.711757 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.711723 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-trusted-ca-bundle\") pod \"console-5844bd5f5-xk6lm\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.712490 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.712441 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-oauth-serving-cert\") pod \"console-5844bd5f5-xk6lm\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.712646 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.712620 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-service-ca\") pod \"console-5844bd5f5-xk6lm\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.712742 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.712632 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-console-config\") pod \"console-5844bd5f5-xk6lm\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.712806 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.712746 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-trusted-ca-bundle\") pod \"console-5844bd5f5-xk6lm\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.714508 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.714483 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/562bcd6d-2e72-4be9-9d98-893487decb92-console-serving-cert\") pod \"console-5844bd5f5-xk6lm\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.715029 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.715009 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/562bcd6d-2e72-4be9-9d98-893487decb92-console-oauth-config\") pod \"console-5844bd5f5-xk6lm\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.730976 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.730957 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t5fc\" (UniqueName: \"kubernetes.io/projected/562bcd6d-2e72-4be9-9d98-893487decb92-kube-api-access-6t5fc\") pod \"console-5844bd5f5-xk6lm\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.837595 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.837507 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:15.854874 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.854848 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-hkdql"] Sep 11 08:01:15.858146 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.858122 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hkdql" Sep 11 08:01:15.873805 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:01:15.873775 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"prometheus-operator-dockercfg-jtbkm\" is forbidden: User \"system:node:ip-10-0-136-245.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-245.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-jtbkm\"" type="*v1.Secret" Sep 11 08:01:15.875134 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:01:15.875095 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"prometheus-operator-kube-rbac-proxy-config\" is forbidden: User \"system:node:ip-10-0-136-245.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-245.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" type="*v1.Secret" Sep 11 08:01:15.875221 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:01:15.875124 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"prometheus-operator-68bcd7f987-hkdql\" is forbidden: User \"system:node:ip-10-0-136-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-245.ec2.internal' and this object" podUID="68645e9e-5833-4cdc-896d-86ee7f234140" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hkdql" Sep 11 08:01:15.875221 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:01:15.875134 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-136-245.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-245.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Sep 11 08:01:15.875221 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:01:15.875148 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-136-245.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-245.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Sep 11 08:01:15.875221 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:01:15.875162 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"prometheus-operator-tls\" is forbidden: User \"system:node:ip-10-0-136-245.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-245.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" type="*v1.Secret" Sep 11 08:01:15.875221 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:01:15.875148 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"metrics-client-ca\" is forbidden: User \"system:node:ip-10-0-136-245.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-245.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" type="*v1.ConfigMap" Sep 11 08:01:15.909271 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:15.909248 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-hkdql"] Sep 11 08:01:16.014301 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:16.014269 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/68645e9e-5833-4cdc-896d-86ee7f234140-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-hkdql\" (UID: \"68645e9e-5833-4cdc-896d-86ee7f234140\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hkdql" Sep 11 08:01:16.014480 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:16.014344 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scmrg\" (UniqueName: \"kubernetes.io/projected/68645e9e-5833-4cdc-896d-86ee7f234140-kube-api-access-scmrg\") pod \"prometheus-operator-68bcd7f987-hkdql\" (UID: \"68645e9e-5833-4cdc-896d-86ee7f234140\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hkdql" Sep 11 08:01:16.014480 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:16.014413 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68645e9e-5833-4cdc-896d-86ee7f234140-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-hkdql\" (UID: \"68645e9e-5833-4cdc-896d-86ee7f234140\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hkdql" Sep 11 08:01:16.014480 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:16.014438 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68645e9e-5833-4cdc-896d-86ee7f234140-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-hkdql\" (UID: \"68645e9e-5833-4cdc-896d-86ee7f234140\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hkdql" Sep 11 08:01:16.115470 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:16.115379 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68645e9e-5833-4cdc-896d-86ee7f234140-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-hkdql\" (UID: \"68645e9e-5833-4cdc-896d-86ee7f234140\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hkdql" Sep 11 08:01:16.115470 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:16.115426 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68645e9e-5833-4cdc-896d-86ee7f234140-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-hkdql\" (UID: \"68645e9e-5833-4cdc-896d-86ee7f234140\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hkdql" Sep 11 08:01:16.115470 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:16.115465 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/68645e9e-5833-4cdc-896d-86ee7f234140-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-hkdql\" (UID: \"68645e9e-5833-4cdc-896d-86ee7f234140\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hkdql" Sep 11 08:01:16.115761 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:16.115551 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scmrg\" (UniqueName: \"kubernetes.io/projected/68645e9e-5833-4cdc-896d-86ee7f234140-kube-api-access-scmrg\") pod \"prometheus-operator-68bcd7f987-hkdql\" (UID: \"68645e9e-5833-4cdc-896d-86ee7f234140\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hkdql" Sep 11 08:01:16.685645 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:16.685612 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 11 08:01:16.685962 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:16.685938 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 11 08:01:16.688121 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:16.688094 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/68645e9e-5833-4cdc-896d-86ee7f234140-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-hkdql\" (UID: \"68645e9e-5833-4cdc-896d-86ee7f234140\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hkdql" Sep 11 08:01:16.698339 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:16.698318 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68645e9e-5833-4cdc-896d-86ee7f234140-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-hkdql\" (UID: \"68645e9e-5833-4cdc-896d-86ee7f234140\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hkdql" Sep 11 08:01:16.965192 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:16.965128 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 11 08:01:16.966563 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:16.966545 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68645e9e-5833-4cdc-896d-86ee7f234140-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-hkdql\" (UID: \"68645e9e-5833-4cdc-896d-86ee7f234140\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hkdql" Sep 11 08:01:17.115213 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:17.115190 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5844bd5f5-xk6lm"] Sep 11 08:01:17.115998 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:01:17.115979 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod562bcd6d_2e72_4be9_9d98_893487decb92.slice/crio-e3f7c0cb831045d25c412b342d503fc4c92af2bfceea53fd87542acb389b0e1b WatchSource:0}: Error finding container e3f7c0cb831045d25c412b342d503fc4c92af2bfceea53fd87542acb389b0e1b: Status 404 returned error can't find the container with id e3f7c0cb831045d25c412b342d503fc4c92af2bfceea53fd87542acb389b0e1b Sep 11 08:01:17.126941 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:01:17.126921 2557 projected.go:291] Couldn't get configMap openshift-monitoring/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Sep 11 08:01:17.147222 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:17.147167 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 11 08:01:17.157125 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:01:17.157104 2557 projected.go:196] Error preparing data for projected volume kube-api-access-scmrg for pod openshift-monitoring/prometheus-operator-68bcd7f987-hkdql: failed to sync configmap cache: timed out waiting for the condition Sep 11 08:01:17.157205 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:01:17.157154 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68645e9e-5833-4cdc-896d-86ee7f234140-kube-api-access-scmrg podName:68645e9e-5833-4cdc-896d-86ee7f234140 nodeName:}" failed. No retries permitted until 2026-09-11 08:01:17.657140735 +0000 UTC m=+114.662858384 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-scmrg" (UniqueName: "kubernetes.io/projected/68645e9e-5833-4cdc-896d-86ee7f234140-kube-api-access-scmrg") pod "prometheus-operator-68bcd7f987-hkdql" (UID: "68645e9e-5833-4cdc-896d-86ee7f234140") : failed to sync configmap cache: timed out waiting for the condition Sep 11 08:01:17.308029 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:17.307961 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-jtbkm\"" Sep 11 08:01:17.468638 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:17.468610 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 11 08:01:17.728350 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:17.728323 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scmrg\" (UniqueName: \"kubernetes.io/projected/68645e9e-5833-4cdc-896d-86ee7f234140-kube-api-access-scmrg\") pod \"prometheus-operator-68bcd7f987-hkdql\" (UID: \"68645e9e-5833-4cdc-896d-86ee7f234140\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hkdql" Sep 11 08:01:17.730830 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:17.730812 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scmrg\" (UniqueName: \"kubernetes.io/projected/68645e9e-5833-4cdc-896d-86ee7f234140-kube-api-access-scmrg\") pod \"prometheus-operator-68bcd7f987-hkdql\" (UID: \"68645e9e-5833-4cdc-896d-86ee7f234140\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hkdql" Sep 11 08:01:17.915807 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:17.915772 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5844bd5f5-xk6lm" event={"ID":"562bcd6d-2e72-4be9-9d98-893487decb92","Type":"ContainerStarted","Data":"29c30a87597666fba0545c5f7628b7d5d87c8c8b1e41290f0b195686c777fe65"} Sep 11 08:01:17.915946 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:17.915827 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5844bd5f5-xk6lm" event={"ID":"562bcd6d-2e72-4be9-9d98-893487decb92","Type":"ContainerStarted","Data":"e3f7c0cb831045d25c412b342d503fc4c92af2bfceea53fd87542acb389b0e1b"} Sep 11 08:01:17.917840 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:17.917816 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fssdv" event={"ID":"0aad543f-d277-4552-8909-de30a3820f5d","Type":"ContainerStarted","Data":"3433f2e1951530f7bfc12cebb66e665275f969fc30ca045c91441fb8aaa486cd"} Sep 11 08:01:17.919290 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:17.919270 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ccb84d448-fxzbx" event={"ID":"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1","Type":"ContainerStarted","Data":"fd73ae1acec0643836c64f32ec94fe6d5ab52bd71828377d9d42ec8027e6a971"} Sep 11 08:01:17.956224 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:17.956183 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5844bd5f5-xk6lm" podStartSLOduration=2.956171233 podStartE2EDuration="2.956171233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:01:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:01:17.955750153 +0000 UTC m=+114.961467821" watchObservedRunningTime="2026-09-11 08:01:17.956171233 +0000 UTC m=+114.961888891" Sep 11 08:01:17.969979 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:17.969958 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hkdql" Sep 11 08:01:17.987081 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:17.982846 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-ccb84d448-fxzbx" podStartSLOduration=2.694105146 podStartE2EDuration="5.98282261s" totalImagesPullingTime="3.288717464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:01:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:01:17.982582432 +0000 UTC m=+114.988300116" watchObservedRunningTime="2026-09-11 08:01:17.98282261 +0000 UTC m=+114.988540266" Sep 11 08:01:18.012682 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:18.012642 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fssdv" podStartSLOduration=33.745341392 podStartE2EDuration="37.012627796s" totalImagesPullingTime="3.267286404s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 08:00:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:01:18.012512444 +0000 UTC m=+115.018230103" watchObservedRunningTime="2026-09-11 08:01:18.012627796 +0000 UTC m=+115.018345454" Sep 11 08:01:18.251425 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:18.251369 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-hkdql"] Sep 11 08:01:18.254073 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:01:18.254050 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68645e9e_5833_4cdc_896d_86ee7f234140.slice/crio-96de8d5b4a786d85e911036c9158ffeade9bcb6752d03c27d6ba722ad4bdd777 WatchSource:0}: Error finding container 96de8d5b4a786d85e911036c9158ffeade9bcb6752d03c27d6ba722ad4bdd777: Status 404 returned error can't find the container with id 96de8d5b4a786d85e911036c9158ffeade9bcb6752d03c27d6ba722ad4bdd777 Sep 11 08:01:18.923771 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:18.923736 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hkdql" event={"ID":"68645e9e-5833-4cdc-896d-86ee7f234140","Type":"ContainerStarted","Data":"96de8d5b4a786d85e911036c9158ffeade9bcb6752d03c27d6ba722ad4bdd777"} Sep 11 08:01:20.241799 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:20.241774 2557 patch_prober.go:28] interesting pod/image-registry-6777dc96c4-2qdpb container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 11 08:01:20.242101 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:20.241818 2557 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" podUID="f8fd7bec-f0a5-43a7-b8f5-59401860b502" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 08:01:20.933575 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:20.933543 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hkdql" event={"ID":"68645e9e-5833-4cdc-896d-86ee7f234140","Type":"ContainerStarted","Data":"d781a658774016717cf84ecb171ffde3e6b33ae2069b5e9b309ee6184da63ad2"} Sep 11 08:01:20.933575 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:20.933593 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hkdql" event={"ID":"68645e9e-5833-4cdc-896d-86ee7f234140","Type":"ContainerStarted","Data":"ddb25feacb52d047ae713aae2d287df266f2c7587277315203e8e64196e9aae4"} Sep 11 08:01:20.955884 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:20.955837 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hkdql" podStartSLOduration=4.079173105 podStartE2EDuration="5.955820812s" totalImagesPullingTime="1.876647707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:01:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:01:20.955045682 +0000 UTC m=+117.960763339" watchObservedRunningTime="2026-09-11 08:01:20.955820812 +0000 UTC m=+117.961538470" Sep 11 08:01:21.841080 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:21.841055 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 08:01:23.248685 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.248641 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:23.249190 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.248742 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:23.253505 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.253484 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:23.326682 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.326656 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct"] Sep 11 08:01:23.329965 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.329949 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct" Sep 11 08:01:23.335108 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.335085 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 11 08:01:23.335359 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.335340 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 11 08:01:23.335408 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.335352 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4bcnf\"" Sep 11 08:01:23.357642 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.357623 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct"] Sep 11 08:01:23.369376 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.369357 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5n7lg"] Sep 11 08:01:23.372713 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.372696 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.377347 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.377330 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 11 08:01:23.378085 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.378067 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 11 08:01:23.379415 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.379398 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 11 08:01:23.380523 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.380506 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-f78r6\"" Sep 11 08:01:23.471656 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.471633 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dcaa449-e314-4d02-ac8f-de897a090057-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-wn6ct\" (UID: \"6dcaa449-e314-4d02-ac8f-de897a090057\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct" Sep 11 08:01:23.471787 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.471662 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e910d42e-6726-46ca-978a-e687d028af81-node-exporter-tls\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.471787 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.471698 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e910d42e-6726-46ca-978a-e687d028af81-metrics-client-ca\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.471787 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.471715 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e910d42e-6726-46ca-978a-e687d028af81-node-exporter-wtmp\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.471787 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.471734 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6v8s\" (UniqueName: \"kubernetes.io/projected/6dcaa449-e314-4d02-ac8f-de897a090057-kube-api-access-h6v8s\") pod \"openshift-state-metrics-67dcf8c979-wn6ct\" (UID: \"6dcaa449-e314-4d02-ac8f-de897a090057\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct" Sep 11 08:01:23.471787 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.471756 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq2w5\" (UniqueName: \"kubernetes.io/projected/e910d42e-6726-46ca-978a-e687d028af81-kube-api-access-gq2w5\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.472033 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.471808 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e910d42e-6726-46ca-978a-e687d028af81-root\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.472033 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.471830 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e910d42e-6726-46ca-978a-e687d028af81-node-exporter-textfile\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.472033 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.471860 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6dcaa449-e314-4d02-ac8f-de897a090057-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-wn6ct\" (UID: \"6dcaa449-e314-4d02-ac8f-de897a090057\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct" Sep 11 08:01:23.472033 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.471874 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e910d42e-6726-46ca-978a-e687d028af81-sys\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.472033 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.471893 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e910d42e-6726-46ca-978a-e687d028af81-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.472033 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.471942 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6dcaa449-e314-4d02-ac8f-de897a090057-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-wn6ct\" (UID: \"6dcaa449-e314-4d02-ac8f-de897a090057\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct" Sep 11 08:01:23.472033 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.471959 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e910d42e-6726-46ca-978a-e687d028af81-node-exporter-accelerators-collector-config\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.573266 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.573192 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e910d42e-6726-46ca-978a-e687d028af81-metrics-client-ca\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.573266 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.573221 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e910d42e-6726-46ca-978a-e687d028af81-node-exporter-wtmp\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.573266 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.573241 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6v8s\" (UniqueName: \"kubernetes.io/projected/6dcaa449-e314-4d02-ac8f-de897a090057-kube-api-access-h6v8s\") pod \"openshift-state-metrics-67dcf8c979-wn6ct\" (UID: \"6dcaa449-e314-4d02-ac8f-de897a090057\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct" Sep 11 08:01:23.573266 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.573256 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq2w5\" (UniqueName: \"kubernetes.io/projected/e910d42e-6726-46ca-978a-e687d028af81-kube-api-access-gq2w5\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.573570 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.573278 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e910d42e-6726-46ca-978a-e687d028af81-root\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.573570 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.573304 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e910d42e-6726-46ca-978a-e687d028af81-node-exporter-textfile\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.573570 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.573337 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6dcaa449-e314-4d02-ac8f-de897a090057-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-wn6ct\" (UID: \"6dcaa449-e314-4d02-ac8f-de897a090057\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct" Sep 11 08:01:23.573570 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.573361 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e910d42e-6726-46ca-978a-e687d028af81-sys\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.573570 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.573382 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e910d42e-6726-46ca-978a-e687d028af81-node-exporter-wtmp\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.573570 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.573395 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e910d42e-6726-46ca-978a-e687d028af81-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.573570 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.573451 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6dcaa449-e314-4d02-ac8f-de897a090057-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-wn6ct\" (UID: \"6dcaa449-e314-4d02-ac8f-de897a090057\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct" Sep 11 08:01:23.573570 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.573382 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e910d42e-6726-46ca-978a-e687d028af81-root\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.573570 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.573481 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e910d42e-6726-46ca-978a-e687d028af81-node-exporter-accelerators-collector-config\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.573570 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.573547 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dcaa449-e314-4d02-ac8f-de897a090057-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-wn6ct\" (UID: \"6dcaa449-e314-4d02-ac8f-de897a090057\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct" Sep 11 08:01:23.573570 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.573460 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e910d42e-6726-46ca-978a-e687d028af81-sys\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.573570 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.573570 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e910d42e-6726-46ca-978a-e687d028af81-node-exporter-tls\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.574167 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:01:23.573672 2557 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 11 08:01:23.574167 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:01:23.573723 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e910d42e-6726-46ca-978a-e687d028af81-node-exporter-tls podName:e910d42e-6726-46ca-978a-e687d028af81 nodeName:}" failed. No retries permitted until 2026-09-11 08:01:24.07370676 +0000 UTC m=+121.079424406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e910d42e-6726-46ca-978a-e687d028af81-node-exporter-tls") pod "node-exporter-5n7lg" (UID: "e910d42e-6726-46ca-978a-e687d028af81") : secret "node-exporter-tls" not found Sep 11 08:01:23.574167 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.573988 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e910d42e-6726-46ca-978a-e687d028af81-metrics-client-ca\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.574167 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.574025 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e910d42e-6726-46ca-978a-e687d028af81-node-exporter-textfile\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.574167 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.574060 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e910d42e-6726-46ca-978a-e687d028af81-node-exporter-accelerators-collector-config\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.574377 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.574243 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6dcaa449-e314-4d02-ac8f-de897a090057-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-wn6ct\" (UID: \"6dcaa449-e314-4d02-ac8f-de897a090057\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct" Sep 11 08:01:23.575624 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.575604 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e910d42e-6726-46ca-978a-e687d028af81-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.598843 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.598815 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 11 08:01:23.598843 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.598842 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 11 08:01:23.603748 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:01:23.603725 2557 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Sep 11 08:01:23.603850 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:01:23.603806 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dcaa449-e314-4d02-ac8f-de897a090057-openshift-state-metrics-tls podName:6dcaa449-e314-4d02-ac8f-de897a090057 nodeName:}" failed. No retries permitted until 2026-09-11 08:01:24.103787748 +0000 UTC m=+121.109505383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/6dcaa449-e314-4d02-ac8f-de897a090057-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-wn6ct" (UID: "6dcaa449-e314-4d02-ac8f-de897a090057") : secret "openshift-state-metrics-tls" not found Sep 11 08:01:23.604550 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.604526 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq2w5\" (UniqueName: \"kubernetes.io/projected/e910d42e-6726-46ca-978a-e687d028af81-kube-api-access-gq2w5\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:23.606049 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.606030 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6dcaa449-e314-4d02-ac8f-de897a090057-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-wn6ct\" (UID: \"6dcaa449-e314-4d02-ac8f-de897a090057\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct" Sep 11 08:01:23.725293 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.725268 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6v8s\" (UniqueName: \"kubernetes.io/projected/6dcaa449-e314-4d02-ac8f-de897a090057-kube-api-access-h6v8s\") pod \"openshift-state-metrics-67dcf8c979-wn6ct\" (UID: \"6dcaa449-e314-4d02-ac8f-de897a090057\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct" Sep 11 08:01:23.950564 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:23.950533 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:24.077758 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:24.077721 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e910d42e-6726-46ca-978a-e687d028af81-node-exporter-tls\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:24.080094 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:24.080070 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e910d42e-6726-46ca-978a-e687d028af81-node-exporter-tls\") pod \"node-exporter-5n7lg\" (UID: \"e910d42e-6726-46ca-978a-e687d028af81\") " pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:24.178463 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:24.178415 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dcaa449-e314-4d02-ac8f-de897a090057-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-wn6ct\" (UID: \"6dcaa449-e314-4d02-ac8f-de897a090057\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct" Sep 11 08:01:24.180748 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:24.180725 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dcaa449-e314-4d02-ac8f-de897a090057-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-wn6ct\" (UID: \"6dcaa449-e314-4d02-ac8f-de897a090057\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct" Sep 11 08:01:24.240937 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:24.240849 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4bcnf\"" Sep 11 08:01:24.249811 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:24.249794 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct" Sep 11 08:01:24.281004 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:24.280972 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5n7lg" Sep 11 08:01:24.293010 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:01:24.292984 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode910d42e_6726_46ca_978a_e687d028af81.slice/crio-bba850bd06ddc6da12a32d6321fd35edd4bde3546ac9e9d13abb116817c9638c WatchSource:0}: Error finding container bba850bd06ddc6da12a32d6321fd35edd4bde3546ac9e9d13abb116817c9638c: Status 404 returned error can't find the container with id bba850bd06ddc6da12a32d6321fd35edd4bde3546ac9e9d13abb116817c9638c Sep 11 08:01:24.553750 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:24.553675 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct"] Sep 11 08:01:24.950410 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:24.950371 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5n7lg" event={"ID":"e910d42e-6726-46ca-978a-e687d028af81","Type":"ContainerStarted","Data":"bba850bd06ddc6da12a32d6321fd35edd4bde3546ac9e9d13abb116817c9638c"} Sep 11 08:01:24.952011 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:24.951988 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct" event={"ID":"6dcaa449-e314-4d02-ac8f-de897a090057","Type":"ContainerStarted","Data":"514b88f314d4d372db2d21ad47e964857493130309d9b2e2d84fbc3f22a373cc"} Sep 11 08:01:24.952094 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:24.952039 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct" event={"ID":"6dcaa449-e314-4d02-ac8f-de897a090057","Type":"ContainerStarted","Data":"05c5acdd85de2c47dd9d4be535aab99b8f8eb556a25996d260c13a3c8512697e"} Sep 11 08:01:24.952094 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:24.952058 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct" event={"ID":"6dcaa449-e314-4d02-ac8f-de897a090057","Type":"ContainerStarted","Data":"353add3165d48d283b18356e956a3f0c0529b2d6f319d4a948af88d9b8e66396"} Sep 11 08:01:25.837753 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:25.837669 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:25.837753 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:25.837733 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:25.842531 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:25.842510 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:25.956908 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:25.956873 2557 generic.go:356] "Generic (PLEG): container finished" podID="e910d42e-6726-46ca-978a-e687d028af81" containerID="e0dcc45934393faee45cf459fbf8cf7e6f59ee7a289d49b23a5b1115db3de091" exitCode=0 Sep 11 08:01:25.957014 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:25.956934 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5n7lg" event={"ID":"e910d42e-6726-46ca-978a-e687d028af81","Type":"ContainerDied","Data":"e0dcc45934393faee45cf459fbf8cf7e6f59ee7a289d49b23a5b1115db3de091"} Sep 11 08:01:25.960789 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:25.960770 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:01:26.106451 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:26.106377 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-ccb84d448-fxzbx"] Sep 11 08:01:26.963126 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:26.963088 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5n7lg" event={"ID":"e910d42e-6726-46ca-978a-e687d028af81","Type":"ContainerStarted","Data":"039823765f7cd2090fa65879d4d58b0697b6c5a2b763595cf162e63ad230ec4f"} Sep 11 08:01:26.963603 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:26.963154 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5n7lg" event={"ID":"e910d42e-6726-46ca-978a-e687d028af81","Type":"ContainerStarted","Data":"0125a0aa202fc9be155ccae53eadd46f053511df12726dd48bef353a32d649b0"} Sep 11 08:01:26.965423 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:26.965394 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct" event={"ID":"6dcaa449-e314-4d02-ac8f-de897a090057","Type":"ContainerStarted","Data":"39721140041f7092a34bc8e5b554fc51174444d31264e7d79cab72a855ad4ca3"} Sep 11 08:01:26.992874 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:26.992832 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5n7lg" podStartSLOduration=2.681818065 podStartE2EDuration="3.992799918s" totalImagesPullingTime="1.242303798s" totalInitContainerRuntime="68.678055ms" isStatefulPod=true podCreationTimestamp="2026-09-11 08:01:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:01:26.991666371 +0000 UTC m=+123.997384028" watchObservedRunningTime="2026-09-11 08:01:26.992799918 +0000 UTC m=+123.998517604" Sep 11 08:01:27.013972 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:27.013935 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wn6ct" podStartSLOduration=2.798740948 podStartE2EDuration="4.013922749s" totalImagesPullingTime="1.215181801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:01:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:01:27.012994233 +0000 UTC m=+124.018711890" watchObservedRunningTime="2026-09-11 08:01:27.013922749 +0000 UTC m=+124.019640398" Sep 11 08:01:27.376163 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:27.376093 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6777dc96c4-2qdpb"] Sep 11 08:01:28.003373 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:28.003341 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-p8lmw"] Sep 11 08:01:28.006454 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:28.006438 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-p8lmw" Sep 11 08:01:28.008942 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:28.008913 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-tfpmb\"" Sep 11 08:01:28.009043 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:28.009026 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 11 08:01:28.024117 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:28.024095 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-p8lmw"] Sep 11 08:01:28.111095 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:28.111064 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhfwl\" (UniqueName: \"kubernetes.io/projected/c391e6df-9648-424a-bb7e-2fca75511930-kube-api-access-jhfwl\") pod \"monitoring-plugin-5d85778b9d-p8lmw\" (UID: \"c391e6df-9648-424a-bb7e-2fca75511930\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-p8lmw" Sep 11 08:01:28.111263 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:28.111236 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c391e6df-9648-424a-bb7e-2fca75511930-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-p8lmw\" (UID: \"c391e6df-9648-424a-bb7e-2fca75511930\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-p8lmw" Sep 11 08:01:28.211651 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:28.211625 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c391e6df-9648-424a-bb7e-2fca75511930-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-p8lmw\" (UID: \"c391e6df-9648-424a-bb7e-2fca75511930\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-p8lmw" Sep 11 08:01:28.211735 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:28.211673 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhfwl\" (UniqueName: \"kubernetes.io/projected/c391e6df-9648-424a-bb7e-2fca75511930-kube-api-access-jhfwl\") pod \"monitoring-plugin-5d85778b9d-p8lmw\" (UID: \"c391e6df-9648-424a-bb7e-2fca75511930\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-p8lmw" Sep 11 08:01:28.214023 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:28.214005 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c391e6df-9648-424a-bb7e-2fca75511930-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-p8lmw\" (UID: \"c391e6df-9648-424a-bb7e-2fca75511930\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-p8lmw" Sep 11 08:01:28.228198 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:28.228178 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhfwl\" (UniqueName: \"kubernetes.io/projected/c391e6df-9648-424a-bb7e-2fca75511930-kube-api-access-jhfwl\") pod \"monitoring-plugin-5d85778b9d-p8lmw\" (UID: \"c391e6df-9648-424a-bb7e-2fca75511930\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-p8lmw" Sep 11 08:01:28.314682 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:28.314637 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-p8lmw" Sep 11 08:01:28.587014 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:28.586948 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-p8lmw"] Sep 11 08:01:28.589750 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:01:28.589727 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc391e6df_9648_424a_bb7e_2fca75511930.slice/crio-5bfd9e6f9204840a76a8eb2a16cd1d5be3683940bf845f9ef20926005b055529 WatchSource:0}: Error finding container 5bfd9e6f9204840a76a8eb2a16cd1d5be3683940bf845f9ef20926005b055529: Status 404 returned error can't find the container with id 5bfd9e6f9204840a76a8eb2a16cd1d5be3683940bf845f9ef20926005b055529 Sep 11 08:01:28.975700 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:28.975669 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-p8lmw" event={"ID":"c391e6df-9648-424a-bb7e-2fca75511930","Type":"ContainerStarted","Data":"5bfd9e6f9204840a76a8eb2a16cd1d5be3683940bf845f9ef20926005b055529"} Sep 11 08:01:29.903536 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:29.903501 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-544f797775-hg8sg"] Sep 11 08:01:29.906661 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:29.906636 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:29.919535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:29.919507 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-544f797775-hg8sg"] Sep 11 08:01:30.024060 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.024028 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-oauth-serving-cert\") pod \"console-544f797775-hg8sg\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.024337 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.024085 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-service-ca\") pod \"console-544f797775-hg8sg\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.024337 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.024112 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-trusted-ca-bundle\") pod \"console-544f797775-hg8sg\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.024337 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.024180 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c985419-504c-4712-9cc7-eafdee18f9ae-console-serving-cert\") pod \"console-544f797775-hg8sg\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.024337 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.024202 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c985419-504c-4712-9cc7-eafdee18f9ae-console-oauth-config\") pod \"console-544f797775-hg8sg\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.024337 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.024244 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv9m4\" (UniqueName: \"kubernetes.io/projected/1c985419-504c-4712-9cc7-eafdee18f9ae-kube-api-access-xv9m4\") pod \"console-544f797775-hg8sg\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.024337 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.024272 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-console-config\") pod \"console-544f797775-hg8sg\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.125463 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.125422 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-oauth-serving-cert\") pod \"console-544f797775-hg8sg\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.125592 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.125481 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-service-ca\") pod \"console-544f797775-hg8sg\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.125592 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.125507 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-trusted-ca-bundle\") pod \"console-544f797775-hg8sg\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.125592 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.125542 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c985419-504c-4712-9cc7-eafdee18f9ae-console-serving-cert\") pod \"console-544f797775-hg8sg\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.125592 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.125563 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c985419-504c-4712-9cc7-eafdee18f9ae-console-oauth-config\") pod \"console-544f797775-hg8sg\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.125797 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.125602 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv9m4\" (UniqueName: \"kubernetes.io/projected/1c985419-504c-4712-9cc7-eafdee18f9ae-kube-api-access-xv9m4\") pod \"console-544f797775-hg8sg\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.125797 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.125632 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-console-config\") pod \"console-544f797775-hg8sg\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.126283 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.126252 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-oauth-serving-cert\") pod \"console-544f797775-hg8sg\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.126381 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.126252 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-service-ca\") pod \"console-544f797775-hg8sg\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.126441 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.126378 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-console-config\") pod \"console-544f797775-hg8sg\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.126772 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.126748 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-trusted-ca-bundle\") pod \"console-544f797775-hg8sg\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.129103 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.128635 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c985419-504c-4712-9cc7-eafdee18f9ae-console-serving-cert\") pod \"console-544f797775-hg8sg\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.129518 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.129492 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c985419-504c-4712-9cc7-eafdee18f9ae-console-oauth-config\") pod \"console-544f797775-hg8sg\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.137232 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.137208 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv9m4\" (UniqueName: \"kubernetes.io/projected/1c985419-504c-4712-9cc7-eafdee18f9ae-kube-api-access-xv9m4\") pod \"console-544f797775-hg8sg\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.217756 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.217700 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:30.509333 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.509299 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-544f797775-hg8sg"] Sep 11 08:01:30.624407 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:01:30.624366 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c985419_504c_4712_9cc7_eafdee18f9ae.slice/crio-67798048deaaef491efb962e59e44e2b5cfe31910d7f7bd284c83fec2e1cb0c5 WatchSource:0}: Error finding container 67798048deaaef491efb962e59e44e2b5cfe31910d7f7bd284c83fec2e1cb0c5: Status 404 returned error can't find the container with id 67798048deaaef491efb962e59e44e2b5cfe31910d7f7bd284c83fec2e1cb0c5 Sep 11 08:01:30.985324 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.985293 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-544f797775-hg8sg" event={"ID":"1c985419-504c-4712-9cc7-eafdee18f9ae","Type":"ContainerStarted","Data":"a04824e0e6b1d841c37dc9a2920a3fbee51033864b393deab8d4cbebc064b7cb"} Sep 11 08:01:30.985741 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.985346 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-544f797775-hg8sg" event={"ID":"1c985419-504c-4712-9cc7-eafdee18f9ae","Type":"ContainerStarted","Data":"67798048deaaef491efb962e59e44e2b5cfe31910d7f7bd284c83fec2e1cb0c5"} Sep 11 08:01:30.986802 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.986781 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-p8lmw" event={"ID":"c391e6df-9648-424a-bb7e-2fca75511930","Type":"ContainerStarted","Data":"f9b09a4e51a1584329aac440e043c67140e99552b2d3491af3c57b99926730fd"} Sep 11 08:01:30.987004 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.986985 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-p8lmw" Sep 11 08:01:30.991380 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:30.991362 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-p8lmw" Sep 11 08:01:31.032122 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:31.032084 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-p8lmw" podStartSLOduration=1.9528440489999999 podStartE2EDuration="4.032071032s" totalImagesPullingTime="2.079226983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:01:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:01:31.031612602 +0000 UTC m=+128.037330273" watchObservedRunningTime="2026-09-11 08:01:31.032071032 +0000 UTC m=+128.037788690" Sep 11 08:01:31.032953 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:31.032925 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-544f797775-hg8sg" podStartSLOduration=2.032915576 podStartE2EDuration="2.032915576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:01:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:01:31.013959084 +0000 UTC m=+128.019676742" watchObservedRunningTime="2026-09-11 08:01:31.032915576 +0000 UTC m=+128.038633224" Sep 11 08:01:33.348250 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:33.348203 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs\") pod \"network-metrics-daemon-mkcn8\" (UID: \"c5234baf-5b8f-46ff-9fa3-f217c0087ded\") " pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 08:01:33.350659 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:33.350627 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5234baf-5b8f-46ff-9fa3-f217c0087ded-metrics-certs\") pod \"network-metrics-daemon-mkcn8\" (UID: \"c5234baf-5b8f-46ff-9fa3-f217c0087ded\") " pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 08:01:33.514394 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:33.514370 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mdzhs\"" Sep 11 08:01:33.522148 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:33.522128 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkcn8" Sep 11 08:01:33.805359 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:33.805331 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mkcn8"] Sep 11 08:01:33.806967 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:01:33.806934 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5234baf_5b8f_46ff_9fa3_f217c0087ded.slice/crio-117514c63439ba6954637a6c2c737c35fad40ac5232f87fa328d52c3979cd997 WatchSource:0}: Error finding container 117514c63439ba6954637a6c2c737c35fad40ac5232f87fa328d52c3979cd997: Status 404 returned error can't find the container with id 117514c63439ba6954637a6c2c737c35fad40ac5232f87fa328d52c3979cd997 Sep 11 08:01:33.999950 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:33.999919 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mkcn8" event={"ID":"c5234baf-5b8f-46ff-9fa3-f217c0087ded","Type":"ContainerStarted","Data":"117514c63439ba6954637a6c2c737c35fad40ac5232f87fa328d52c3979cd997"} Sep 11 08:01:36.011781 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:36.011741 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mkcn8" event={"ID":"c5234baf-5b8f-46ff-9fa3-f217c0087ded","Type":"ContainerStarted","Data":"e896215d7e689dcee538e8b2c359d706e06777ddc7c08a4619d8b663ddd02208"} Sep 11 08:01:36.012157 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:36.011837 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mkcn8" event={"ID":"c5234baf-5b8f-46ff-9fa3-f217c0087ded","Type":"ContainerStarted","Data":"b5ee5826dbf2032b222fff3b35d987f4616b6f0a354e9117c80178bd113f073a"} Sep 11 08:01:36.034816 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:36.034765 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mkcn8" podStartSLOduration=130.50450459 podStartE2EDuration="2m12.034751486s" totalImagesPullingTime="1.530246896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 07:59:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:01:36.033298936 +0000 UTC m=+133.039016606" watchObservedRunningTime="2026-09-11 08:01:36.034751486 +0000 UTC m=+133.040469144" Sep 11 08:01:40.218183 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:40.218144 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:40.218663 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:40.218207 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:40.223114 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:40.223086 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:41.036601 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:41.036568 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:01:41.134798 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:41.134766 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5844bd5f5-xk6lm"] Sep 11 08:01:51.126994 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:51.126954 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-ccb84d448-fxzbx" podUID="c17f4ec2-c74a-4227-ac55-a4dfc6b290e1" containerName="console" containerID="cri-o://fd73ae1acec0643836c64f32ec94fe6d5ab52bd71828377d9d42ec8027e6a971" gracePeriod=15 Sep 11 08:01:51.391510 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:51.391485 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:51.473057 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:51.473035 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-console-oauth-config\") pod \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " Sep 11 08:01:51.473186 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:51.473069 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rrdc\" (UniqueName: \"kubernetes.io/projected/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-kube-api-access-5rrdc\") pod \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " Sep 11 08:01:51.473186 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:51.473088 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-oauth-serving-cert\") pod \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " Sep 11 08:01:51.473186 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:51.473117 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-service-ca\") pod \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " Sep 11 08:01:51.473346 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:51.473287 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-console-serving-cert\") pod \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " Sep 11 08:01:51.473346 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:51.473333 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-console-config\") pod \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\" (UID: \"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1\") " Sep 11 08:01:51.473469 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:51.473449 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-service-ca" (OuterVolumeSpecName: "service-ca") pod "c17f4ec2-c74a-4227-ac55-a4dfc6b290e1" (UID: "c17f4ec2-c74a-4227-ac55-a4dfc6b290e1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:01:51.473525 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:51.473490 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c17f4ec2-c74a-4227-ac55-a4dfc6b290e1" (UID: "c17f4ec2-c74a-4227-ac55-a4dfc6b290e1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:01:51.473699 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:51.473671 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-oauth-serving-cert\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:01:51.473699 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:51.473694 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-service-ca\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:01:51.473865 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:51.473746 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-console-config" (OuterVolumeSpecName: "console-config") pod "c17f4ec2-c74a-4227-ac55-a4dfc6b290e1" (UID: "c17f4ec2-c74a-4227-ac55-a4dfc6b290e1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:01:51.475317 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:51.475287 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c17f4ec2-c74a-4227-ac55-a4dfc6b290e1" (UID: "c17f4ec2-c74a-4227-ac55-a4dfc6b290e1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:01:51.475408 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:51.475333 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c17f4ec2-c74a-4227-ac55-a4dfc6b290e1" (UID: "c17f4ec2-c74a-4227-ac55-a4dfc6b290e1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:01:51.475408 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:51.475340 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-kube-api-access-5rrdc" (OuterVolumeSpecName: "kube-api-access-5rrdc") pod "c17f4ec2-c74a-4227-ac55-a4dfc6b290e1" (UID: "c17f4ec2-c74a-4227-ac55-a4dfc6b290e1"). InnerVolumeSpecName "kube-api-access-5rrdc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:01:51.574196 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:51.574173 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-console-oauth-config\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:01:51.574196 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:51.574193 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rrdc\" (UniqueName: \"kubernetes.io/projected/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-kube-api-access-5rrdc\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:01:51.574328 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:51.574202 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-console-serving-cert\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:01:51.574328 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:51.574211 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1-console-config\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:01:52.083665 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.083622 2557 generic.go:356] "Generic (PLEG): container finished" podID="c17f4ec2-c74a-4227-ac55-a4dfc6b290e1" containerID="fd73ae1acec0643836c64f32ec94fe6d5ab52bd71828377d9d42ec8027e6a971" exitCode=0 Sep 11 08:01:52.083817 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.083689 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ccb84d448-fxzbx" Sep 11 08:01:52.083817 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.083683 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ccb84d448-fxzbx" event={"ID":"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1","Type":"ContainerDied","Data":"fd73ae1acec0643836c64f32ec94fe6d5ab52bd71828377d9d42ec8027e6a971"} Sep 11 08:01:52.083927 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.083840 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ccb84d448-fxzbx" event={"ID":"c17f4ec2-c74a-4227-ac55-a4dfc6b290e1","Type":"ContainerDied","Data":"fb7268b4df421b8f771bc6a68e97cdc5d2c7ac951b1b962b333c4590aa2cd910"} Sep 11 08:01:52.083927 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.083874 2557 scope.go:117] "RemoveContainer" containerID="fd73ae1acec0643836c64f32ec94fe6d5ab52bd71828377d9d42ec8027e6a971" Sep 11 08:01:52.092204 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.092184 2557 scope.go:117] "RemoveContainer" containerID="fd73ae1acec0643836c64f32ec94fe6d5ab52bd71828377d9d42ec8027e6a971" Sep 11 08:01:52.092498 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:01:52.092477 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd73ae1acec0643836c64f32ec94fe6d5ab52bd71828377d9d42ec8027e6a971\": container with ID starting with fd73ae1acec0643836c64f32ec94fe6d5ab52bd71828377d9d42ec8027e6a971 not found: ID does not exist" containerID="fd73ae1acec0643836c64f32ec94fe6d5ab52bd71828377d9d42ec8027e6a971" Sep 11 08:01:52.092567 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.092510 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd73ae1acec0643836c64f32ec94fe6d5ab52bd71828377d9d42ec8027e6a971"} err="failed to get container status \"fd73ae1acec0643836c64f32ec94fe6d5ab52bd71828377d9d42ec8027e6a971\": rpc error: code = NotFound desc = could not find container \"fd73ae1acec0643836c64f32ec94fe6d5ab52bd71828377d9d42ec8027e6a971\": container with ID starting with fd73ae1acec0643836c64f32ec94fe6d5ab52bd71828377d9d42ec8027e6a971 not found: ID does not exist" Sep 11 08:01:52.104010 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.103986 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-ccb84d448-fxzbx"] Sep 11 08:01:52.110730 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.110706 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-ccb84d448-fxzbx"] Sep 11 08:01:52.394685 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.394653 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" podUID="f8fd7bec-f0a5-43a7-b8f5-59401860b502" containerName="registry" containerID="cri-o://7a18806c45930a6758aec89c436dfb922cf838637994af16f243d705eca9974a" gracePeriod=30 Sep 11 08:01:52.634881 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.634859 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 08:01:52.685471 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.685418 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8fd7bec-f0a5-43a7-b8f5-59401860b502-installation-pull-secrets\") pod \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " Sep 11 08:01:52.685471 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.685447 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8fd7bec-f0a5-43a7-b8f5-59401860b502-ca-trust-extracted\") pod \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " Sep 11 08:01:52.685471 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.685462 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-bound-sa-token\") pod \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " Sep 11 08:01:52.685653 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.685499 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls\") pod \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " Sep 11 08:01:52.685653 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.685514 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8fd7bec-f0a5-43a7-b8f5-59401860b502-trusted-ca\") pod \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " Sep 11 08:01:52.685653 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.685540 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-certificates\") pod \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " Sep 11 08:01:52.685653 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.685563 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8fd7bec-f0a5-43a7-b8f5-59401860b502-image-registry-private-configuration\") pod \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " Sep 11 08:01:52.685653 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.685592 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lm9qd\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-kube-api-access-lm9qd\") pod \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\" (UID: \"f8fd7bec-f0a5-43a7-b8f5-59401860b502\") " Sep 11 08:01:52.686190 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.686125 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8fd7bec-f0a5-43a7-b8f5-59401860b502-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f8fd7bec-f0a5-43a7-b8f5-59401860b502" (UID: "f8fd7bec-f0a5-43a7-b8f5-59401860b502"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:01:52.686303 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.686216 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f8fd7bec-f0a5-43a7-b8f5-59401860b502" (UID: "f8fd7bec-f0a5-43a7-b8f5-59401860b502"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:01:52.688058 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.688016 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8fd7bec-f0a5-43a7-b8f5-59401860b502-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f8fd7bec-f0a5-43a7-b8f5-59401860b502" (UID: "f8fd7bec-f0a5-43a7-b8f5-59401860b502"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:01:52.688165 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.688117 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8fd7bec-f0a5-43a7-b8f5-59401860b502-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f8fd7bec-f0a5-43a7-b8f5-59401860b502" (UID: "f8fd7bec-f0a5-43a7-b8f5-59401860b502"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:01:52.688235 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.688170 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-kube-api-access-lm9qd" (OuterVolumeSpecName: "kube-api-access-lm9qd") pod "f8fd7bec-f0a5-43a7-b8f5-59401860b502" (UID: "f8fd7bec-f0a5-43a7-b8f5-59401860b502"). InnerVolumeSpecName "kube-api-access-lm9qd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:01:52.688292 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.688239 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f8fd7bec-f0a5-43a7-b8f5-59401860b502" (UID: "f8fd7bec-f0a5-43a7-b8f5-59401860b502"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:01:52.688292 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.688239 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f8fd7bec-f0a5-43a7-b8f5-59401860b502" (UID: "f8fd7bec-f0a5-43a7-b8f5-59401860b502"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:01:52.693967 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.693941 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8fd7bec-f0a5-43a7-b8f5-59401860b502-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f8fd7bec-f0a5-43a7-b8f5-59401860b502" (UID: "f8fd7bec-f0a5-43a7-b8f5-59401860b502"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 08:01:52.786287 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.786255 2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-certificates\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:01:52.786287 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.786285 2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8fd7bec-f0a5-43a7-b8f5-59401860b502-image-registry-private-configuration\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:01:52.786287 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.786296 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lm9qd\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-kube-api-access-lm9qd\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:01:52.786440 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.786307 2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8fd7bec-f0a5-43a7-b8f5-59401860b502-installation-pull-secrets\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:01:52.786440 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.786316 2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8fd7bec-f0a5-43a7-b8f5-59401860b502-ca-trust-extracted\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:01:52.786440 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.786324 2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-bound-sa-token\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:01:52.786440 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.786332 2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8fd7bec-f0a5-43a7-b8f5-59401860b502-registry-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:01:52.786440 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:52.786339 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8fd7bec-f0a5-43a7-b8f5-59401860b502-trusted-ca\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:01:53.090062 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:53.090010 2557 generic.go:356] "Generic (PLEG): container finished" podID="f8fd7bec-f0a5-43a7-b8f5-59401860b502" containerID="7a18806c45930a6758aec89c436dfb922cf838637994af16f243d705eca9974a" exitCode=0 Sep 11 08:01:53.090062 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:53.090056 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" Sep 11 08:01:53.090195 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:53.090069 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" event={"ID":"f8fd7bec-f0a5-43a7-b8f5-59401860b502","Type":"ContainerDied","Data":"7a18806c45930a6758aec89c436dfb922cf838637994af16f243d705eca9974a"} Sep 11 08:01:53.090195 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:53.090105 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6777dc96c4-2qdpb" event={"ID":"f8fd7bec-f0a5-43a7-b8f5-59401860b502","Type":"ContainerDied","Data":"01bf483041a389998dd057db069c0fbc5f310b1466b022cf620254bad39ea74b"} Sep 11 08:01:53.090195 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:53.090124 2557 scope.go:117] "RemoveContainer" containerID="7a18806c45930a6758aec89c436dfb922cf838637994af16f243d705eca9974a" Sep 11 08:01:53.098692 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:53.098675 2557 scope.go:117] "RemoveContainer" containerID="7a18806c45930a6758aec89c436dfb922cf838637994af16f243d705eca9974a" Sep 11 08:01:53.098972 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:01:53.098954 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a18806c45930a6758aec89c436dfb922cf838637994af16f243d705eca9974a\": container with ID starting with 7a18806c45930a6758aec89c436dfb922cf838637994af16f243d705eca9974a not found: ID does not exist" containerID="7a18806c45930a6758aec89c436dfb922cf838637994af16f243d705eca9974a" Sep 11 08:01:53.099022 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:53.098980 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a18806c45930a6758aec89c436dfb922cf838637994af16f243d705eca9974a"} err="failed to get container status \"7a18806c45930a6758aec89c436dfb922cf838637994af16f243d705eca9974a\": rpc error: code = NotFound desc = could not find container \"7a18806c45930a6758aec89c436dfb922cf838637994af16f243d705eca9974a\": container with ID starting with 7a18806c45930a6758aec89c436dfb922cf838637994af16f243d705eca9974a not found: ID does not exist" Sep 11 08:01:53.115720 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:53.115699 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6777dc96c4-2qdpb"] Sep 11 08:01:53.123665 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:53.123643 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6777dc96c4-2qdpb"] Sep 11 08:01:53.501834 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:53.501810 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c17f4ec2-c74a-4227-ac55-a4dfc6b290e1" path="/var/lib/kubelet/pods/c17f4ec2-c74a-4227-ac55-a4dfc6b290e1/volumes" Sep 11 08:01:53.502223 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:01:53.502209 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8fd7bec-f0a5-43a7-b8f5-59401860b502" path="/var/lib/kubelet/pods/f8fd7bec-f0a5-43a7-b8f5-59401860b502/volumes" Sep 11 08:02:06.154139 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.154072 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5844bd5f5-xk6lm" podUID="562bcd6d-2e72-4be9-9d98-893487decb92" containerName="console" containerID="cri-o://29c30a87597666fba0545c5f7628b7d5d87c8c8b1e41290f0b195686c777fe65" gracePeriod=15 Sep 11 08:02:06.405929 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.405862 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:02:06.477939 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.477914 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/562bcd6d-2e72-4be9-9d98-893487decb92-console-serving-cert\") pod \"562bcd6d-2e72-4be9-9d98-893487decb92\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " Sep 11 08:02:06.478038 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.477948 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6t5fc\" (UniqueName: \"kubernetes.io/projected/562bcd6d-2e72-4be9-9d98-893487decb92-kube-api-access-6t5fc\") pod \"562bcd6d-2e72-4be9-9d98-893487decb92\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " Sep 11 08:02:06.478038 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.477970 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-trusted-ca-bundle\") pod \"562bcd6d-2e72-4be9-9d98-893487decb92\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " Sep 11 08:02:06.478038 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.477994 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-oauth-serving-cert\") pod \"562bcd6d-2e72-4be9-9d98-893487decb92\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " Sep 11 08:02:06.478038 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.478028 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/562bcd6d-2e72-4be9-9d98-893487decb92-console-oauth-config\") pod \"562bcd6d-2e72-4be9-9d98-893487decb92\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " Sep 11 08:02:06.478223 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.478050 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-service-ca\") pod \"562bcd6d-2e72-4be9-9d98-893487decb92\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " Sep 11 08:02:06.478223 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.478079 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-console-config\") pod \"562bcd6d-2e72-4be9-9d98-893487decb92\" (UID: \"562bcd6d-2e72-4be9-9d98-893487decb92\") " Sep 11 08:02:06.478482 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.478434 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "562bcd6d-2e72-4be9-9d98-893487decb92" (UID: "562bcd6d-2e72-4be9-9d98-893487decb92"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:02:06.478482 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.478465 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "562bcd6d-2e72-4be9-9d98-893487decb92" (UID: "562bcd6d-2e72-4be9-9d98-893487decb92"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:02:06.478642 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.478474 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-service-ca" (OuterVolumeSpecName: "service-ca") pod "562bcd6d-2e72-4be9-9d98-893487decb92" (UID: "562bcd6d-2e72-4be9-9d98-893487decb92"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:02:06.478642 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.478551 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-console-config" (OuterVolumeSpecName: "console-config") pod "562bcd6d-2e72-4be9-9d98-893487decb92" (UID: "562bcd6d-2e72-4be9-9d98-893487decb92"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:02:06.480045 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.480022 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/562bcd6d-2e72-4be9-9d98-893487decb92-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "562bcd6d-2e72-4be9-9d98-893487decb92" (UID: "562bcd6d-2e72-4be9-9d98-893487decb92"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:02:06.480134 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.480099 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/562bcd6d-2e72-4be9-9d98-893487decb92-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "562bcd6d-2e72-4be9-9d98-893487decb92" (UID: "562bcd6d-2e72-4be9-9d98-893487decb92"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:02:06.480276 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.480257 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/562bcd6d-2e72-4be9-9d98-893487decb92-kube-api-access-6t5fc" (OuterVolumeSpecName: "kube-api-access-6t5fc") pod "562bcd6d-2e72-4be9-9d98-893487decb92" (UID: "562bcd6d-2e72-4be9-9d98-893487decb92"). InnerVolumeSpecName "kube-api-access-6t5fc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:02:06.579089 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.579056 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-console-config\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:02:06.579089 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.579087 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/562bcd6d-2e72-4be9-9d98-893487decb92-console-serving-cert\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:02:06.579089 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.579097 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6t5fc\" (UniqueName: \"kubernetes.io/projected/562bcd6d-2e72-4be9-9d98-893487decb92-kube-api-access-6t5fc\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:02:06.579232 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.579107 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-trusted-ca-bundle\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:02:06.579232 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.579115 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-oauth-serving-cert\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:02:06.579232 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.579123 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/562bcd6d-2e72-4be9-9d98-893487decb92-console-oauth-config\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:02:06.579232 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:06.579132 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/562bcd6d-2e72-4be9-9d98-893487decb92-service-ca\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:02:07.148109 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:07.148078 2557 generic.go:356] "Generic (PLEG): container finished" podID="562bcd6d-2e72-4be9-9d98-893487decb92" containerID="29c30a87597666fba0545c5f7628b7d5d87c8c8b1e41290f0b195686c777fe65" exitCode=0 Sep 11 08:02:07.148251 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:07.148126 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5844bd5f5-xk6lm" Sep 11 08:02:07.148251 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:07.148134 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5844bd5f5-xk6lm" event={"ID":"562bcd6d-2e72-4be9-9d98-893487decb92","Type":"ContainerDied","Data":"29c30a87597666fba0545c5f7628b7d5d87c8c8b1e41290f0b195686c777fe65"} Sep 11 08:02:07.148251 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:07.148173 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5844bd5f5-xk6lm" event={"ID":"562bcd6d-2e72-4be9-9d98-893487decb92","Type":"ContainerDied","Data":"e3f7c0cb831045d25c412b342d503fc4c92af2bfceea53fd87542acb389b0e1b"} Sep 11 08:02:07.148251 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:07.148193 2557 scope.go:117] "RemoveContainer" containerID="29c30a87597666fba0545c5f7628b7d5d87c8c8b1e41290f0b195686c777fe65" Sep 11 08:02:07.157327 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:07.157172 2557 scope.go:117] "RemoveContainer" containerID="29c30a87597666fba0545c5f7628b7d5d87c8c8b1e41290f0b195686c777fe65" Sep 11 08:02:07.157571 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:02:07.157482 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29c30a87597666fba0545c5f7628b7d5d87c8c8b1e41290f0b195686c777fe65\": container with ID starting with 29c30a87597666fba0545c5f7628b7d5d87c8c8b1e41290f0b195686c777fe65 not found: ID does not exist" containerID="29c30a87597666fba0545c5f7628b7d5d87c8c8b1e41290f0b195686c777fe65" Sep 11 08:02:07.157571 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:07.157504 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c30a87597666fba0545c5f7628b7d5d87c8c8b1e41290f0b195686c777fe65"} err="failed to get container status \"29c30a87597666fba0545c5f7628b7d5d87c8c8b1e41290f0b195686c777fe65\": rpc error: code = NotFound desc = could not find container \"29c30a87597666fba0545c5f7628b7d5d87c8c8b1e41290f0b195686c777fe65\": container with ID starting with 29c30a87597666fba0545c5f7628b7d5d87c8c8b1e41290f0b195686c777fe65 not found: ID does not exist" Sep 11 08:02:07.172013 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:07.170215 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5844bd5f5-xk6lm"] Sep 11 08:02:07.175950 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:07.175926 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5844bd5f5-xk6lm"] Sep 11 08:02:07.502363 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:07.502294 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="562bcd6d-2e72-4be9-9d98-893487decb92" path="/var/lib/kubelet/pods/562bcd6d-2e72-4be9-9d98-893487decb92/volumes" Sep 11 08:02:50.974937 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:50.974890 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77b5d6989b-h8sc4"] Sep 11 08:02:50.975489 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:50.975269 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c17f4ec2-c74a-4227-ac55-a4dfc6b290e1" containerName="console" Sep 11 08:02:50.975489 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:50.975286 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c17f4ec2-c74a-4227-ac55-a4dfc6b290e1" containerName="console" Sep 11 08:02:50.975489 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:50.975299 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8fd7bec-f0a5-43a7-b8f5-59401860b502" containerName="registry" Sep 11 08:02:50.975489 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:50.975308 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8fd7bec-f0a5-43a7-b8f5-59401860b502" containerName="registry" Sep 11 08:02:50.975489 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:50.975331 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562bcd6d-2e72-4be9-9d98-893487decb92" containerName="console" Sep 11 08:02:50.975489 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:50.975338 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="562bcd6d-2e72-4be9-9d98-893487decb92" containerName="console" Sep 11 08:02:50.975489 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:50.975406 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="562bcd6d-2e72-4be9-9d98-893487decb92" containerName="console" Sep 11 08:02:50.975489 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:50.975416 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c17f4ec2-c74a-4227-ac55-a4dfc6b290e1" containerName="console" Sep 11 08:02:50.975489 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:50.975427 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8fd7bec-f0a5-43a7-b8f5-59401860b502" containerName="registry" Sep 11 08:02:50.978379 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:50.978360 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:50.988893 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:50.988870 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b5d6989b-h8sc4"] Sep 11 08:02:51.089764 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.089739 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36a5b75b-a454-45c5-81cb-4b5faf84565e-console-serving-cert\") pod \"console-77b5d6989b-h8sc4\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.089926 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.089767 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36a5b75b-a454-45c5-81cb-4b5faf84565e-console-oauth-config\") pod \"console-77b5d6989b-h8sc4\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.089926 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.089788 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-console-config\") pod \"console-77b5d6989b-h8sc4\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.089926 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.089813 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-trusted-ca-bundle\") pod \"console-77b5d6989b-h8sc4\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.089926 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.089856 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkhcq\" (UniqueName: \"kubernetes.io/projected/36a5b75b-a454-45c5-81cb-4b5faf84565e-kube-api-access-xkhcq\") pod \"console-77b5d6989b-h8sc4\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.089926 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.089912 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-service-ca\") pod \"console-77b5d6989b-h8sc4\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.089926 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.089928 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-oauth-serving-cert\") pod \"console-77b5d6989b-h8sc4\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.190514 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.190484 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36a5b75b-a454-45c5-81cb-4b5faf84565e-console-serving-cert\") pod \"console-77b5d6989b-h8sc4\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.190661 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.190519 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36a5b75b-a454-45c5-81cb-4b5faf84565e-console-oauth-config\") pod \"console-77b5d6989b-h8sc4\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.190661 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.190540 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-console-config\") pod \"console-77b5d6989b-h8sc4\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.190661 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.190561 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-trusted-ca-bundle\") pod \"console-77b5d6989b-h8sc4\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.190661 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.190578 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkhcq\" (UniqueName: \"kubernetes.io/projected/36a5b75b-a454-45c5-81cb-4b5faf84565e-kube-api-access-xkhcq\") pod \"console-77b5d6989b-h8sc4\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.190661 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.190604 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-service-ca\") pod \"console-77b5d6989b-h8sc4\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.190661 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.190626 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-oauth-serving-cert\") pod \"console-77b5d6989b-h8sc4\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.191429 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.191401 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-oauth-serving-cert\") pod \"console-77b5d6989b-h8sc4\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.191543 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.191445 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-service-ca\") pod \"console-77b5d6989b-h8sc4\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.191543 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.191530 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-trusted-ca-bundle\") pod \"console-77b5d6989b-h8sc4\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.191962 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.191937 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-console-config\") pod \"console-77b5d6989b-h8sc4\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.193203 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.193183 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36a5b75b-a454-45c5-81cb-4b5faf84565e-console-serving-cert\") pod \"console-77b5d6989b-h8sc4\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.193497 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.193471 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36a5b75b-a454-45c5-81cb-4b5faf84565e-console-oauth-config\") pod \"console-77b5d6989b-h8sc4\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.200268 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.200222 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkhcq\" (UniqueName: \"kubernetes.io/projected/36a5b75b-a454-45c5-81cb-4b5faf84565e-kube-api-access-xkhcq\") pod \"console-77b5d6989b-h8sc4\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.287116 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.287054 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:02:51.572765 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:51.572685 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b5d6989b-h8sc4"] Sep 11 08:02:51.576527 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:02:51.576503 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36a5b75b_a454_45c5_81cb_4b5faf84565e.slice/crio-d53e3af839ec632bb57c8f6a4fe43ef2a9c5bf7f5ba2964ff7bc0dfe0f1a21e5 WatchSource:0}: Error finding container d53e3af839ec632bb57c8f6a4fe43ef2a9c5bf7f5ba2964ff7bc0dfe0f1a21e5: Status 404 returned error can't find the container with id d53e3af839ec632bb57c8f6a4fe43ef2a9c5bf7f5ba2964ff7bc0dfe0f1a21e5 Sep 11 08:02:52.321229 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:52.321189 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b5d6989b-h8sc4" event={"ID":"36a5b75b-a454-45c5-81cb-4b5faf84565e","Type":"ContainerStarted","Data":"44d5cc8d5a14417907c7a63d9b37b9e5de6d00774ce751919fd6aea8d02c9f63"} Sep 11 08:02:52.321631 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:52.321236 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b5d6989b-h8sc4" event={"ID":"36a5b75b-a454-45c5-81cb-4b5faf84565e","Type":"ContainerStarted","Data":"d53e3af839ec632bb57c8f6a4fe43ef2a9c5bf7f5ba2964ff7bc0dfe0f1a21e5"} Sep 11 08:02:52.340530 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:02:52.340481 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77b5d6989b-h8sc4" podStartSLOduration=2.340465667 podStartE2EDuration="2.340465667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:02:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:02:52.33909497 +0000 UTC m=+209.344812625" watchObservedRunningTime="2026-09-11 08:02:52.340465667 +0000 UTC m=+209.346183325" Sep 11 08:03:01.287570 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:01.287533 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:03:01.287970 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:01.287614 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:03:01.293070 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:01.293041 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:03:01.359145 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:01.359126 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:03:01.423096 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:01.423059 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-544f797775-hg8sg"] Sep 11 08:03:26.442317 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.442267 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-544f797775-hg8sg" podUID="1c985419-504c-4712-9cc7-eafdee18f9ae" containerName="console" containerID="cri-o://a04824e0e6b1d841c37dc9a2920a3fbee51033864b393deab8d4cbebc064b7cb" gracePeriod=15 Sep 11 08:03:26.674301 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.674280 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:03:26.825572 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.825495 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-trusted-ca-bundle\") pod \"1c985419-504c-4712-9cc7-eafdee18f9ae\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " Sep 11 08:03:26.825572 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.825532 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c985419-504c-4712-9cc7-eafdee18f9ae-console-oauth-config\") pod \"1c985419-504c-4712-9cc7-eafdee18f9ae\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " Sep 11 08:03:26.825572 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.825563 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xv9m4\" (UniqueName: \"kubernetes.io/projected/1c985419-504c-4712-9cc7-eafdee18f9ae-kube-api-access-xv9m4\") pod \"1c985419-504c-4712-9cc7-eafdee18f9ae\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " Sep 11 08:03:26.825810 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.825595 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-oauth-serving-cert\") pod \"1c985419-504c-4712-9cc7-eafdee18f9ae\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " Sep 11 08:03:26.825810 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.825617 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-service-ca\") pod \"1c985419-504c-4712-9cc7-eafdee18f9ae\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " Sep 11 08:03:26.825810 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.825644 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-console-config\") pod \"1c985419-504c-4712-9cc7-eafdee18f9ae\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " Sep 11 08:03:26.825810 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.825680 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c985419-504c-4712-9cc7-eafdee18f9ae-console-serving-cert\") pod \"1c985419-504c-4712-9cc7-eafdee18f9ae\" (UID: \"1c985419-504c-4712-9cc7-eafdee18f9ae\") " Sep 11 08:03:26.826154 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.826121 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-service-ca" (OuterVolumeSpecName: "service-ca") pod "1c985419-504c-4712-9cc7-eafdee18f9ae" (UID: "1c985419-504c-4712-9cc7-eafdee18f9ae"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:03:26.826154 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.826132 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-console-config" (OuterVolumeSpecName: "console-config") pod "1c985419-504c-4712-9cc7-eafdee18f9ae" (UID: "1c985419-504c-4712-9cc7-eafdee18f9ae"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:03:26.826308 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.826273 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1c985419-504c-4712-9cc7-eafdee18f9ae" (UID: "1c985419-504c-4712-9cc7-eafdee18f9ae"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:03:26.826399 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.826323 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1c985419-504c-4712-9cc7-eafdee18f9ae" (UID: "1c985419-504c-4712-9cc7-eafdee18f9ae"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:03:26.827883 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.827859 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c985419-504c-4712-9cc7-eafdee18f9ae-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1c985419-504c-4712-9cc7-eafdee18f9ae" (UID: "1c985419-504c-4712-9cc7-eafdee18f9ae"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:03:26.828188 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.828167 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c985419-504c-4712-9cc7-eafdee18f9ae-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1c985419-504c-4712-9cc7-eafdee18f9ae" (UID: "1c985419-504c-4712-9cc7-eafdee18f9ae"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:03:26.828226 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.828191 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c985419-504c-4712-9cc7-eafdee18f9ae-kube-api-access-xv9m4" (OuterVolumeSpecName: "kube-api-access-xv9m4") pod "1c985419-504c-4712-9cc7-eafdee18f9ae" (UID: "1c985419-504c-4712-9cc7-eafdee18f9ae"). InnerVolumeSpecName "kube-api-access-xv9m4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:03:26.926890 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.926867 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-trusted-ca-bundle\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:03:26.926890 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.926887 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c985419-504c-4712-9cc7-eafdee18f9ae-console-oauth-config\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:03:26.927024 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.926915 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xv9m4\" (UniqueName: \"kubernetes.io/projected/1c985419-504c-4712-9cc7-eafdee18f9ae-kube-api-access-xv9m4\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:03:26.927024 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.926925 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-oauth-serving-cert\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:03:26.927024 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.926933 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-service-ca\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:03:26.927024 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.926941 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c985419-504c-4712-9cc7-eafdee18f9ae-console-config\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:03:26.927024 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:26.926950 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c985419-504c-4712-9cc7-eafdee18f9ae-console-serving-cert\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:03:27.451694 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:27.451662 2557 generic.go:356] "Generic (PLEG): container finished" podID="1c985419-504c-4712-9cc7-eafdee18f9ae" containerID="a04824e0e6b1d841c37dc9a2920a3fbee51033864b393deab8d4cbebc064b7cb" exitCode=0 Sep 11 08:03:27.452168 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:27.451705 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-544f797775-hg8sg" event={"ID":"1c985419-504c-4712-9cc7-eafdee18f9ae","Type":"ContainerDied","Data":"a04824e0e6b1d841c37dc9a2920a3fbee51033864b393deab8d4cbebc064b7cb"} Sep 11 08:03:27.452168 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:27.451724 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-544f797775-hg8sg" Sep 11 08:03:27.452168 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:27.451750 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-544f797775-hg8sg" event={"ID":"1c985419-504c-4712-9cc7-eafdee18f9ae","Type":"ContainerDied","Data":"67798048deaaef491efb962e59e44e2b5cfe31910d7f7bd284c83fec2e1cb0c5"} Sep 11 08:03:27.452168 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:27.451773 2557 scope.go:117] "RemoveContainer" containerID="a04824e0e6b1d841c37dc9a2920a3fbee51033864b393deab8d4cbebc064b7cb" Sep 11 08:03:27.461072 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:27.461053 2557 scope.go:117] "RemoveContainer" containerID="a04824e0e6b1d841c37dc9a2920a3fbee51033864b393deab8d4cbebc064b7cb" Sep 11 08:03:27.461399 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:03:27.461377 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a04824e0e6b1d841c37dc9a2920a3fbee51033864b393deab8d4cbebc064b7cb\": container with ID starting with a04824e0e6b1d841c37dc9a2920a3fbee51033864b393deab8d4cbebc064b7cb not found: ID does not exist" containerID="a04824e0e6b1d841c37dc9a2920a3fbee51033864b393deab8d4cbebc064b7cb" Sep 11 08:03:27.461456 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:27.461409 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a04824e0e6b1d841c37dc9a2920a3fbee51033864b393deab8d4cbebc064b7cb"} err="failed to get container status \"a04824e0e6b1d841c37dc9a2920a3fbee51033864b393deab8d4cbebc064b7cb\": rpc error: code = NotFound desc = could not find container \"a04824e0e6b1d841c37dc9a2920a3fbee51033864b393deab8d4cbebc064b7cb\": container with ID starting with a04824e0e6b1d841c37dc9a2920a3fbee51033864b393deab8d4cbebc064b7cb not found: ID does not exist" Sep 11 08:03:27.474156 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:27.474132 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-544f797775-hg8sg"] Sep 11 08:03:27.478713 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:27.478692 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-544f797775-hg8sg"] Sep 11 08:03:27.502000 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:27.501967 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c985419-504c-4712-9cc7-eafdee18f9ae" path="/var/lib/kubelet/pods/1c985419-504c-4712-9cc7-eafdee18f9ae/volumes" Sep 11 08:03:59.962055 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:59.962027 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f5dfcd956-zb57g"] Sep 11 08:03:59.962498 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:59.962259 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c985419-504c-4712-9cc7-eafdee18f9ae" containerName="console" Sep 11 08:03:59.962498 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:59.962269 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c985419-504c-4712-9cc7-eafdee18f9ae" containerName="console" Sep 11 08:03:59.962498 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:59.962333 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c985419-504c-4712-9cc7-eafdee18f9ae" containerName="console" Sep 11 08:03:59.965330 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:59.965313 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:03:59.995487 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:03:59.995462 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f5dfcd956-zb57g"] Sep 11 08:04:00.034528 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.034503 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42mb8\" (UniqueName: \"kubernetes.io/projected/0834f8c7-8504-4283-b59f-0e66494795f2-kube-api-access-42mb8\") pod \"console-f5dfcd956-zb57g\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.034629 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.034539 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0834f8c7-8504-4283-b59f-0e66494795f2-console-serving-cert\") pod \"console-f5dfcd956-zb57g\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.034629 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.034560 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0834f8c7-8504-4283-b59f-0e66494795f2-console-oauth-config\") pod \"console-f5dfcd956-zb57g\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.034744 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.034669 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-console-config\") pod \"console-f5dfcd956-zb57g\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.034744 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.034706 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-trusted-ca-bundle\") pod \"console-f5dfcd956-zb57g\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.034849 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.034753 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-service-ca\") pod \"console-f5dfcd956-zb57g\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.034849 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.034777 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-oauth-serving-cert\") pod \"console-f5dfcd956-zb57g\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.135840 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.135816 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-console-config\") pod \"console-f5dfcd956-zb57g\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.135963 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.135842 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-trusted-ca-bundle\") pod \"console-f5dfcd956-zb57g\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.135963 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.135865 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-service-ca\") pod \"console-f5dfcd956-zb57g\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.135963 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.135879 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-oauth-serving-cert\") pod \"console-f5dfcd956-zb57g\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.136138 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.136075 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42mb8\" (UniqueName: \"kubernetes.io/projected/0834f8c7-8504-4283-b59f-0e66494795f2-kube-api-access-42mb8\") pod \"console-f5dfcd956-zb57g\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.136138 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.136123 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0834f8c7-8504-4283-b59f-0e66494795f2-console-serving-cert\") pod \"console-f5dfcd956-zb57g\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.136244 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.136146 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0834f8c7-8504-4283-b59f-0e66494795f2-console-oauth-config\") pod \"console-f5dfcd956-zb57g\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.136627 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.136598 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-console-config\") pod \"console-f5dfcd956-zb57g\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.136743 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.136619 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-service-ca\") pod \"console-f5dfcd956-zb57g\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.136743 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.136685 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-oauth-serving-cert\") pod \"console-f5dfcd956-zb57g\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.136833 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.136767 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-trusted-ca-bundle\") pod \"console-f5dfcd956-zb57g\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.138533 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.138511 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0834f8c7-8504-4283-b59f-0e66494795f2-console-serving-cert\") pod \"console-f5dfcd956-zb57g\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.138652 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.138634 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0834f8c7-8504-4283-b59f-0e66494795f2-console-oauth-config\") pod \"console-f5dfcd956-zb57g\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.146619 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.146600 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42mb8\" (UniqueName: \"kubernetes.io/projected/0834f8c7-8504-4283-b59f-0e66494795f2-kube-api-access-42mb8\") pod \"console-f5dfcd956-zb57g\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.274976 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.274930 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:00.550361 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.550292 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f5dfcd956-zb57g"] Sep 11 08:04:00.553209 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:04:00.553184 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0834f8c7_8504_4283_b59f_0e66494795f2.slice/crio-907bcb69e2fe1a56904d7cd21464a5bb66aa8f2985893b12ef71a360cf14b2a2 WatchSource:0}: Error finding container 907bcb69e2fe1a56904d7cd21464a5bb66aa8f2985893b12ef71a360cf14b2a2: Status 404 returned error can't find the container with id 907bcb69e2fe1a56904d7cd21464a5bb66aa8f2985893b12ef71a360cf14b2a2 Sep 11 08:04:00.573810 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:00.573787 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f5dfcd956-zb57g" event={"ID":"0834f8c7-8504-4283-b59f-0e66494795f2","Type":"ContainerStarted","Data":"907bcb69e2fe1a56904d7cd21464a5bb66aa8f2985893b12ef71a360cf14b2a2"} Sep 11 08:04:01.578677 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:01.578635 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f5dfcd956-zb57g" event={"ID":"0834f8c7-8504-4283-b59f-0e66494795f2","Type":"ContainerStarted","Data":"a0be8df625331287dd936abd9458c3d6fd7b8ea78c4c5471869a341080bfa041"} Sep 11 08:04:01.600666 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:01.600619 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f5dfcd956-zb57g" podStartSLOduration=2.6006048980000003 podStartE2EDuration="2.600604898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:03:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:04:01.600125578 +0000 UTC m=+278.605843254" watchObservedRunningTime="2026-09-11 08:04:01.600604898 +0000 UTC m=+278.606322556" Sep 11 08:04:10.276008 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:10.275975 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:10.276360 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:10.276033 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:10.280843 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:10.280823 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:10.619960 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:10.619878 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:04:10.672599 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:10.672565 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77b5d6989b-h8sc4"] Sep 11 08:04:23.419922 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:23.419875 2557 kubelet.go:1648] "Image garbage collection succeeded" Sep 11 08:04:35.695748 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:35.695680 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77b5d6989b-h8sc4" podUID="36a5b75b-a454-45c5-81cb-4b5faf84565e" containerName="console" containerID="cri-o://44d5cc8d5a14417907c7a63d9b37b9e5de6d00774ce751919fd6aea8d02c9f63" gracePeriod=15 Sep 11 08:04:35.942712 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:35.942683 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:04:35.978783 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:35.978719 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36a5b75b-a454-45c5-81cb-4b5faf84565e-console-serving-cert\") pod \"36a5b75b-a454-45c5-81cb-4b5faf84565e\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " Sep 11 08:04:35.978783 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:35.978764 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkhcq\" (UniqueName: \"kubernetes.io/projected/36a5b75b-a454-45c5-81cb-4b5faf84565e-kube-api-access-xkhcq\") pod \"36a5b75b-a454-45c5-81cb-4b5faf84565e\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " Sep 11 08:04:35.978959 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:35.978818 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36a5b75b-a454-45c5-81cb-4b5faf84565e-console-oauth-config\") pod \"36a5b75b-a454-45c5-81cb-4b5faf84565e\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " Sep 11 08:04:35.978959 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:35.978857 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-oauth-serving-cert\") pod \"36a5b75b-a454-45c5-81cb-4b5faf84565e\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " Sep 11 08:04:35.978959 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:35.978914 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-service-ca\") pod \"36a5b75b-a454-45c5-81cb-4b5faf84565e\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " Sep 11 08:04:35.978959 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:35.978950 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-console-config\") pod \"36a5b75b-a454-45c5-81cb-4b5faf84565e\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " Sep 11 08:04:35.979177 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:35.978989 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-trusted-ca-bundle\") pod \"36a5b75b-a454-45c5-81cb-4b5faf84565e\" (UID: \"36a5b75b-a454-45c5-81cb-4b5faf84565e\") " Sep 11 08:04:35.979536 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:35.979505 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-console-config" (OuterVolumeSpecName: "console-config") pod "36a5b75b-a454-45c5-81cb-4b5faf84565e" (UID: "36a5b75b-a454-45c5-81cb-4b5faf84565e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:04:35.979619 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:35.979542 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "36a5b75b-a454-45c5-81cb-4b5faf84565e" (UID: "36a5b75b-a454-45c5-81cb-4b5faf84565e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:04:35.979867 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:35.979787 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "36a5b75b-a454-45c5-81cb-4b5faf84565e" (UID: "36a5b75b-a454-45c5-81cb-4b5faf84565e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:04:35.979965 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:35.979934 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-service-ca" (OuterVolumeSpecName: "service-ca") pod "36a5b75b-a454-45c5-81cb-4b5faf84565e" (UID: "36a5b75b-a454-45c5-81cb-4b5faf84565e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:04:35.980951 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:35.980921 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36a5b75b-a454-45c5-81cb-4b5faf84565e-kube-api-access-xkhcq" (OuterVolumeSpecName: "kube-api-access-xkhcq") pod "36a5b75b-a454-45c5-81cb-4b5faf84565e" (UID: "36a5b75b-a454-45c5-81cb-4b5faf84565e"). InnerVolumeSpecName "kube-api-access-xkhcq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:04:35.981205 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:35.981177 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36a5b75b-a454-45c5-81cb-4b5faf84565e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "36a5b75b-a454-45c5-81cb-4b5faf84565e" (UID: "36a5b75b-a454-45c5-81cb-4b5faf84565e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:04:35.981286 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:35.981206 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36a5b75b-a454-45c5-81cb-4b5faf84565e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "36a5b75b-a454-45c5-81cb-4b5faf84565e" (UID: "36a5b75b-a454-45c5-81cb-4b5faf84565e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:04:36.080310 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:36.080276 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36a5b75b-a454-45c5-81cb-4b5faf84565e-console-oauth-config\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:04:36.080310 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:36.080310 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-oauth-serving-cert\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:04:36.080460 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:36.080324 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-service-ca\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:04:36.080460 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:36.080339 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-console-config\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:04:36.080460 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:36.080350 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36a5b75b-a454-45c5-81cb-4b5faf84565e-trusted-ca-bundle\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:04:36.080460 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:36.080364 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36a5b75b-a454-45c5-81cb-4b5faf84565e-console-serving-cert\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:04:36.080460 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:36.080377 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkhcq\" (UniqueName: \"kubernetes.io/projected/36a5b75b-a454-45c5-81cb-4b5faf84565e-kube-api-access-xkhcq\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:04:36.708372 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:36.708344 2557 generic.go:356] "Generic (PLEG): container finished" podID="36a5b75b-a454-45c5-81cb-4b5faf84565e" containerID="44d5cc8d5a14417907c7a63d9b37b9e5de6d00774ce751919fd6aea8d02c9f63" exitCode=0 Sep 11 08:04:36.708713 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:36.708397 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b5d6989b-h8sc4" Sep 11 08:04:36.708713 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:36.708429 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b5d6989b-h8sc4" event={"ID":"36a5b75b-a454-45c5-81cb-4b5faf84565e","Type":"ContainerDied","Data":"44d5cc8d5a14417907c7a63d9b37b9e5de6d00774ce751919fd6aea8d02c9f63"} Sep 11 08:04:36.708713 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:36.708487 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b5d6989b-h8sc4" event={"ID":"36a5b75b-a454-45c5-81cb-4b5faf84565e","Type":"ContainerDied","Data":"d53e3af839ec632bb57c8f6a4fe43ef2a9c5bf7f5ba2964ff7bc0dfe0f1a21e5"} Sep 11 08:04:36.708713 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:36.708510 2557 scope.go:117] "RemoveContainer" containerID="44d5cc8d5a14417907c7a63d9b37b9e5de6d00774ce751919fd6aea8d02c9f63" Sep 11 08:04:36.717848 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:36.717826 2557 scope.go:117] "RemoveContainer" containerID="44d5cc8d5a14417907c7a63d9b37b9e5de6d00774ce751919fd6aea8d02c9f63" Sep 11 08:04:36.718165 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:04:36.718145 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44d5cc8d5a14417907c7a63d9b37b9e5de6d00774ce751919fd6aea8d02c9f63\": container with ID starting with 44d5cc8d5a14417907c7a63d9b37b9e5de6d00774ce751919fd6aea8d02c9f63 not found: ID does not exist" containerID="44d5cc8d5a14417907c7a63d9b37b9e5de6d00774ce751919fd6aea8d02c9f63" Sep 11 08:04:36.718268 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:36.718172 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44d5cc8d5a14417907c7a63d9b37b9e5de6d00774ce751919fd6aea8d02c9f63"} err="failed to get container status \"44d5cc8d5a14417907c7a63d9b37b9e5de6d00774ce751919fd6aea8d02c9f63\": rpc error: code = NotFound desc = could not find container \"44d5cc8d5a14417907c7a63d9b37b9e5de6d00774ce751919fd6aea8d02c9f63\": container with ID starting with 44d5cc8d5a14417907c7a63d9b37b9e5de6d00774ce751919fd6aea8d02c9f63 not found: ID does not exist" Sep 11 08:04:36.732818 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:36.732794 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77b5d6989b-h8sc4"] Sep 11 08:04:36.740061 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:36.740039 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77b5d6989b-h8sc4"] Sep 11 08:04:37.503454 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:04:37.503419 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36a5b75b-a454-45c5-81cb-4b5faf84565e" path="/var/lib/kubelet/pods/36a5b75b-a454-45c5-81cb-4b5faf84565e/volumes" Sep 11 08:06:15.965926 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:15.965874 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5bdff656f4-bpjpc"] Sep 11 08:06:15.966504 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:15.966285 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36a5b75b-a454-45c5-81cb-4b5faf84565e" containerName="console" Sep 11 08:06:15.966504 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:15.966305 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="36a5b75b-a454-45c5-81cb-4b5faf84565e" containerName="console" Sep 11 08:06:15.966504 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:15.966381 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="36a5b75b-a454-45c5-81cb-4b5faf84565e" containerName="console" Sep 11 08:06:15.969322 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:15.969301 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5bdff656f4-bpjpc" Sep 11 08:06:15.971472 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:15.971440 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 11 08:06:15.971561 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:15.971507 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-jsk6r\"" Sep 11 08:06:15.971866 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:15.971844 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 11 08:06:15.992370 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:15.992349 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5bdff656f4-bpjpc"] Sep 11 08:06:16.028835 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:16.028810 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbkx8\" (UniqueName: \"kubernetes.io/projected/76e423e0-9983-435c-a776-771356fd7c69-kube-api-access-hbkx8\") pod \"feast-operator-controller-manager-5bdff656f4-bpjpc\" (UID: \"76e423e0-9983-435c-a776-771356fd7c69\") " pod="feast-operator-system/feast-operator-controller-manager-5bdff656f4-bpjpc" Sep 11 08:06:16.129766 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:16.129746 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbkx8\" (UniqueName: \"kubernetes.io/projected/76e423e0-9983-435c-a776-771356fd7c69-kube-api-access-hbkx8\") pod \"feast-operator-controller-manager-5bdff656f4-bpjpc\" (UID: \"76e423e0-9983-435c-a776-771356fd7c69\") " pod="feast-operator-system/feast-operator-controller-manager-5bdff656f4-bpjpc" Sep 11 08:06:16.142376 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:16.142352 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbkx8\" (UniqueName: \"kubernetes.io/projected/76e423e0-9983-435c-a776-771356fd7c69-kube-api-access-hbkx8\") pod \"feast-operator-controller-manager-5bdff656f4-bpjpc\" (UID: \"76e423e0-9983-435c-a776-771356fd7c69\") " pod="feast-operator-system/feast-operator-controller-manager-5bdff656f4-bpjpc" Sep 11 08:06:16.279624 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:16.279550 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5bdff656f4-bpjpc" Sep 11 08:06:16.549854 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:16.549831 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5bdff656f4-bpjpc"] Sep 11 08:06:16.552123 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:06:16.552089 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76e423e0_9983_435c_a776_771356fd7c69.slice/crio-9ef347dd7c30ba80a016b5b271fb50c543dd1976d38fb97cc76024eaae605bd4 WatchSource:0}: Error finding container 9ef347dd7c30ba80a016b5b271fb50c543dd1976d38fb97cc76024eaae605bd4: Status 404 returned error can't find the container with id 9ef347dd7c30ba80a016b5b271fb50c543dd1976d38fb97cc76024eaae605bd4 Sep 11 08:06:16.554003 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:16.553987 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 11 08:06:17.070018 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:17.069985 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5bdff656f4-bpjpc" event={"ID":"76e423e0-9983-435c-a776-771356fd7c69","Type":"ContainerStarted","Data":"9ef347dd7c30ba80a016b5b271fb50c543dd1976d38fb97cc76024eaae605bd4"} Sep 11 08:06:19.078740 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:19.078701 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5bdff656f4-bpjpc" event={"ID":"76e423e0-9983-435c-a776-771356fd7c69","Type":"ContainerStarted","Data":"ea8e67b921776ec21d027543d672c4680bea230e1f2223dcee3b94613360e30a"} Sep 11 08:06:19.079159 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:19.078768 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-5bdff656f4-bpjpc" Sep 11 08:06:19.100231 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:19.100182 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-5bdff656f4-bpjpc" podStartSLOduration=1.771939783 podStartE2EDuration="4.100167407s" totalImagesPullingTime="2.328227624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:06:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:06:19.098616029 +0000 UTC m=+416.104333689" watchObservedRunningTime="2026-09-11 08:06:19.100167407 +0000 UTC m=+416.105885064" Sep 11 08:06:30.084705 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:30.084675 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-5bdff656f4-bpjpc" Sep 11 08:06:33.200194 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.200153 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bb4f78d5c-gbt9v"] Sep 11 08:06:33.204320 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.204295 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.216279 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.216254 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bb4f78d5c-gbt9v"] Sep 11 08:06:33.251611 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.251583 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/34628316-b370-492d-a638-797607438036-oauth-serving-cert\") pod \"console-7bb4f78d5c-gbt9v\" (UID: \"34628316-b370-492d-a638-797607438036\") " pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.251735 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.251616 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzqpl\" (UniqueName: \"kubernetes.io/projected/34628316-b370-492d-a638-797607438036-kube-api-access-wzqpl\") pod \"console-7bb4f78d5c-gbt9v\" (UID: \"34628316-b370-492d-a638-797607438036\") " pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.251735 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.251645 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/34628316-b370-492d-a638-797607438036-console-config\") pod \"console-7bb4f78d5c-gbt9v\" (UID: \"34628316-b370-492d-a638-797607438036\") " pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.251735 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.251704 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/34628316-b370-492d-a638-797607438036-console-serving-cert\") pod \"console-7bb4f78d5c-gbt9v\" (UID: \"34628316-b370-492d-a638-797607438036\") " pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.251852 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.251753 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/34628316-b370-492d-a638-797607438036-console-oauth-config\") pod \"console-7bb4f78d5c-gbt9v\" (UID: \"34628316-b370-492d-a638-797607438036\") " pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.251852 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.251789 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/34628316-b370-492d-a638-797607438036-service-ca\") pod \"console-7bb4f78d5c-gbt9v\" (UID: \"34628316-b370-492d-a638-797607438036\") " pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.251852 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.251819 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34628316-b370-492d-a638-797607438036-trusted-ca-bundle\") pod \"console-7bb4f78d5c-gbt9v\" (UID: \"34628316-b370-492d-a638-797607438036\") " pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.352232 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.352210 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/34628316-b370-492d-a638-797607438036-oauth-serving-cert\") pod \"console-7bb4f78d5c-gbt9v\" (UID: \"34628316-b370-492d-a638-797607438036\") " pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.352298 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.352239 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzqpl\" (UniqueName: \"kubernetes.io/projected/34628316-b370-492d-a638-797607438036-kube-api-access-wzqpl\") pod \"console-7bb4f78d5c-gbt9v\" (UID: \"34628316-b370-492d-a638-797607438036\") " pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.352298 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.352267 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/34628316-b370-492d-a638-797607438036-console-config\") pod \"console-7bb4f78d5c-gbt9v\" (UID: \"34628316-b370-492d-a638-797607438036\") " pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.352298 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.352290 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/34628316-b370-492d-a638-797607438036-console-serving-cert\") pod \"console-7bb4f78d5c-gbt9v\" (UID: \"34628316-b370-492d-a638-797607438036\") " pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.352443 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.352305 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/34628316-b370-492d-a638-797607438036-console-oauth-config\") pod \"console-7bb4f78d5c-gbt9v\" (UID: \"34628316-b370-492d-a638-797607438036\") " pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.352443 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.352322 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/34628316-b370-492d-a638-797607438036-service-ca\") pod \"console-7bb4f78d5c-gbt9v\" (UID: \"34628316-b370-492d-a638-797607438036\") " pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.352443 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.352345 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34628316-b370-492d-a638-797607438036-trusted-ca-bundle\") pod \"console-7bb4f78d5c-gbt9v\" (UID: \"34628316-b370-492d-a638-797607438036\") " pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.353038 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.353014 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/34628316-b370-492d-a638-797607438036-oauth-serving-cert\") pod \"console-7bb4f78d5c-gbt9v\" (UID: \"34628316-b370-492d-a638-797607438036\") " pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.353038 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.353030 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/34628316-b370-492d-a638-797607438036-console-config\") pod \"console-7bb4f78d5c-gbt9v\" (UID: \"34628316-b370-492d-a638-797607438036\") " pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.353186 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.353059 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/34628316-b370-492d-a638-797607438036-service-ca\") pod \"console-7bb4f78d5c-gbt9v\" (UID: \"34628316-b370-492d-a638-797607438036\") " pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.353186 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.353173 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34628316-b370-492d-a638-797607438036-trusted-ca-bundle\") pod \"console-7bb4f78d5c-gbt9v\" (UID: \"34628316-b370-492d-a638-797607438036\") " pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.354797 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.354770 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/34628316-b370-492d-a638-797607438036-console-oauth-config\") pod \"console-7bb4f78d5c-gbt9v\" (UID: \"34628316-b370-492d-a638-797607438036\") " pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.354917 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.354882 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/34628316-b370-492d-a638-797607438036-console-serving-cert\") pod \"console-7bb4f78d5c-gbt9v\" (UID: \"34628316-b370-492d-a638-797607438036\") " pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.361566 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.361545 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzqpl\" (UniqueName: \"kubernetes.io/projected/34628316-b370-492d-a638-797607438036-kube-api-access-wzqpl\") pod \"console-7bb4f78d5c-gbt9v\" (UID: \"34628316-b370-492d-a638-797607438036\") " pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.512813 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.512764 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:33.790458 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:33.790398 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bb4f78d5c-gbt9v"] Sep 11 08:06:33.793356 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:06:33.793333 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34628316_b370_492d_a638_797607438036.slice/crio-5c473d2e6bbe911c425f7d9a1f15caf299f6e90548c007bb819a79f276de0677 WatchSource:0}: Error finding container 5c473d2e6bbe911c425f7d9a1f15caf299f6e90548c007bb819a79f276de0677: Status 404 returned error can't find the container with id 5c473d2e6bbe911c425f7d9a1f15caf299f6e90548c007bb819a79f276de0677 Sep 11 08:06:34.131784 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:34.131753 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bb4f78d5c-gbt9v" event={"ID":"34628316-b370-492d-a638-797607438036","Type":"ContainerStarted","Data":"3aa66d4a5b4e844f9b17af3599eb855c82f9d04c705498c73af16091a3048e45"} Sep 11 08:06:34.131950 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:34.131806 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bb4f78d5c-gbt9v" event={"ID":"34628316-b370-492d-a638-797607438036","Type":"ContainerStarted","Data":"5c473d2e6bbe911c425f7d9a1f15caf299f6e90548c007bb819a79f276de0677"} Sep 11 08:06:34.155945 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:34.155798 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7bb4f78d5c-gbt9v" podStartSLOduration=1.15577934 podStartE2EDuration="1.15577934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:06:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:06:34.155690779 +0000 UTC m=+431.161408438" watchObservedRunningTime="2026-09-11 08:06:34.15577934 +0000 UTC m=+431.161496999" Sep 11 08:06:43.513822 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:43.513784 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:43.514260 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:43.513951 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:43.519081 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:43.519054 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:44.172540 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:44.172515 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7bb4f78d5c-gbt9v" Sep 11 08:06:44.239255 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:06:44.239217 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f5dfcd956-zb57g"] Sep 11 08:07:09.259932 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.259879 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-f5dfcd956-zb57g" podUID="0834f8c7-8504-4283-b59f-0e66494795f2" containerName="console" containerID="cri-o://a0be8df625331287dd936abd9458c3d6fd7b8ea78c4c5471869a341080bfa041" gracePeriod=15 Sep 11 08:07:09.501018 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.500989 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:07:09.596300 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.596248 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-service-ca\") pod \"0834f8c7-8504-4283-b59f-0e66494795f2\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " Sep 11 08:07:09.596300 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.596284 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-oauth-serving-cert\") pod \"0834f8c7-8504-4283-b59f-0e66494795f2\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " Sep 11 08:07:09.596477 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.596325 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0834f8c7-8504-4283-b59f-0e66494795f2-console-oauth-config\") pod \"0834f8c7-8504-4283-b59f-0e66494795f2\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " Sep 11 08:07:09.596477 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.596385 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-console-config\") pod \"0834f8c7-8504-4283-b59f-0e66494795f2\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " Sep 11 08:07:09.596477 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.596416 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42mb8\" (UniqueName: \"kubernetes.io/projected/0834f8c7-8504-4283-b59f-0e66494795f2-kube-api-access-42mb8\") pod \"0834f8c7-8504-4283-b59f-0e66494795f2\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " Sep 11 08:07:09.596477 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.596432 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-trusted-ca-bundle\") pod \"0834f8c7-8504-4283-b59f-0e66494795f2\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " Sep 11 08:07:09.596477 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.596464 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0834f8c7-8504-4283-b59f-0e66494795f2-console-serving-cert\") pod \"0834f8c7-8504-4283-b59f-0e66494795f2\" (UID: \"0834f8c7-8504-4283-b59f-0e66494795f2\") " Sep 11 08:07:09.596799 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.596757 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-service-ca" (OuterVolumeSpecName: "service-ca") pod "0834f8c7-8504-4283-b59f-0e66494795f2" (UID: "0834f8c7-8504-4283-b59f-0e66494795f2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:07:09.596924 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.596809 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0834f8c7-8504-4283-b59f-0e66494795f2" (UID: "0834f8c7-8504-4283-b59f-0e66494795f2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:07:09.596981 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.596892 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-console-config" (OuterVolumeSpecName: "console-config") pod "0834f8c7-8504-4283-b59f-0e66494795f2" (UID: "0834f8c7-8504-4283-b59f-0e66494795f2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:07:09.596981 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.596961 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0834f8c7-8504-4283-b59f-0e66494795f2" (UID: "0834f8c7-8504-4283-b59f-0e66494795f2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:07:09.598349 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.598327 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0834f8c7-8504-4283-b59f-0e66494795f2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0834f8c7-8504-4283-b59f-0e66494795f2" (UID: "0834f8c7-8504-4283-b59f-0e66494795f2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:07:09.598507 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.598486 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0834f8c7-8504-4283-b59f-0e66494795f2-kube-api-access-42mb8" (OuterVolumeSpecName: "kube-api-access-42mb8") pod "0834f8c7-8504-4283-b59f-0e66494795f2" (UID: "0834f8c7-8504-4283-b59f-0e66494795f2"). InnerVolumeSpecName "kube-api-access-42mb8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:07:09.598593 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.598577 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0834f8c7-8504-4283-b59f-0e66494795f2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0834f8c7-8504-4283-b59f-0e66494795f2" (UID: "0834f8c7-8504-4283-b59f-0e66494795f2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:07:09.697455 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.697421 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-trusted-ca-bundle\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:07:09.697455 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.697452 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0834f8c7-8504-4283-b59f-0e66494795f2-console-serving-cert\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:07:09.697591 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.697467 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-service-ca\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:07:09.697591 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.697480 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-oauth-serving-cert\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:07:09.697591 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.697492 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0834f8c7-8504-4283-b59f-0e66494795f2-console-oauth-config\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:07:09.697591 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.697504 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0834f8c7-8504-4283-b59f-0e66494795f2-console-config\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:07:09.697591 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:09.697516 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42mb8\" (UniqueName: \"kubernetes.io/projected/0834f8c7-8504-4283-b59f-0e66494795f2-kube-api-access-42mb8\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:07:10.268137 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:10.268103 2557 generic.go:356] "Generic (PLEG): container finished" podID="0834f8c7-8504-4283-b59f-0e66494795f2" containerID="a0be8df625331287dd936abd9458c3d6fd7b8ea78c4c5471869a341080bfa041" exitCode=0 Sep 11 08:07:10.268466 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:10.268163 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f5dfcd956-zb57g" Sep 11 08:07:10.268466 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:10.268187 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f5dfcd956-zb57g" event={"ID":"0834f8c7-8504-4283-b59f-0e66494795f2","Type":"ContainerDied","Data":"a0be8df625331287dd936abd9458c3d6fd7b8ea78c4c5471869a341080bfa041"} Sep 11 08:07:10.268466 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:10.268265 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f5dfcd956-zb57g" event={"ID":"0834f8c7-8504-4283-b59f-0e66494795f2","Type":"ContainerDied","Data":"907bcb69e2fe1a56904d7cd21464a5bb66aa8f2985893b12ef71a360cf14b2a2"} Sep 11 08:07:10.268466 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:10.268288 2557 scope.go:117] "RemoveContainer" containerID="a0be8df625331287dd936abd9458c3d6fd7b8ea78c4c5471869a341080bfa041" Sep 11 08:07:10.276852 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:10.276829 2557 scope.go:117] "RemoveContainer" containerID="a0be8df625331287dd936abd9458c3d6fd7b8ea78c4c5471869a341080bfa041" Sep 11 08:07:10.277152 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:07:10.277131 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0be8df625331287dd936abd9458c3d6fd7b8ea78c4c5471869a341080bfa041\": container with ID starting with a0be8df625331287dd936abd9458c3d6fd7b8ea78c4c5471869a341080bfa041 not found: ID does not exist" containerID="a0be8df625331287dd936abd9458c3d6fd7b8ea78c4c5471869a341080bfa041" Sep 11 08:07:10.277223 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:10.277164 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0be8df625331287dd936abd9458c3d6fd7b8ea78c4c5471869a341080bfa041"} err="failed to get container status \"a0be8df625331287dd936abd9458c3d6fd7b8ea78c4c5471869a341080bfa041\": rpc error: code = NotFound desc = could not find container \"a0be8df625331287dd936abd9458c3d6fd7b8ea78c4c5471869a341080bfa041\": container with ID starting with a0be8df625331287dd936abd9458c3d6fd7b8ea78c4c5471869a341080bfa041 not found: ID does not exist" Sep 11 08:07:10.289255 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:10.289237 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f5dfcd956-zb57g"] Sep 11 08:07:10.294380 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:10.294360 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f5dfcd956-zb57g"] Sep 11 08:07:11.501868 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:07:11.501836 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0834f8c7-8504-4283-b59f-0e66494795f2" path="/var/lib/kubelet/pods/0834f8c7-8504-4283-b59f-0e66494795f2/volumes" Sep 11 08:12:35.990628 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:35.990584 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-mq8b8"] Sep 11 08:12:35.991228 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:35.991003 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0834f8c7-8504-4283-b59f-0e66494795f2" containerName="console" Sep 11 08:12:35.991228 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:35.991023 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0834f8c7-8504-4283-b59f-0e66494795f2" containerName="console" Sep 11 08:12:35.991228 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:35.991110 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0834f8c7-8504-4283-b59f-0e66494795f2" containerName="console" Sep 11 08:12:35.994387 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:35.994365 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-mq8b8" Sep 11 08:12:35.997113 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:35.997079 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 11 08:12:35.997363 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:35.997344 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 11 08:12:35.997465 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:35.997344 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-7wv2k\"" Sep 11 08:12:35.997465 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:35.997407 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Sep 11 08:12:36.004962 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:36.004942 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-mq8b8"] Sep 11 08:12:36.058647 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:36.058618 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6dh8\" (UniqueName: \"kubernetes.io/projected/e8c7128a-fa88-45c5-802d-8ccf9e69cfd8-kube-api-access-h6dh8\") pod \"postgres-864846d4f5-mq8b8\" (UID: \"e8c7128a-fa88-45c5-802d-8ccf9e69cfd8\") " pod="test-ns-feast/postgres-864846d4f5-mq8b8" Sep 11 08:12:36.058755 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:36.058655 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/e8c7128a-fa88-45c5-802d-8ccf9e69cfd8-postgresdata\") pod \"postgres-864846d4f5-mq8b8\" (UID: \"e8c7128a-fa88-45c5-802d-8ccf9e69cfd8\") " pod="test-ns-feast/postgres-864846d4f5-mq8b8" Sep 11 08:12:36.159547 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:36.159518 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6dh8\" (UniqueName: \"kubernetes.io/projected/e8c7128a-fa88-45c5-802d-8ccf9e69cfd8-kube-api-access-h6dh8\") pod \"postgres-864846d4f5-mq8b8\" (UID: \"e8c7128a-fa88-45c5-802d-8ccf9e69cfd8\") " pod="test-ns-feast/postgres-864846d4f5-mq8b8" Sep 11 08:12:36.159666 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:36.159560 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/e8c7128a-fa88-45c5-802d-8ccf9e69cfd8-postgresdata\") pod \"postgres-864846d4f5-mq8b8\" (UID: \"e8c7128a-fa88-45c5-802d-8ccf9e69cfd8\") " pod="test-ns-feast/postgres-864846d4f5-mq8b8" Sep 11 08:12:36.159923 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:36.159885 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/e8c7128a-fa88-45c5-802d-8ccf9e69cfd8-postgresdata\") pod \"postgres-864846d4f5-mq8b8\" (UID: \"e8c7128a-fa88-45c5-802d-8ccf9e69cfd8\") " pod="test-ns-feast/postgres-864846d4f5-mq8b8" Sep 11 08:12:36.172763 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:36.172741 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6dh8\" (UniqueName: \"kubernetes.io/projected/e8c7128a-fa88-45c5-802d-8ccf9e69cfd8-kube-api-access-h6dh8\") pod \"postgres-864846d4f5-mq8b8\" (UID: \"e8c7128a-fa88-45c5-802d-8ccf9e69cfd8\") " pod="test-ns-feast/postgres-864846d4f5-mq8b8" Sep 11 08:12:36.314550 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:36.314499 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-mq8b8" Sep 11 08:12:36.581804 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:36.581735 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-mq8b8"] Sep 11 08:12:36.585647 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:12:36.585623 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8c7128a_fa88_45c5_802d_8ccf9e69cfd8.slice/crio-738dac0f67ec666bbc96b7d21370703dc66e4db37df469d552529cb8f64b3d16 WatchSource:0}: Error finding container 738dac0f67ec666bbc96b7d21370703dc66e4db37df469d552529cb8f64b3d16: Status 404 returned error can't find the container with id 738dac0f67ec666bbc96b7d21370703dc66e4db37df469d552529cb8f64b3d16 Sep 11 08:12:36.587888 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:36.587869 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 11 08:12:37.446018 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:37.445978 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-mq8b8" event={"ID":"e8c7128a-fa88-45c5-802d-8ccf9e69cfd8","Type":"ContainerStarted","Data":"738dac0f67ec666bbc96b7d21370703dc66e4db37df469d552529cb8f64b3d16"} Sep 11 08:12:42.467424 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:42.467384 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-mq8b8" event={"ID":"e8c7128a-fa88-45c5-802d-8ccf9e69cfd8","Type":"ContainerStarted","Data":"61befbc5391156dfabf9d292bfd3d4a7c060ef5a02cb7f873a428a706de1fc08"} Sep 11 08:12:42.486183 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:12:42.486140 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-mq8b8" podStartSLOduration=2.107580916 podStartE2EDuration="7.486126255s" totalImagesPullingTime="5.378545339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:12:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:12:42.485161576 +0000 UTC m=+799.490879235" watchObservedRunningTime="2026-09-11 08:12:42.486126255 +0000 UTC m=+799.491843912" Sep 11 08:13:13.536535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:13.536504 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-test-apply-ndfvt"] Sep 11 08:13:13.542567 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:13.542545 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-ndfvt" Sep 11 08:13:13.544685 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:13.544663 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 11 08:13:13.544791 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:13.544759 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-cronjob-dockercfg-9pvgp\"" Sep 11 08:13:13.547988 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:13.547966 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-ndfvt"] Sep 11 08:13:13.607268 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:13.607244 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-588x2\" (UniqueName: \"kubernetes.io/projected/a7f325c9-69d1-4491-afcf-a2dc7167cc6a-kube-api-access-588x2\") pod \"feast-test-apply-ndfvt\" (UID: \"a7f325c9-69d1-4491-afcf-a2dc7167cc6a\") " pod="test-ns-feast/feast-test-apply-ndfvt" Sep 11 08:13:13.707748 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:13.707724 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-588x2\" (UniqueName: \"kubernetes.io/projected/a7f325c9-69d1-4491-afcf-a2dc7167cc6a-kube-api-access-588x2\") pod \"feast-test-apply-ndfvt\" (UID: \"a7f325c9-69d1-4491-afcf-a2dc7167cc6a\") " pod="test-ns-feast/feast-test-apply-ndfvt" Sep 11 08:13:13.716575 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:13.716545 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-588x2\" (UniqueName: \"kubernetes.io/projected/a7f325c9-69d1-4491-afcf-a2dc7167cc6a-kube-api-access-588x2\") pod \"feast-test-apply-ndfvt\" (UID: \"a7f325c9-69d1-4491-afcf-a2dc7167cc6a\") " pod="test-ns-feast/feast-test-apply-ndfvt" Sep 11 08:13:13.852482 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:13.852432 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-ndfvt" Sep 11 08:13:14.119955 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:14.119852 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-ndfvt"] Sep 11 08:13:14.123325 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:13:14.123300 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7f325c9_69d1_4491_afcf_a2dc7167cc6a.slice/crio-44c38daa3dd6b8bca1b95c19192138e57b0109194e420e34d992d62e3132a71f WatchSource:0}: Error finding container 44c38daa3dd6b8bca1b95c19192138e57b0109194e420e34d992d62e3132a71f: Status 404 returned error can't find the container with id 44c38daa3dd6b8bca1b95c19192138e57b0109194e420e34d992d62e3132a71f Sep 11 08:13:14.580490 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:14.580451 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-ndfvt" event={"ID":"a7f325c9-69d1-4491-afcf-a2dc7167cc6a","Type":"ContainerStarted","Data":"44c38daa3dd6b8bca1b95c19192138e57b0109194e420e34d992d62e3132a71f"} Sep 11 08:13:20.605472 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:20.605440 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-ndfvt" event={"ID":"a7f325c9-69d1-4491-afcf-a2dc7167cc6a","Type":"ContainerStarted","Data":"c295e560649e1ec9fcedf76a4f8f60c0814d13eb480fe1e1d342c4fab8c86b72"} Sep 11 08:13:26.628602 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:26.628573 2557 generic.go:356] "Generic (PLEG): container finished" podID="a7f325c9-69d1-4491-afcf-a2dc7167cc6a" containerID="c295e560649e1ec9fcedf76a4f8f60c0814d13eb480fe1e1d342c4fab8c86b72" exitCode=0 Sep 11 08:13:26.628912 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:26.628647 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-ndfvt" event={"ID":"a7f325c9-69d1-4491-afcf-a2dc7167cc6a","Type":"ContainerDied","Data":"c295e560649e1ec9fcedf76a4f8f60c0814d13eb480fe1e1d342c4fab8c86b72"} Sep 11 08:13:27.633558 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:27.633528 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-ndfvt" event={"ID":"a7f325c9-69d1-4491-afcf-a2dc7167cc6a","Type":"ContainerStarted","Data":"7fdce5534c56ca66e9842af76a5686e01a75cd8bf49ac933cd02b9aba5e55e6d"} Sep 11 08:13:27.655186 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:27.655136 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-test-apply-ndfvt" podStartSLOduration=2.25427031 podStartE2EDuration="14.655118118s" totalImagesPullingTime="5.895818613s" totalInitContainerRuntime="6.505029195s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:13:27.653041902 +0000 UTC m=+844.658759571" watchObservedRunningTime="2026-09-11 08:13:27.655118118 +0000 UTC m=+844.660835776" Sep 11 08:13:36.668471 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:36.668400 2557 generic.go:356] "Generic (PLEG): container finished" podID="a7f325c9-69d1-4491-afcf-a2dc7167cc6a" containerID="7fdce5534c56ca66e9842af76a5686e01a75cd8bf49ac933cd02b9aba5e55e6d" exitCode=0 Sep 11 08:13:36.668791 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:36.668472 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-ndfvt" event={"ID":"a7f325c9-69d1-4491-afcf-a2dc7167cc6a","Type":"ContainerDied","Data":"7fdce5534c56ca66e9842af76a5686e01a75cd8bf49ac933cd02b9aba5e55e6d"} Sep 11 08:13:37.795010 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:37.794986 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-ndfvt" Sep 11 08:13:37.880150 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:37.880126 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-588x2\" (UniqueName: \"kubernetes.io/projected/a7f325c9-69d1-4491-afcf-a2dc7167cc6a-kube-api-access-588x2\") pod \"a7f325c9-69d1-4491-afcf-a2dc7167cc6a\" (UID: \"a7f325c9-69d1-4491-afcf-a2dc7167cc6a\") " Sep 11 08:13:37.882163 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:37.882142 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7f325c9-69d1-4491-afcf-a2dc7167cc6a-kube-api-access-588x2" (OuterVolumeSpecName: "kube-api-access-588x2") pod "a7f325c9-69d1-4491-afcf-a2dc7167cc6a" (UID: "a7f325c9-69d1-4491-afcf-a2dc7167cc6a"). InnerVolumeSpecName "kube-api-access-588x2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:13:37.981028 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:37.980976 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-588x2\" (UniqueName: \"kubernetes.io/projected/a7f325c9-69d1-4491-afcf-a2dc7167cc6a-kube-api-access-588x2\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:13:38.677398 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:38.677372 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-ndfvt" event={"ID":"a7f325c9-69d1-4491-afcf-a2dc7167cc6a","Type":"ContainerDied","Data":"44c38daa3dd6b8bca1b95c19192138e57b0109194e420e34d992d62e3132a71f"} Sep 11 08:13:38.677526 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:38.677406 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="44c38daa3dd6b8bca1b95c19192138e57b0109194e420e34d992d62e3132a71f" Sep 11 08:13:38.677526 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:38.677387 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-ndfvt" Sep 11 08:13:38.878708 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:38.878685 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-ndfvt_a7f325c9-69d1-4491-afcf-a2dc7167cc6a/feast-0/0.log" Sep 11 08:13:38.887292 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:13:38.887275 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-ndfvt_a7f325c9-69d1-4491-afcf-a2dc7167cc6a/feast-1/0.log" Sep 11 08:14:17.273893 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:17.273806 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-test-apply-ndfvt"] Sep 11 08:14:17.276705 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:17.276677 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-test-apply-ndfvt"] Sep 11 08:14:17.281972 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:17.281949 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-mq8b8"] Sep 11 08:14:17.282151 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:17.282114 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-mq8b8" podUID="e8c7128a-fa88-45c5-802d-8ccf9e69cfd8" containerName="postgres" containerID="cri-o://61befbc5391156dfabf9d292bfd3d4a7c060ef5a02cb7f873a428a706de1fc08" gracePeriod=30 Sep 11 08:14:17.507193 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:17.507163 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7f325c9-69d1-4491-afcf-a2dc7167cc6a" path="/var/lib/kubelet/pods/a7f325c9-69d1-4491-afcf-a2dc7167cc6a/volumes" Sep 11 08:14:17.524483 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:17.524440 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-mq8b8" Sep 11 08:14:17.646579 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:17.646545 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/e8c7128a-fa88-45c5-802d-8ccf9e69cfd8-postgresdata\") pod \"e8c7128a-fa88-45c5-802d-8ccf9e69cfd8\" (UID: \"e8c7128a-fa88-45c5-802d-8ccf9e69cfd8\") " Sep 11 08:14:17.646711 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:17.646637 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6dh8\" (UniqueName: \"kubernetes.io/projected/e8c7128a-fa88-45c5-802d-8ccf9e69cfd8-kube-api-access-h6dh8\") pod \"e8c7128a-fa88-45c5-802d-8ccf9e69cfd8\" (UID: \"e8c7128a-fa88-45c5-802d-8ccf9e69cfd8\") " Sep 11 08:14:17.648481 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:17.648456 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8c7128a-fa88-45c5-802d-8ccf9e69cfd8-kube-api-access-h6dh8" (OuterVolumeSpecName: "kube-api-access-h6dh8") pod "e8c7128a-fa88-45c5-802d-8ccf9e69cfd8" (UID: "e8c7128a-fa88-45c5-802d-8ccf9e69cfd8"). InnerVolumeSpecName "kube-api-access-h6dh8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:14:17.675128 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:17.675095 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8c7128a-fa88-45c5-802d-8ccf9e69cfd8-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "e8c7128a-fa88-45c5-802d-8ccf9e69cfd8" (UID: "e8c7128a-fa88-45c5-802d-8ccf9e69cfd8"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 08:14:17.747996 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:17.747971 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6dh8\" (UniqueName: \"kubernetes.io/projected/e8c7128a-fa88-45c5-802d-8ccf9e69cfd8-kube-api-access-h6dh8\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:14:17.747996 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:17.747993 2557 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/e8c7128a-fa88-45c5-802d-8ccf9e69cfd8-postgresdata\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:14:17.815867 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:17.815805 2557 generic.go:356] "Generic (PLEG): container finished" podID="e8c7128a-fa88-45c5-802d-8ccf9e69cfd8" containerID="61befbc5391156dfabf9d292bfd3d4a7c060ef5a02cb7f873a428a706de1fc08" exitCode=0 Sep 11 08:14:17.815867 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:17.815853 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-mq8b8" Sep 11 08:14:17.816045 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:17.815881 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-mq8b8" event={"ID":"e8c7128a-fa88-45c5-802d-8ccf9e69cfd8","Type":"ContainerDied","Data":"61befbc5391156dfabf9d292bfd3d4a7c060ef5a02cb7f873a428a706de1fc08"} Sep 11 08:14:17.816045 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:17.815957 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-mq8b8" event={"ID":"e8c7128a-fa88-45c5-802d-8ccf9e69cfd8","Type":"ContainerDied","Data":"738dac0f67ec666bbc96b7d21370703dc66e4db37df469d552529cb8f64b3d16"} Sep 11 08:14:17.816045 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:17.815974 2557 scope.go:117] "RemoveContainer" containerID="61befbc5391156dfabf9d292bfd3d4a7c060ef5a02cb7f873a428a706de1fc08" Sep 11 08:14:17.823794 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:17.823778 2557 scope.go:117] "RemoveContainer" containerID="61befbc5391156dfabf9d292bfd3d4a7c060ef5a02cb7f873a428a706de1fc08" Sep 11 08:14:17.824093 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:14:17.824050 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61befbc5391156dfabf9d292bfd3d4a7c060ef5a02cb7f873a428a706de1fc08\": container with ID starting with 61befbc5391156dfabf9d292bfd3d4a7c060ef5a02cb7f873a428a706de1fc08 not found: ID does not exist" containerID="61befbc5391156dfabf9d292bfd3d4a7c060ef5a02cb7f873a428a706de1fc08" Sep 11 08:14:17.824151 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:17.824105 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61befbc5391156dfabf9d292bfd3d4a7c060ef5a02cb7f873a428a706de1fc08"} err="failed to get container status \"61befbc5391156dfabf9d292bfd3d4a7c060ef5a02cb7f873a428a706de1fc08\": rpc error: code = NotFound desc = could not find container \"61befbc5391156dfabf9d292bfd3d4a7c060ef5a02cb7f873a428a706de1fc08\": container with ID starting with 61befbc5391156dfabf9d292bfd3d4a7c060ef5a02cb7f873a428a706de1fc08 not found: ID does not exist" Sep 11 08:14:17.837316 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:17.837292 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-mq8b8"] Sep 11 08:14:17.839576 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:17.839555 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-mq8b8"] Sep 11 08:14:19.502144 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:14:19.502110 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8c7128a-fa88-45c5-802d-8ccf9e69cfd8" path="/var/lib/kubelet/pods/e8c7128a-fa88-45c5-802d-8ccf9e69cfd8/volumes" Sep 11 08:17:50.815816 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:50.815739 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-rj2ss"] Sep 11 08:17:50.816387 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:50.816314 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7f325c9-69d1-4491-afcf-a2dc7167cc6a" containerName="feast-0" Sep 11 08:17:50.816387 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:50.816336 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7f325c9-69d1-4491-afcf-a2dc7167cc6a" containerName="feast-0" Sep 11 08:17:50.816387 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:50.816347 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7f325c9-69d1-4491-afcf-a2dc7167cc6a" containerName="feast-1" Sep 11 08:17:50.816387 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:50.816355 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7f325c9-69d1-4491-afcf-a2dc7167cc6a" containerName="feast-1" Sep 11 08:17:50.816387 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:50.816384 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8c7128a-fa88-45c5-802d-8ccf9e69cfd8" containerName="postgres" Sep 11 08:17:50.816638 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:50.816393 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8c7128a-fa88-45c5-802d-8ccf9e69cfd8" containerName="postgres" Sep 11 08:17:50.816638 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:50.816479 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8c7128a-fa88-45c5-802d-8ccf9e69cfd8" containerName="postgres" Sep 11 08:17:50.816638 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:50.816492 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7f325c9-69d1-4491-afcf-a2dc7167cc6a" containerName="feast-1" Sep 11 08:17:50.819464 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:50.819443 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-rj2ss" Sep 11 08:17:50.822933 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:50.822910 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Sep 11 08:17:50.822933 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:50.822936 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Sep 11 08:17:50.823112 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:50.822975 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-5hdfz\"" Sep 11 08:17:50.823265 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:50.823244 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Sep 11 08:17:50.832115 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:50.832095 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-rj2ss"] Sep 11 08:17:50.871283 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:50.871263 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqrrr\" (UniqueName: \"kubernetes.io/projected/2d2e611b-3624-425f-b04e-34e8b53f82be-kube-api-access-pqrrr\") pod \"postgres-864846d4f5-rj2ss\" (UID: \"2d2e611b-3624-425f-b04e-34e8b53f82be\") " pod="test-ns-feast-rest/postgres-864846d4f5-rj2ss" Sep 11 08:17:50.871369 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:50.871292 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/2d2e611b-3624-425f-b04e-34e8b53f82be-postgresdata\") pod \"postgres-864846d4f5-rj2ss\" (UID: \"2d2e611b-3624-425f-b04e-34e8b53f82be\") " pod="test-ns-feast-rest/postgres-864846d4f5-rj2ss" Sep 11 08:17:50.972645 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:50.972610 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqrrr\" (UniqueName: \"kubernetes.io/projected/2d2e611b-3624-425f-b04e-34e8b53f82be-kube-api-access-pqrrr\") pod \"postgres-864846d4f5-rj2ss\" (UID: \"2d2e611b-3624-425f-b04e-34e8b53f82be\") " pod="test-ns-feast-rest/postgres-864846d4f5-rj2ss" Sep 11 08:17:50.972789 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:50.972654 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/2d2e611b-3624-425f-b04e-34e8b53f82be-postgresdata\") pod \"postgres-864846d4f5-rj2ss\" (UID: \"2d2e611b-3624-425f-b04e-34e8b53f82be\") " pod="test-ns-feast-rest/postgres-864846d4f5-rj2ss" Sep 11 08:17:50.973132 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:50.973114 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/2d2e611b-3624-425f-b04e-34e8b53f82be-postgresdata\") pod \"postgres-864846d4f5-rj2ss\" (UID: \"2d2e611b-3624-425f-b04e-34e8b53f82be\") " pod="test-ns-feast-rest/postgres-864846d4f5-rj2ss" Sep 11 08:17:50.983067 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:50.983048 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqrrr\" (UniqueName: \"kubernetes.io/projected/2d2e611b-3624-425f-b04e-34e8b53f82be-kube-api-access-pqrrr\") pod \"postgres-864846d4f5-rj2ss\" (UID: \"2d2e611b-3624-425f-b04e-34e8b53f82be\") " pod="test-ns-feast-rest/postgres-864846d4f5-rj2ss" Sep 11 08:17:51.140078 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:51.140036 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-rj2ss" Sep 11 08:17:51.410237 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:51.410162 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-rj2ss"] Sep 11 08:17:51.412815 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:17:51.412782 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d2e611b_3624_425f_b04e_34e8b53f82be.slice/crio-6dcb676f7e8050cd388c29e80cfe9fcf48200ab571db7e8b6d0c2fb0306b3887 WatchSource:0}: Error finding container 6dcb676f7e8050cd388c29e80cfe9fcf48200ab571db7e8b6d0c2fb0306b3887: Status 404 returned error can't find the container with id 6dcb676f7e8050cd388c29e80cfe9fcf48200ab571db7e8b6d0c2fb0306b3887 Sep 11 08:17:51.605868 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:51.605830 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-rj2ss" event={"ID":"2d2e611b-3624-425f-b04e-34e8b53f82be","Type":"ContainerStarted","Data":"65d1bdf575ac168a1710a108757087f73fe61a02bd1badbf398ebd3c145cd847"} Sep 11 08:17:51.605868 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:51.605874 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-rj2ss" event={"ID":"2d2e611b-3624-425f-b04e-34e8b53f82be","Type":"ContainerStarted","Data":"6dcb676f7e8050cd388c29e80cfe9fcf48200ab571db7e8b6d0c2fb0306b3887"} Sep 11 08:17:51.623617 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:17:51.623566 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-rj2ss" podStartSLOduration=1.6235455349999999 podStartE2EDuration="1.623545535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:17:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:17:51.623032198 +0000 UTC m=+1108.628749869" watchObservedRunningTime="2026-09-11 08:17:51.623545535 +0000 UTC m=+1108.629263194" Sep 11 08:19:23.531687 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:19:23.531660 2557 scope.go:117] "RemoveContainer" containerID="c295e560649e1ec9fcedf76a4f8f60c0814d13eb480fe1e1d342c4fab8c86b72" Sep 11 08:20:10.921252 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:10.921210 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5bdff656f4-bpjpc"] Sep 11 08:20:10.921750 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:10.921398 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-5bdff656f4-bpjpc" podUID="76e423e0-9983-435c-a776-771356fd7c69" containerName="manager" containerID="cri-o://ea8e67b921776ec21d027543d672c4680bea230e1f2223dcee3b94613360e30a" gracePeriod=10 Sep 11 08:20:11.119041 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:11.119005 2557 generic.go:356] "Generic (PLEG): container finished" podID="76e423e0-9983-435c-a776-771356fd7c69" containerID="ea8e67b921776ec21d027543d672c4680bea230e1f2223dcee3b94613360e30a" exitCode=0 Sep 11 08:20:11.119174 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:11.119079 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5bdff656f4-bpjpc" event={"ID":"76e423e0-9983-435c-a776-771356fd7c69","Type":"ContainerDied","Data":"ea8e67b921776ec21d027543d672c4680bea230e1f2223dcee3b94613360e30a"} Sep 11 08:20:11.148450 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:11.148432 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5bdff656f4-bpjpc" Sep 11 08:20:11.204381 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:11.204333 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbkx8\" (UniqueName: \"kubernetes.io/projected/76e423e0-9983-435c-a776-771356fd7c69-kube-api-access-hbkx8\") pod \"76e423e0-9983-435c-a776-771356fd7c69\" (UID: \"76e423e0-9983-435c-a776-771356fd7c69\") " Sep 11 08:20:11.206454 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:11.206429 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76e423e0-9983-435c-a776-771356fd7c69-kube-api-access-hbkx8" (OuterVolumeSpecName: "kube-api-access-hbkx8") pod "76e423e0-9983-435c-a776-771356fd7c69" (UID: "76e423e0-9983-435c-a776-771356fd7c69"). InnerVolumeSpecName "kube-api-access-hbkx8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:20:11.305358 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:11.305323 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbkx8\" (UniqueName: \"kubernetes.io/projected/76e423e0-9983-435c-a776-771356fd7c69-kube-api-access-hbkx8\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:20:12.123913 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:12.123874 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5bdff656f4-bpjpc" event={"ID":"76e423e0-9983-435c-a776-771356fd7c69","Type":"ContainerDied","Data":"9ef347dd7c30ba80a016b5b271fb50c543dd1976d38fb97cc76024eaae605bd4"} Sep 11 08:20:12.123913 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:12.123894 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5bdff656f4-bpjpc" Sep 11 08:20:12.124436 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:12.123956 2557 scope.go:117] "RemoveContainer" containerID="ea8e67b921776ec21d027543d672c4680bea230e1f2223dcee3b94613360e30a" Sep 11 08:20:12.160728 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:12.160704 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5bdff656f4-bpjpc"] Sep 11 08:20:12.167512 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:12.167493 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5bdff656f4-bpjpc"] Sep 11 08:20:13.425780 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:13.425748 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-rj2ss"] Sep 11 08:20:13.426190 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:13.425921 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-rj2ss" podUID="2d2e611b-3624-425f-b04e-34e8b53f82be" containerName="postgres" containerID="cri-o://65d1bdf575ac168a1710a108757087f73fe61a02bd1badbf398ebd3c145cd847" gracePeriod=30 Sep 11 08:20:13.501803 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:13.501767 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76e423e0-9983-435c-a776-771356fd7c69" path="/var/lib/kubelet/pods/76e423e0-9983-435c-a776-771356fd7c69/volumes" Sep 11 08:20:13.660642 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:13.660621 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-rj2ss" Sep 11 08:20:13.723960 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:13.723872 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqrrr\" (UniqueName: \"kubernetes.io/projected/2d2e611b-3624-425f-b04e-34e8b53f82be-kube-api-access-pqrrr\") pod \"2d2e611b-3624-425f-b04e-34e8b53f82be\" (UID: \"2d2e611b-3624-425f-b04e-34e8b53f82be\") " Sep 11 08:20:13.724096 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:13.723988 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/2d2e611b-3624-425f-b04e-34e8b53f82be-postgresdata\") pod \"2d2e611b-3624-425f-b04e-34e8b53f82be\" (UID: \"2d2e611b-3624-425f-b04e-34e8b53f82be\") " Sep 11 08:20:13.725865 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:13.725837 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d2e611b-3624-425f-b04e-34e8b53f82be-kube-api-access-pqrrr" (OuterVolumeSpecName: "kube-api-access-pqrrr") pod "2d2e611b-3624-425f-b04e-34e8b53f82be" (UID: "2d2e611b-3624-425f-b04e-34e8b53f82be"). InnerVolumeSpecName "kube-api-access-pqrrr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:20:13.757847 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:13.757819 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d2e611b-3624-425f-b04e-34e8b53f82be-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "2d2e611b-3624-425f-b04e-34e8b53f82be" (UID: "2d2e611b-3624-425f-b04e-34e8b53f82be"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 08:20:13.824610 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:13.824587 2557 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/2d2e611b-3624-425f-b04e-34e8b53f82be-postgresdata\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:20:13.824610 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:13.824610 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqrrr\" (UniqueName: \"kubernetes.io/projected/2d2e611b-3624-425f-b04e-34e8b53f82be-kube-api-access-pqrrr\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:20:14.132835 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:14.132806 2557 generic.go:356] "Generic (PLEG): container finished" podID="2d2e611b-3624-425f-b04e-34e8b53f82be" containerID="65d1bdf575ac168a1710a108757087f73fe61a02bd1badbf398ebd3c145cd847" exitCode=0 Sep 11 08:20:14.132960 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:14.132849 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-rj2ss" Sep 11 08:20:14.132960 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:14.132884 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-rj2ss" event={"ID":"2d2e611b-3624-425f-b04e-34e8b53f82be","Type":"ContainerDied","Data":"65d1bdf575ac168a1710a108757087f73fe61a02bd1badbf398ebd3c145cd847"} Sep 11 08:20:14.132960 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:14.132942 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-rj2ss" event={"ID":"2d2e611b-3624-425f-b04e-34e8b53f82be","Type":"ContainerDied","Data":"6dcb676f7e8050cd388c29e80cfe9fcf48200ab571db7e8b6d0c2fb0306b3887"} Sep 11 08:20:14.133188 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:14.132966 2557 scope.go:117] "RemoveContainer" containerID="65d1bdf575ac168a1710a108757087f73fe61a02bd1badbf398ebd3c145cd847" Sep 11 08:20:14.142104 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:14.142081 2557 scope.go:117] "RemoveContainer" containerID="65d1bdf575ac168a1710a108757087f73fe61a02bd1badbf398ebd3c145cd847" Sep 11 08:20:14.142404 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:20:14.142384 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65d1bdf575ac168a1710a108757087f73fe61a02bd1badbf398ebd3c145cd847\": container with ID starting with 65d1bdf575ac168a1710a108757087f73fe61a02bd1badbf398ebd3c145cd847 not found: ID does not exist" containerID="65d1bdf575ac168a1710a108757087f73fe61a02bd1badbf398ebd3c145cd847" Sep 11 08:20:14.142454 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:14.142412 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65d1bdf575ac168a1710a108757087f73fe61a02bd1badbf398ebd3c145cd847"} err="failed to get container status \"65d1bdf575ac168a1710a108757087f73fe61a02bd1badbf398ebd3c145cd847\": rpc error: code = NotFound desc = could not find container \"65d1bdf575ac168a1710a108757087f73fe61a02bd1badbf398ebd3c145cd847\": container with ID starting with 65d1bdf575ac168a1710a108757087f73fe61a02bd1badbf398ebd3c145cd847 not found: ID does not exist" Sep 11 08:20:14.155869 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:14.155847 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-rj2ss"] Sep 11 08:20:14.160546 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:14.160528 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-rj2ss"] Sep 11 08:20:15.502677 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:15.502641 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d2e611b-3624-425f-b04e-34e8b53f82be" path="/var/lib/kubelet/pods/2d2e611b-3624-425f-b04e-34e8b53f82be/volumes" Sep 11 08:20:22.741098 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:22.741043 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-fv5tj"] Sep 11 08:20:22.741496 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:22.741300 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76e423e0-9983-435c-a776-771356fd7c69" containerName="manager" Sep 11 08:20:22.741496 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:22.741310 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="76e423e0-9983-435c-a776-771356fd7c69" containerName="manager" Sep 11 08:20:22.741496 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:22.741325 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d2e611b-3624-425f-b04e-34e8b53f82be" containerName="postgres" Sep 11 08:20:22.741496 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:22.741331 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d2e611b-3624-425f-b04e-34e8b53f82be" containerName="postgres" Sep 11 08:20:22.741496 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:22.741379 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="76e423e0-9983-435c-a776-771356fd7c69" containerName="manager" Sep 11 08:20:22.741496 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:22.741388 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d2e611b-3624-425f-b04e-34e8b53f82be" containerName="postgres" Sep 11 08:20:22.744098 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:22.744079 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fv5tj" Sep 11 08:20:22.746506 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:22.746474 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 11 08:20:22.746506 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:22.746475 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 11 08:20:22.747054 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:22.747038 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-5ndwn\"" Sep 11 08:20:22.764874 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:22.764850 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-fv5tj"] Sep 11 08:20:22.788066 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:22.788042 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w74gv\" (UniqueName: \"kubernetes.io/projected/d0728cb4-9ac5-4d37-9415-ba38caa0a3c9-kube-api-access-w74gv\") pod \"feast-operator-controller-manager-689b877b4b-fv5tj\" (UID: \"d0728cb4-9ac5-4d37-9415-ba38caa0a3c9\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fv5tj" Sep 11 08:20:22.888582 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:22.888556 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w74gv\" (UniqueName: \"kubernetes.io/projected/d0728cb4-9ac5-4d37-9415-ba38caa0a3c9-kube-api-access-w74gv\") pod \"feast-operator-controller-manager-689b877b4b-fv5tj\" (UID: \"d0728cb4-9ac5-4d37-9415-ba38caa0a3c9\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fv5tj" Sep 11 08:20:22.902362 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:22.902336 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w74gv\" (UniqueName: \"kubernetes.io/projected/d0728cb4-9ac5-4d37-9415-ba38caa0a3c9-kube-api-access-w74gv\") pod \"feast-operator-controller-manager-689b877b4b-fv5tj\" (UID: \"d0728cb4-9ac5-4d37-9415-ba38caa0a3c9\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fv5tj" Sep 11 08:20:23.054345 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:23.054293 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fv5tj" Sep 11 08:20:23.324636 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:23.324612 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-fv5tj"] Sep 11 08:20:23.326968 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:20:23.326926 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0728cb4_9ac5_4d37_9415_ba38caa0a3c9.slice/crio-d39edf3aff1ff188c0fb019dfb3f15e55a8f18d14cfddfb9d0f30aeeabbdd289 WatchSource:0}: Error finding container d39edf3aff1ff188c0fb019dfb3f15e55a8f18d14cfddfb9d0f30aeeabbdd289: Status 404 returned error can't find the container with id d39edf3aff1ff188c0fb019dfb3f15e55a8f18d14cfddfb9d0f30aeeabbdd289 Sep 11 08:20:23.328735 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:23.328717 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 11 08:20:23.543782 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:23.543759 2557 scope.go:117] "RemoveContainer" containerID="7fdce5534c56ca66e9842af76a5686e01a75cd8bf49ac933cd02b9aba5e55e6d" Sep 11 08:20:24.178174 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:24.178131 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fv5tj" event={"ID":"d0728cb4-9ac5-4d37-9415-ba38caa0a3c9","Type":"ContainerStarted","Data":"d39edf3aff1ff188c0fb019dfb3f15e55a8f18d14cfddfb9d0f30aeeabbdd289"} Sep 11 08:20:26.187425 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:26.187394 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fv5tj" event={"ID":"d0728cb4-9ac5-4d37-9415-ba38caa0a3c9","Type":"ContainerStarted","Data":"cb0a994991234f404961d90b80f6156b2b8ee0db7be859de58c9ccfcf9bbb9e2"} Sep 11 08:20:26.189485 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:26.187530 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fv5tj" Sep 11 08:20:26.211066 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:26.211018 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fv5tj" podStartSLOduration=1.84067157 podStartE2EDuration="4.211003308s" totalImagesPullingTime="2.370331738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:20:26.210044322 +0000 UTC m=+1263.215761983" watchObservedRunningTime="2026-09-11 08:20:26.211003308 +0000 UTC m=+1263.216720989" Sep 11 08:20:37.193389 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:37.193359 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fv5tj" Sep 11 08:20:50.818379 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.818348 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4"] Sep 11 08:20:50.821909 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.821883 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:20:50.824461 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.824429 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 11 08:20:50.824576 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.824434 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 11 08:20:50.824576 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.824500 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 11 08:20:50.824576 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.824555 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-8qgff\"" Sep 11 08:20:50.824837 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.824821 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 11 08:20:50.824936 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.824821 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 11 08:20:50.824936 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.824827 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 11 08:20:50.836741 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.836723 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4"] Sep 11 08:20:50.872593 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.872574 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tfkk4\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:20:50.872710 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.872604 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tfkk4\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:20:50.872710 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.872623 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tfkk4\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:20:50.872822 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.872723 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tfkk4\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:20:50.872879 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.872819 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nckjd\" (UniqueName: \"kubernetes.io/projected/147ff3d9-9679-4e4b-a1d4-03e21570f58d-kube-api-access-nckjd\") pod \"feast-simple-feast-setup-57576b8fcf-tfkk4\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:20:50.872879 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.872861 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/147ff3d9-9679-4e4b-a1d4-03e21570f58d-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-tfkk4\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:20:50.974009 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.973987 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nckjd\" (UniqueName: \"kubernetes.io/projected/147ff3d9-9679-4e4b-a1d4-03e21570f58d-kube-api-access-nckjd\") pod \"feast-simple-feast-setup-57576b8fcf-tfkk4\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:20:50.974119 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.974029 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/147ff3d9-9679-4e4b-a1d4-03e21570f58d-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-tfkk4\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:20:50.974119 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.974058 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tfkk4\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:20:50.974119 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.974109 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tfkk4\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:20:50.974281 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.974134 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tfkk4\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:20:50.974281 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.974169 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tfkk4\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:20:50.974383 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:20:50.974286 2557 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 11 08:20:50.974383 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:20:50.974362 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-registry-tls podName:147ff3d9-9679-4e4b-a1d4-03e21570f58d nodeName:}" failed. No retries permitted until 2026-09-11 08:20:51.474340988 +0000 UTC m=+1288.480058643 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-registry-tls") pod "feast-simple-feast-setup-57576b8fcf-tfkk4" (UID: "147ff3d9-9679-4e4b-a1d4-03e21570f58d") : secret "feast-simple-feast-setup-registry-tls" not found Sep 11 08:20:50.974572 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.974546 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/147ff3d9-9679-4e4b-a1d4-03e21570f58d-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-tfkk4\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:20:50.976561 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.976541 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tfkk4\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:20:50.976651 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.976597 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tfkk4\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:20:50.976724 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.976703 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tfkk4\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:20:50.982978 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:50.982957 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nckjd\" (UniqueName: \"kubernetes.io/projected/147ff3d9-9679-4e4b-a1d4-03e21570f58d-kube-api-access-nckjd\") pod \"feast-simple-feast-setup-57576b8fcf-tfkk4\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:20:51.478163 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:51.478132 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tfkk4\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:20:51.480384 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:51.480365 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-tfkk4\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:20:51.743925 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:51.743841 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:20:52.020574 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:52.020548 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4"] Sep 11 08:20:52.022117 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:20:52.022090 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod147ff3d9_9679_4e4b_a1d4_03e21570f58d.slice/crio-87c5e1305feb52924e29a9f1e1cac4068801ca2ff587ed4da75e48d09c18bb1a WatchSource:0}: Error finding container 87c5e1305feb52924e29a9f1e1cac4068801ca2ff587ed4da75e48d09c18bb1a: Status 404 returned error can't find the container with id 87c5e1305feb52924e29a9f1e1cac4068801ca2ff587ed4da75e48d09c18bb1a Sep 11 08:20:52.279359 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:20:52.279300 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" event={"ID":"147ff3d9-9679-4e4b-a1d4-03e21570f58d","Type":"ContainerStarted","Data":"87c5e1305feb52924e29a9f1e1cac4068801ca2ff587ed4da75e48d09c18bb1a"} Sep 11 08:21:05.344709 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:05.344666 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" event={"ID":"147ff3d9-9679-4e4b-a1d4-03e21570f58d","Type":"ContainerStarted","Data":"5c043c9cb34e6730322e5338b150bd697a94a4c3522233b9c1a0e25db5b20d53"} Sep 11 08:21:10.364580 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:10.364547 2557 generic.go:356] "Generic (PLEG): container finished" podID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerID="5c043c9cb34e6730322e5338b150bd697a94a4c3522233b9c1a0e25db5b20d53" exitCode=0 Sep 11 08:21:10.364954 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:10.364592 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" event={"ID":"147ff3d9-9679-4e4b-a1d4-03e21570f58d","Type":"ContainerDied","Data":"5c043c9cb34e6730322e5338b150bd697a94a4c3522233b9c1a0e25db5b20d53"} Sep 11 08:21:11.370667 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:11.370634 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" event={"ID":"147ff3d9-9679-4e4b-a1d4-03e21570f58d","Type":"ContainerStarted","Data":"c19db7159244da6f602dbb3b0a3c4e1be00e7743cbbc0e9497f34a86ca9ab862"} Sep 11 08:21:16.390136 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:16.390102 2557 generic.go:356] "Generic (PLEG): container finished" podID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerID="c19db7159244da6f602dbb3b0a3c4e1be00e7743cbbc0e9497f34a86ca9ab862" exitCode=0 Sep 11 08:21:16.390489 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:16.390141 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" event={"ID":"147ff3d9-9679-4e4b-a1d4-03e21570f58d","Type":"ContainerDied","Data":"c19db7159244da6f602dbb3b0a3c4e1be00e7743cbbc0e9497f34a86ca9ab862"} Sep 11 08:21:17.400124 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:17.400080 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" event={"ID":"147ff3d9-9679-4e4b-a1d4-03e21570f58d","Type":"ContainerStarted","Data":"d5850a41ef258d15e2d610d147e1cce6ea4230bc834c7ed3ef5270e1fbbfe5ab"} Sep 11 08:21:17.400576 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:17.400144 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" event={"ID":"147ff3d9-9679-4e4b-a1d4-03e21570f58d","Type":"ContainerStarted","Data":"9b476b4bff403697f17397fc3b2b0921ff667f917a0d65de4b2d6a517a52da2b"} Sep 11 08:21:17.400576 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:17.400168 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" event={"ID":"147ff3d9-9679-4e4b-a1d4-03e21570f58d","Type":"ContainerStarted","Data":"c17ac1979d987204459579049e559ca3b39323fcf1ddb221e8b1ed15a52452f1"} Sep 11 08:21:17.400576 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:17.400188 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" event={"ID":"147ff3d9-9679-4e4b-a1d4-03e21570f58d","Type":"ContainerStarted","Data":"a24227569acf5e9ddefba0d6f98a9df3ef5fc986a86e600598f529460968cda3"} Sep 11 08:21:17.427196 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:17.427140 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" podStartSLOduration=14.342722745 podStartE2EDuration="27.427118958s" totalImagesPullingTime="13.084396213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:21:17.426315418 +0000 UTC m=+1314.432033113" watchObservedRunningTime="2026-09-11 08:21:17.427118958 +0000 UTC m=+1314.432836617" Sep 11 08:21:18.744973 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:18.744920 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:21:18.745424 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:18.744990 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:21:18.745424 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:18.745009 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:21:18.745424 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:18.745039 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:21:18.747207 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:18.747055 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Sep 11 08:21:18.747207 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:18.747075 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.28:6571: connect: connection refused" Sep 11 08:21:18.747207 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:18.747082 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.28:8816: connect: connection refused" Sep 11 08:21:18.747207 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:18.747137 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Sep 11 08:21:21.744423 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:21.744380 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:21:21.745084 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:21.745053 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:21:21.745169 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:21.744584 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Sep 11 08:21:21.745169 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:21.745099 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:21:21.745169 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:21.745118 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:21:21.745169 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:21.744677 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.28:8816: connect: connection refused" Sep 11 08:21:21.745169 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:21.744623 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.28:6571: connect: connection refused" Sep 11 08:21:21.745431 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:21.744628 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Sep 11 08:21:24.745608 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:24.745580 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:21:24.746029 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:24.745706 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:21:24.746029 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:24.745953 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:21:24.746597 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:24.746572 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:21:24.746687 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:24.746646 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:21:24.746835 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:24.746820 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:21:24.750931 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:24.750884 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:21:25.442396 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:21:25.442369 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:22:27.926363 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:27.926334 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4"] Sep 11 08:22:27.926886 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:27.926566 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="registry" containerID="cri-o://a24227569acf5e9ddefba0d6f98a9df3ef5fc986a86e600598f529460968cda3" gracePeriod=30 Sep 11 08:22:27.926886 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:27.926682 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="online" containerID="cri-o://c17ac1979d987204459579049e559ca3b39323fcf1ddb221e8b1ed15a52452f1" gracePeriod=30 Sep 11 08:22:27.926886 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:27.926622 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="ui" containerID="cri-o://d5850a41ef258d15e2d610d147e1cce6ea4230bc834c7ed3ef5270e1fbbfe5ab" gracePeriod=30 Sep 11 08:22:27.926886 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:27.926623 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="offline" containerID="cri-o://9b476b4bff403697f17397fc3b2b0921ff667f917a0d65de4b2d6a517a52da2b" gracePeriod=30 Sep 11 08:22:28.369892 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.369807 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j"] Sep 11 08:22:28.373105 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.373088 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:28.376115 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.376090 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-l4kq9\"" Sep 11 08:22:28.389764 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.389735 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j"] Sep 11 08:22:28.475793 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.475759 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfrzs\" (UniqueName: \"kubernetes.io/projected/5ac530b0-45c2-4de2-bed9-3527de2ff310-kube-api-access-rfrzs\") pod \"feast-simple-feast-setup-57576b8fcf-hrj6j\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:28.475929 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.475811 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hrj6j\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:28.475929 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.475860 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hrj6j\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:28.475929 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.475911 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hrj6j\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:28.476035 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.475945 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hrj6j\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:28.476069 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.476037 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5ac530b0-45c2-4de2-bed9-3527de2ff310-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-hrj6j\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:28.576720 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.576689 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfrzs\" (UniqueName: \"kubernetes.io/projected/5ac530b0-45c2-4de2-bed9-3527de2ff310-kube-api-access-rfrzs\") pod \"feast-simple-feast-setup-57576b8fcf-hrj6j\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:28.576875 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.576729 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hrj6j\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:28.576875 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.576758 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hrj6j\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:28.576875 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.576788 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hrj6j\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:28.576875 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.576817 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hrj6j\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:28.576875 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.576872 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5ac530b0-45c2-4de2-bed9-3527de2ff310-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-hrj6j\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:28.577145 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:22:28.576880 2557 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 11 08:22:28.577145 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:22:28.577000 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-ui-tls podName:5ac530b0-45c2-4de2-bed9-3527de2ff310 nodeName:}" failed. No retries permitted until 2026-09-11 08:22:29.076963744 +0000 UTC m=+1386.082681402 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-ui-tls") pod "feast-simple-feast-setup-57576b8fcf-hrj6j" (UID: "5ac530b0-45c2-4de2-bed9-3527de2ff310") : secret "feast-simple-feast-setup-ui-tls" not found Sep 11 08:22:28.577327 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.577308 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5ac530b0-45c2-4de2-bed9-3527de2ff310-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-hrj6j\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:28.579285 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.579265 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hrj6j\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:28.579378 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.579280 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hrj6j\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:28.579378 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.579269 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hrj6j\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:28.588132 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.588109 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfrzs\" (UniqueName: \"kubernetes.io/projected/5ac530b0-45c2-4de2-bed9-3527de2ff310-kube-api-access-rfrzs\") pod \"feast-simple-feast-setup-57576b8fcf-hrj6j\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:28.679951 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.679923 2557 generic.go:356] "Generic (PLEG): container finished" podID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerID="d5850a41ef258d15e2d610d147e1cce6ea4230bc834c7ed3ef5270e1fbbfe5ab" exitCode=0 Sep 11 08:22:28.680043 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:28.679928 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" event={"ID":"147ff3d9-9679-4e4b-a1d4-03e21570f58d","Type":"ContainerDied","Data":"d5850a41ef258d15e2d610d147e1cce6ea4230bc834c7ed3ef5270e1fbbfe5ab"} Sep 11 08:22:29.081088 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:29.081037 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hrj6j\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:29.083226 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:29.083207 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-hrj6j\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:29.282702 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:29.282677 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:29.559658 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:29.559629 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j"] Sep 11 08:22:29.560972 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:22:29.560949 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ac530b0_45c2_4de2_bed9_3527de2ff310.slice/crio-da57e48292884c5bcd7a67948c229e6a717532edb5c5923e3774742ead1275b4 WatchSource:0}: Error finding container da57e48292884c5bcd7a67948c229e6a717532edb5c5923e3774742ead1275b4: Status 404 returned error can't find the container with id da57e48292884c5bcd7a67948c229e6a717532edb5c5923e3774742ead1275b4 Sep 11 08:22:29.684887 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:29.684857 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" event={"ID":"5ac530b0-45c2-4de2-bed9-3527de2ff310","Type":"ContainerStarted","Data":"a604143d9e53e64aca5e367086e8cc9e175801add8efba25225bf289dbbf8aeb"} Sep 11 08:22:29.690535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:29.685143 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" event={"ID":"5ac530b0-45c2-4de2-bed9-3527de2ff310","Type":"ContainerStarted","Data":"da57e48292884c5bcd7a67948c229e6a717532edb5c5923e3774742ead1275b4"} Sep 11 08:22:29.694029 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:29.694009 2557 generic.go:356] "Generic (PLEG): container finished" podID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerID="c17ac1979d987204459579049e559ca3b39323fcf1ddb221e8b1ed15a52452f1" exitCode=0 Sep 11 08:22:29.694119 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:29.694047 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" event={"ID":"147ff3d9-9679-4e4b-a1d4-03e21570f58d","Type":"ContainerDied","Data":"c17ac1979d987204459579049e559ca3b39323fcf1ddb221e8b1ed15a52452f1"} Sep 11 08:22:34.714175 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:34.714149 2557 generic.go:356] "Generic (PLEG): container finished" podID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerID="a604143d9e53e64aca5e367086e8cc9e175801add8efba25225bf289dbbf8aeb" exitCode=0 Sep 11 08:22:34.714527 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:34.714218 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" event={"ID":"5ac530b0-45c2-4de2-bed9-3527de2ff310","Type":"ContainerDied","Data":"a604143d9e53e64aca5e367086e8cc9e175801add8efba25225bf289dbbf8aeb"} Sep 11 08:22:34.746844 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:34.746799 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Sep 11 08:22:35.439752 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:35.439704 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Sep 11 08:22:35.719311 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:35.719226 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" event={"ID":"5ac530b0-45c2-4de2-bed9-3527de2ff310","Type":"ContainerStarted","Data":"00894be7da28186be42596fd5568389572dae7bee8a4d609ac7c017511137681"} Sep 11 08:22:40.740574 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:40.740544 2557 generic.go:356] "Generic (PLEG): container finished" podID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerID="00894be7da28186be42596fd5568389572dae7bee8a4d609ac7c017511137681" exitCode=0 Sep 11 08:22:40.740872 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:40.740581 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" event={"ID":"5ac530b0-45c2-4de2-bed9-3527de2ff310","Type":"ContainerDied","Data":"00894be7da28186be42596fd5568389572dae7bee8a4d609ac7c017511137681"} Sep 11 08:22:41.751189 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:41.751145 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" event={"ID":"5ac530b0-45c2-4de2-bed9-3527de2ff310","Type":"ContainerStarted","Data":"e9b16a2ceb6e114e568541639c461da61f2054b4644ac2d31276c222a219f6c7"} Sep 11 08:22:41.751655 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:41.751212 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" event={"ID":"5ac530b0-45c2-4de2-bed9-3527de2ff310","Type":"ContainerStarted","Data":"a602276dea343185a89eab0b94d135088aec86dd0b02a9032ce30ed783f0fba0"} Sep 11 08:22:41.751655 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:41.751235 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" event={"ID":"5ac530b0-45c2-4de2-bed9-3527de2ff310","Type":"ContainerStarted","Data":"23480169aafd65c8ceb7185b61b5eb5e5aeab5f68ce9448fd9148c3ee52c35eb"} Sep 11 08:22:41.751655 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:41.751253 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" event={"ID":"5ac530b0-45c2-4de2-bed9-3527de2ff310","Type":"ContainerStarted","Data":"0ec607bd908d4028be9f2cc2a573654cfafe442eabb0374f3297e050c487b7bb"} Sep 11 08:22:41.781726 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:41.781600 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" podStartSLOduration=13.781579346000001 podStartE2EDuration="13.781579346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:22:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:22:41.78059152 +0000 UTC m=+1398.786309203" watchObservedRunningTime="2026-09-11 08:22:41.781579346 +0000 UTC m=+1398.787297037" Sep 11 08:22:44.283418 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:44.283373 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:44.283924 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:44.283439 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:44.283924 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:44.283458 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:44.283924 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:44.283484 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:44.285630 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:44.285590 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.29:8816: connect: connection refused" Sep 11 08:22:44.285946 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:44.285621 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Sep 11 08:22:44.285946 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:44.285621 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.29:6571: connect: connection refused" Sep 11 08:22:44.285946 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:44.285683 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Sep 11 08:22:44.746751 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:44.746519 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Sep 11 08:22:45.440377 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:45.440331 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Sep 11 08:22:47.283742 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:47.283675 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.29:6571: connect: connection refused" Sep 11 08:22:47.283742 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:47.283690 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.29:8816: connect: connection refused" Sep 11 08:22:47.284302 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:47.283689 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Sep 11 08:22:47.284302 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:47.284117 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Sep 11 08:22:49.283032 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:49.282997 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:49.283388 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:49.283043 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:49.283388 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:49.283057 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:49.283388 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:49.283069 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:50.283730 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:50.283696 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:50.284383 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:50.284349 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:50.284530 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:50.284491 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:50.284596 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:50.284576 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:50.289775 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:50.289755 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:50.800081 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:50.800052 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:50.800456 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:50.800435 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:50.803152 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:50.803132 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:22:54.747213 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:54.747167 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Sep 11 08:22:54.747743 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:54.747320 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:22:55.439523 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:55.439487 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Sep 11 08:22:55.439652 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:55.439630 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:22:58.572205 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.572182 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:22:58.715487 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.715397 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-ui-tls\") pod \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " Sep 11 08:22:58.715487 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.715455 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/147ff3d9-9679-4e4b-a1d4-03e21570f58d-feast-data\") pod \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " Sep 11 08:22:58.715699 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.715497 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-registry-tls\") pod \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " Sep 11 08:22:58.715699 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.715523 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nckjd\" (UniqueName: \"kubernetes.io/projected/147ff3d9-9679-4e4b-a1d4-03e21570f58d-kube-api-access-nckjd\") pod \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " Sep 11 08:22:58.715699 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.715560 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-offline-tls\") pod \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " Sep 11 08:22:58.715699 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.715590 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-online-tls\") pod \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\" (UID: \"147ff3d9-9679-4e4b-a1d4-03e21570f58d\") " Sep 11 08:22:58.716087 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.716053 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/147ff3d9-9679-4e4b-a1d4-03e21570f58d-feast-data" (OuterVolumeSpecName: "feast-data") pod "147ff3d9-9679-4e4b-a1d4-03e21570f58d" (UID: "147ff3d9-9679-4e4b-a1d4-03e21570f58d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 08:22:58.717828 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.717802 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-online-tls" (OuterVolumeSpecName: "online-tls") pod "147ff3d9-9679-4e4b-a1d4-03e21570f58d" (UID: "147ff3d9-9679-4e4b-a1d4-03e21570f58d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:22:58.717969 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.717878 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "147ff3d9-9679-4e4b-a1d4-03e21570f58d" (UID: "147ff3d9-9679-4e4b-a1d4-03e21570f58d"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:22:58.717969 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.717940 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/147ff3d9-9679-4e4b-a1d4-03e21570f58d-kube-api-access-nckjd" (OuterVolumeSpecName: "kube-api-access-nckjd") pod "147ff3d9-9679-4e4b-a1d4-03e21570f58d" (UID: "147ff3d9-9679-4e4b-a1d4-03e21570f58d"). InnerVolumeSpecName "kube-api-access-nckjd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:22:58.718081 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.718032 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "147ff3d9-9679-4e4b-a1d4-03e21570f58d" (UID: "147ff3d9-9679-4e4b-a1d4-03e21570f58d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:22:58.718081 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.718052 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "147ff3d9-9679-4e4b-a1d4-03e21570f58d" (UID: "147ff3d9-9679-4e4b-a1d4-03e21570f58d"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:22:58.816049 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.816027 2557 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-offline-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:22:58.816049 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.816045 2557 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-online-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:22:58.816170 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.816054 2557 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-ui-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:22:58.816170 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.816062 2557 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/147ff3d9-9679-4e4b-a1d4-03e21570f58d-feast-data\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:22:58.816170 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.816070 2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/147ff3d9-9679-4e4b-a1d4-03e21570f58d-registry-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:22:58.816170 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.816079 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nckjd\" (UniqueName: \"kubernetes.io/projected/147ff3d9-9679-4e4b-a1d4-03e21570f58d-kube-api-access-nckjd\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:22:58.833923 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.833882 2557 generic.go:356] "Generic (PLEG): container finished" podID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerID="9b476b4bff403697f17397fc3b2b0921ff667f917a0d65de4b2d6a517a52da2b" exitCode=137 Sep 11 08:22:58.834038 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.833923 2557 generic.go:356] "Generic (PLEG): container finished" podID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerID="a24227569acf5e9ddefba0d6f98a9df3ef5fc986a86e600598f529460968cda3" exitCode=137 Sep 11 08:22:58.834038 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.833969 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" event={"ID":"147ff3d9-9679-4e4b-a1d4-03e21570f58d","Type":"ContainerDied","Data":"9b476b4bff403697f17397fc3b2b0921ff667f917a0d65de4b2d6a517a52da2b"} Sep 11 08:22:58.834038 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.834025 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" event={"ID":"147ff3d9-9679-4e4b-a1d4-03e21570f58d","Type":"ContainerDied","Data":"a24227569acf5e9ddefba0d6f98a9df3ef5fc986a86e600598f529460968cda3"} Sep 11 08:22:58.834203 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.834046 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" event={"ID":"147ff3d9-9679-4e4b-a1d4-03e21570f58d","Type":"ContainerDied","Data":"87c5e1305feb52924e29a9f1e1cac4068801ca2ff587ed4da75e48d09c18bb1a"} Sep 11 08:22:58.834203 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.833979 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4" Sep 11 08:22:58.834203 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.834073 2557 scope.go:117] "RemoveContainer" containerID="d5850a41ef258d15e2d610d147e1cce6ea4230bc834c7ed3ef5270e1fbbfe5ab" Sep 11 08:22:58.841850 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.841794 2557 scope.go:117] "RemoveContainer" containerID="9b476b4bff403697f17397fc3b2b0921ff667f917a0d65de4b2d6a517a52da2b" Sep 11 08:22:58.848973 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.848957 2557 scope.go:117] "RemoveContainer" containerID="c17ac1979d987204459579049e559ca3b39323fcf1ddb221e8b1ed15a52452f1" Sep 11 08:22:58.856091 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.856077 2557 scope.go:117] "RemoveContainer" containerID="a24227569acf5e9ddefba0d6f98a9df3ef5fc986a86e600598f529460968cda3" Sep 11 08:22:58.859995 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.859975 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4"] Sep 11 08:22:58.863228 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.863211 2557 scope.go:117] "RemoveContainer" containerID="c19db7159244da6f602dbb3b0a3c4e1be00e7743cbbc0e9497f34a86ca9ab862" Sep 11 08:22:58.867130 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.867109 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-tfkk4"] Sep 11 08:22:58.870249 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.870234 2557 scope.go:117] "RemoveContainer" containerID="5c043c9cb34e6730322e5338b150bd697a94a4c3522233b9c1a0e25db5b20d53" Sep 11 08:22:58.876923 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.876894 2557 scope.go:117] "RemoveContainer" containerID="d5850a41ef258d15e2d610d147e1cce6ea4230bc834c7ed3ef5270e1fbbfe5ab" Sep 11 08:22:58.877174 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:22:58.877156 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5850a41ef258d15e2d610d147e1cce6ea4230bc834c7ed3ef5270e1fbbfe5ab\": container with ID starting with d5850a41ef258d15e2d610d147e1cce6ea4230bc834c7ed3ef5270e1fbbfe5ab not found: ID does not exist" containerID="d5850a41ef258d15e2d610d147e1cce6ea4230bc834c7ed3ef5270e1fbbfe5ab" Sep 11 08:22:58.877246 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.877186 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5850a41ef258d15e2d610d147e1cce6ea4230bc834c7ed3ef5270e1fbbfe5ab"} err="failed to get container status \"d5850a41ef258d15e2d610d147e1cce6ea4230bc834c7ed3ef5270e1fbbfe5ab\": rpc error: code = NotFound desc = could not find container \"d5850a41ef258d15e2d610d147e1cce6ea4230bc834c7ed3ef5270e1fbbfe5ab\": container with ID starting with d5850a41ef258d15e2d610d147e1cce6ea4230bc834c7ed3ef5270e1fbbfe5ab not found: ID does not exist" Sep 11 08:22:58.877246 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.877209 2557 scope.go:117] "RemoveContainer" containerID="9b476b4bff403697f17397fc3b2b0921ff667f917a0d65de4b2d6a517a52da2b" Sep 11 08:22:58.877457 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:22:58.877442 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b476b4bff403697f17397fc3b2b0921ff667f917a0d65de4b2d6a517a52da2b\": container with ID starting with 9b476b4bff403697f17397fc3b2b0921ff667f917a0d65de4b2d6a517a52da2b not found: ID does not exist" containerID="9b476b4bff403697f17397fc3b2b0921ff667f917a0d65de4b2d6a517a52da2b" Sep 11 08:22:58.877500 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.877463 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b476b4bff403697f17397fc3b2b0921ff667f917a0d65de4b2d6a517a52da2b"} err="failed to get container status \"9b476b4bff403697f17397fc3b2b0921ff667f917a0d65de4b2d6a517a52da2b\": rpc error: code = NotFound desc = could not find container \"9b476b4bff403697f17397fc3b2b0921ff667f917a0d65de4b2d6a517a52da2b\": container with ID starting with 9b476b4bff403697f17397fc3b2b0921ff667f917a0d65de4b2d6a517a52da2b not found: ID does not exist" Sep 11 08:22:58.877500 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.877476 2557 scope.go:117] "RemoveContainer" containerID="c17ac1979d987204459579049e559ca3b39323fcf1ddb221e8b1ed15a52452f1" Sep 11 08:22:58.877680 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:22:58.877661 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c17ac1979d987204459579049e559ca3b39323fcf1ddb221e8b1ed15a52452f1\": container with ID starting with c17ac1979d987204459579049e559ca3b39323fcf1ddb221e8b1ed15a52452f1 not found: ID does not exist" containerID="c17ac1979d987204459579049e559ca3b39323fcf1ddb221e8b1ed15a52452f1" Sep 11 08:22:58.877719 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.877686 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c17ac1979d987204459579049e559ca3b39323fcf1ddb221e8b1ed15a52452f1"} err="failed to get container status \"c17ac1979d987204459579049e559ca3b39323fcf1ddb221e8b1ed15a52452f1\": rpc error: code = NotFound desc = could not find container \"c17ac1979d987204459579049e559ca3b39323fcf1ddb221e8b1ed15a52452f1\": container with ID starting with c17ac1979d987204459579049e559ca3b39323fcf1ddb221e8b1ed15a52452f1 not found: ID does not exist" Sep 11 08:22:58.877719 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.877710 2557 scope.go:117] "RemoveContainer" containerID="a24227569acf5e9ddefba0d6f98a9df3ef5fc986a86e600598f529460968cda3" Sep 11 08:22:58.877937 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:22:58.877920 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a24227569acf5e9ddefba0d6f98a9df3ef5fc986a86e600598f529460968cda3\": container with ID starting with a24227569acf5e9ddefba0d6f98a9df3ef5fc986a86e600598f529460968cda3 not found: ID does not exist" containerID="a24227569acf5e9ddefba0d6f98a9df3ef5fc986a86e600598f529460968cda3" Sep 11 08:22:58.878006 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.877944 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a24227569acf5e9ddefba0d6f98a9df3ef5fc986a86e600598f529460968cda3"} err="failed to get container status \"a24227569acf5e9ddefba0d6f98a9df3ef5fc986a86e600598f529460968cda3\": rpc error: code = NotFound desc = could not find container \"a24227569acf5e9ddefba0d6f98a9df3ef5fc986a86e600598f529460968cda3\": container with ID starting with a24227569acf5e9ddefba0d6f98a9df3ef5fc986a86e600598f529460968cda3 not found: ID does not exist" Sep 11 08:22:58.878006 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.877963 2557 scope.go:117] "RemoveContainer" containerID="c19db7159244da6f602dbb3b0a3c4e1be00e7743cbbc0e9497f34a86ca9ab862" Sep 11 08:22:58.878215 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:22:58.878200 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c19db7159244da6f602dbb3b0a3c4e1be00e7743cbbc0e9497f34a86ca9ab862\": container with ID starting with c19db7159244da6f602dbb3b0a3c4e1be00e7743cbbc0e9497f34a86ca9ab862 not found: ID does not exist" containerID="c19db7159244da6f602dbb3b0a3c4e1be00e7743cbbc0e9497f34a86ca9ab862" Sep 11 08:22:58.878258 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.878218 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c19db7159244da6f602dbb3b0a3c4e1be00e7743cbbc0e9497f34a86ca9ab862"} err="failed to get container status \"c19db7159244da6f602dbb3b0a3c4e1be00e7743cbbc0e9497f34a86ca9ab862\": rpc error: code = NotFound desc = could not find container \"c19db7159244da6f602dbb3b0a3c4e1be00e7743cbbc0e9497f34a86ca9ab862\": container with ID starting with c19db7159244da6f602dbb3b0a3c4e1be00e7743cbbc0e9497f34a86ca9ab862 not found: ID does not exist" Sep 11 08:22:58.878258 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.878232 2557 scope.go:117] "RemoveContainer" containerID="5c043c9cb34e6730322e5338b150bd697a94a4c3522233b9c1a0e25db5b20d53" Sep 11 08:22:58.878468 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:22:58.878451 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c043c9cb34e6730322e5338b150bd697a94a4c3522233b9c1a0e25db5b20d53\": container with ID starting with 5c043c9cb34e6730322e5338b150bd697a94a4c3522233b9c1a0e25db5b20d53 not found: ID does not exist" containerID="5c043c9cb34e6730322e5338b150bd697a94a4c3522233b9c1a0e25db5b20d53" Sep 11 08:22:58.878530 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.878476 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c043c9cb34e6730322e5338b150bd697a94a4c3522233b9c1a0e25db5b20d53"} err="failed to get container status \"5c043c9cb34e6730322e5338b150bd697a94a4c3522233b9c1a0e25db5b20d53\": rpc error: code = NotFound desc = could not find container \"5c043c9cb34e6730322e5338b150bd697a94a4c3522233b9c1a0e25db5b20d53\": container with ID starting with 5c043c9cb34e6730322e5338b150bd697a94a4c3522233b9c1a0e25db5b20d53 not found: ID does not exist" Sep 11 08:22:58.878530 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.878496 2557 scope.go:117] "RemoveContainer" containerID="d5850a41ef258d15e2d610d147e1cce6ea4230bc834c7ed3ef5270e1fbbfe5ab" Sep 11 08:22:58.878696 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.878679 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5850a41ef258d15e2d610d147e1cce6ea4230bc834c7ed3ef5270e1fbbfe5ab"} err="failed to get container status \"d5850a41ef258d15e2d610d147e1cce6ea4230bc834c7ed3ef5270e1fbbfe5ab\": rpc error: code = NotFound desc = could not find container \"d5850a41ef258d15e2d610d147e1cce6ea4230bc834c7ed3ef5270e1fbbfe5ab\": container with ID starting with d5850a41ef258d15e2d610d147e1cce6ea4230bc834c7ed3ef5270e1fbbfe5ab not found: ID does not exist" Sep 11 08:22:58.878745 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.878697 2557 scope.go:117] "RemoveContainer" containerID="9b476b4bff403697f17397fc3b2b0921ff667f917a0d65de4b2d6a517a52da2b" Sep 11 08:22:58.878912 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.878879 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b476b4bff403697f17397fc3b2b0921ff667f917a0d65de4b2d6a517a52da2b"} err="failed to get container status \"9b476b4bff403697f17397fc3b2b0921ff667f917a0d65de4b2d6a517a52da2b\": rpc error: code = NotFound desc = could not find container \"9b476b4bff403697f17397fc3b2b0921ff667f917a0d65de4b2d6a517a52da2b\": container with ID starting with 9b476b4bff403697f17397fc3b2b0921ff667f917a0d65de4b2d6a517a52da2b not found: ID does not exist" Sep 11 08:22:58.878957 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.878919 2557 scope.go:117] "RemoveContainer" containerID="c17ac1979d987204459579049e559ca3b39323fcf1ddb221e8b1ed15a52452f1" Sep 11 08:22:58.879127 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.879112 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c17ac1979d987204459579049e559ca3b39323fcf1ddb221e8b1ed15a52452f1"} err="failed to get container status \"c17ac1979d987204459579049e559ca3b39323fcf1ddb221e8b1ed15a52452f1\": rpc error: code = NotFound desc = could not find container \"c17ac1979d987204459579049e559ca3b39323fcf1ddb221e8b1ed15a52452f1\": container with ID starting with c17ac1979d987204459579049e559ca3b39323fcf1ddb221e8b1ed15a52452f1 not found: ID does not exist" Sep 11 08:22:58.879174 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.879127 2557 scope.go:117] "RemoveContainer" containerID="a24227569acf5e9ddefba0d6f98a9df3ef5fc986a86e600598f529460968cda3" Sep 11 08:22:58.879309 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.879293 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a24227569acf5e9ddefba0d6f98a9df3ef5fc986a86e600598f529460968cda3"} err="failed to get container status \"a24227569acf5e9ddefba0d6f98a9df3ef5fc986a86e600598f529460968cda3\": rpc error: code = NotFound desc = could not find container \"a24227569acf5e9ddefba0d6f98a9df3ef5fc986a86e600598f529460968cda3\": container with ID starting with a24227569acf5e9ddefba0d6f98a9df3ef5fc986a86e600598f529460968cda3 not found: ID does not exist" Sep 11 08:22:58.879354 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.879310 2557 scope.go:117] "RemoveContainer" containerID="c19db7159244da6f602dbb3b0a3c4e1be00e7743cbbc0e9497f34a86ca9ab862" Sep 11 08:22:58.879512 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.879495 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c19db7159244da6f602dbb3b0a3c4e1be00e7743cbbc0e9497f34a86ca9ab862"} err="failed to get container status \"c19db7159244da6f602dbb3b0a3c4e1be00e7743cbbc0e9497f34a86ca9ab862\": rpc error: code = NotFound desc = could not find container \"c19db7159244da6f602dbb3b0a3c4e1be00e7743cbbc0e9497f34a86ca9ab862\": container with ID starting with c19db7159244da6f602dbb3b0a3c4e1be00e7743cbbc0e9497f34a86ca9ab862 not found: ID does not exist" Sep 11 08:22:58.879556 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.879513 2557 scope.go:117] "RemoveContainer" containerID="5c043c9cb34e6730322e5338b150bd697a94a4c3522233b9c1a0e25db5b20d53" Sep 11 08:22:58.879729 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:58.879712 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c043c9cb34e6730322e5338b150bd697a94a4c3522233b9c1a0e25db5b20d53"} err="failed to get container status \"5c043c9cb34e6730322e5338b150bd697a94a4c3522233b9c1a0e25db5b20d53\": rpc error: code = NotFound desc = could not find container \"5c043c9cb34e6730322e5338b150bd697a94a4c3522233b9c1a0e25db5b20d53\": container with ID starting with 5c043c9cb34e6730322e5338b150bd697a94a4c3522233b9c1a0e25db5b20d53 not found: ID does not exist" Sep 11 08:22:59.504227 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:22:59.504182 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" path="/var/lib/kubelet/pods/147ff3d9-9679-4e4b-a1d4-03e21570f58d/volumes" Sep 11 08:23:53.356887 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.356853 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g"] Sep 11 08:23:53.357335 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.357151 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="registry" Sep 11 08:23:53.357335 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.357162 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="registry" Sep 11 08:23:53.357335 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.357177 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="feast-init" Sep 11 08:23:53.357335 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.357185 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="feast-init" Sep 11 08:23:53.357335 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.357191 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="online" Sep 11 08:23:53.357335 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.357196 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="online" Sep 11 08:23:53.357335 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.357204 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="feast-apply" Sep 11 08:23:53.357335 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.357210 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="feast-apply" Sep 11 08:23:53.357335 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.357222 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="ui" Sep 11 08:23:53.357335 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.357227 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="ui" Sep 11 08:23:53.357335 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.357235 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="offline" Sep 11 08:23:53.357335 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.357241 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="offline" Sep 11 08:23:53.357335 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.357294 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="ui" Sep 11 08:23:53.357335 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.357303 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="offline" Sep 11 08:23:53.357335 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.357309 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="online" Sep 11 08:23:53.357335 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.357315 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="147ff3d9-9679-4e4b-a1d4-03e21570f58d" containerName="registry" Sep 11 08:23:53.360577 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.360561 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:23:53.362652 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.362628 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 11 08:23:53.362783 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.362660 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 11 08:23:53.362783 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.362675 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 11 08:23:53.362973 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.362958 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 11 08:23:53.363053 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.362989 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 11 08:23:53.363053 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.362998 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-jnjxm\"" Sep 11 08:23:53.363053 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.363001 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 11 08:23:53.371669 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.371651 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g"] Sep 11 08:23:53.476846 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.476819 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a288e67c-e21e-425f-b961-74d507e80755-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4nc9g\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:23:53.476986 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.476854 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hh4t\" (UniqueName: \"kubernetes.io/projected/a288e67c-e21e-425f-b961-74d507e80755-kube-api-access-8hh4t\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4nc9g\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:23:53.476986 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.476884 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a288e67c-e21e-425f-b961-74d507e80755-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4nc9g\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:23:53.476986 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.476951 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a288e67c-e21e-425f-b961-74d507e80755-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4nc9g\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:23:53.477109 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.477001 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a288e67c-e21e-425f-b961-74d507e80755-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4nc9g\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:23:53.477109 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.477038 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a288e67c-e21e-425f-b961-74d507e80755-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4nc9g\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:23:53.577421 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.577400 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a288e67c-e21e-425f-b961-74d507e80755-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4nc9g\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:23:53.577534 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.577431 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a288e67c-e21e-425f-b961-74d507e80755-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4nc9g\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:23:53.577534 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.577478 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a288e67c-e21e-425f-b961-74d507e80755-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4nc9g\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:23:53.577534 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.577505 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hh4t\" (UniqueName: \"kubernetes.io/projected/a288e67c-e21e-425f-b961-74d507e80755-kube-api-access-8hh4t\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4nc9g\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:23:53.577534 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.577531 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a288e67c-e21e-425f-b961-74d507e80755-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4nc9g\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:23:53.577752 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:23:53.577677 2557 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Sep 11 08:23:53.577752 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:23:53.577734 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a288e67c-e21e-425f-b961-74d507e80755-online-tls podName:a288e67c-e21e-425f-b961-74d507e80755 nodeName:}" failed. No retries permitted until 2026-09-11 08:23:54.077716567 +0000 UTC m=+1471.083434207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/a288e67c-e21e-425f-b961-74d507e80755-online-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" (UID: "a288e67c-e21e-425f-b961-74d507e80755") : secret "feast-simple-feast-remote-setup-online-tls" not found Sep 11 08:23:53.577865 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.577753 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a288e67c-e21e-425f-b961-74d507e80755-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4nc9g\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:23:53.577865 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.577840 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a288e67c-e21e-425f-b961-74d507e80755-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4nc9g\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:23:53.578315 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.578299 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a288e67c-e21e-425f-b961-74d507e80755-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4nc9g\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:23:53.579906 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.579872 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a288e67c-e21e-425f-b961-74d507e80755-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4nc9g\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:23:53.580057 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.580042 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a288e67c-e21e-425f-b961-74d507e80755-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4nc9g\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:23:53.586217 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:53.586199 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hh4t\" (UniqueName: \"kubernetes.io/projected/a288e67c-e21e-425f-b961-74d507e80755-kube-api-access-8hh4t\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4nc9g\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:23:54.081105 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:54.081072 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a288e67c-e21e-425f-b961-74d507e80755-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4nc9g\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:23:54.083452 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:54.083429 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a288e67c-e21e-425f-b961-74d507e80755-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4nc9g\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:23:54.269121 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:54.269099 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:23:54.547298 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:54.547272 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g"] Sep 11 08:23:54.549205 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:23:54.549177 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda288e67c_e21e_425f_b961_74d507e80755.slice/crio-73da4a24f2ed41821661dbde382c96fca78637302345e07e11e68cc3fad7f85b WatchSource:0}: Error finding container 73da4a24f2ed41821661dbde382c96fca78637302345e07e11e68cc3fad7f85b: Status 404 returned error can't find the container with id 73da4a24f2ed41821661dbde382c96fca78637302345e07e11e68cc3fad7f85b Sep 11 08:23:55.046651 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:55.046617 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" event={"ID":"a288e67c-e21e-425f-b961-74d507e80755","Type":"ContainerStarted","Data":"d409f3d1efd327188875ec1ce254bd38635060035e0f9f0748d027459f2bbcb3"} Sep 11 08:23:55.046831 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:23:55.046661 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" event={"ID":"a288e67c-e21e-425f-b961-74d507e80755","Type":"ContainerStarted","Data":"73da4a24f2ed41821661dbde382c96fca78637302345e07e11e68cc3fad7f85b"} Sep 11 08:24:00.068095 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:00.068061 2557 generic.go:356] "Generic (PLEG): container finished" podID="a288e67c-e21e-425f-b961-74d507e80755" containerID="d409f3d1efd327188875ec1ce254bd38635060035e0f9f0748d027459f2bbcb3" exitCode=0 Sep 11 08:24:00.068456 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:00.068134 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" event={"ID":"a288e67c-e21e-425f-b961-74d507e80755","Type":"ContainerDied","Data":"d409f3d1efd327188875ec1ce254bd38635060035e0f9f0748d027459f2bbcb3"} Sep 11 08:24:01.074629 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:01.074592 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" event={"ID":"a288e67c-e21e-425f-b961-74d507e80755","Type":"ContainerStarted","Data":"2d44a7418b79da8a9de7218b40e1b9c66b191aff7fe1bdd7776c51755dab9b72"} Sep 11 08:24:26.850494 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:24:26.850464 2557 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda288e67c_e21e_425f_b961_74d507e80755.slice/crio-conmon-2d44a7418b79da8a9de7218b40e1b9c66b191aff7fe1bdd7776c51755dab9b72.scope\": RecentStats: unable to find data in memory cache]" Sep 11 08:24:27.170979 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:27.170950 2557 generic.go:356] "Generic (PLEG): container finished" podID="a288e67c-e21e-425f-b961-74d507e80755" containerID="2d44a7418b79da8a9de7218b40e1b9c66b191aff7fe1bdd7776c51755dab9b72" exitCode=0 Sep 11 08:24:27.171106 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:27.171026 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" event={"ID":"a288e67c-e21e-425f-b961-74d507e80755","Type":"ContainerDied","Data":"2d44a7418b79da8a9de7218b40e1b9c66b191aff7fe1bdd7776c51755dab9b72"} Sep 11 08:24:28.179565 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:28.179527 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" event={"ID":"a288e67c-e21e-425f-b961-74d507e80755","Type":"ContainerStarted","Data":"bfbc5852fe0908614814be20a8e94abc175d91f0cee50621cddfdf29c7fa5a43"} Sep 11 08:24:28.180071 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:28.179578 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" event={"ID":"a288e67c-e21e-425f-b961-74d507e80755","Type":"ContainerStarted","Data":"6ed23563296681bffe0b8464f6bf57ac65fdbe9bde3483b168bd7fd24a2e4e40"} Sep 11 08:24:28.180071 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:28.179593 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" event={"ID":"a288e67c-e21e-425f-b961-74d507e80755","Type":"ContainerStarted","Data":"76d453098b9c26a79ee71267b599db8d00e0e5ce06aa3aa7bf46dc49dd903ff1"} Sep 11 08:24:28.209455 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:28.209402 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podStartSLOduration=35.209383159 podStartE2EDuration="35.209383159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:23:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:24:28.207356843 +0000 UTC m=+1505.213074500" watchObservedRunningTime="2026-09-11 08:24:28.209383159 +0000 UTC m=+1505.215100817" Sep 11 08:24:30.269964 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:30.269934 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:24:30.270421 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:30.269985 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:24:30.270421 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:30.270004 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:24:30.271732 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:30.271701 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Sep 11 08:24:30.271854 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:30.271732 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.30:8816: connect: connection refused" Sep 11 08:24:30.271854 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:30.271767 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Sep 11 08:24:33.269726 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:33.269684 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.30:8816: connect: connection refused" Sep 11 08:24:33.270227 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:33.269769 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Sep 11 08:24:33.270227 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:33.269680 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Sep 11 08:24:34.269759 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:34.269723 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:24:34.270268 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:34.269805 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:24:34.270268 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:34.269834 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:24:36.270175 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:36.270144 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:24:36.270602 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:36.270253 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:24:36.270877 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:36.270857 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:24:36.271119 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:36.271103 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:24:37.270329 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:37.270301 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": context deadline exceeded" Sep 11 08:24:40.275018 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:40.274973 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 11 08:24:43.194062 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:43.194033 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:24:44.182995 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:44.182958 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:24:54.199064 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:24:54.198982 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 11 08:25:04.200040 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:04.199995 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 11 08:25:14.199046 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:14.198993 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 11 08:25:14.199610 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:14.199115 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:25:15.204023 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:15.203982 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 11 08:25:15.274024 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:15.273985 2557 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 11 08:25:25.198357 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:25.198328 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:25:35.275027 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:35.274988 2557 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 11 08:25:45.807233 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:45.807190 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g"] Sep 11 08:25:45.807696 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:45.807431 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="online" containerID="cri-o://76d453098b9c26a79ee71267b599db8d00e0e5ce06aa3aa7bf46dc49dd903ff1" gracePeriod=30 Sep 11 08:25:45.807696 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:45.807538 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="ui" containerID="cri-o://bfbc5852fe0908614814be20a8e94abc175d91f0cee50621cddfdf29c7fa5a43" gracePeriod=30 Sep 11 08:25:45.807696 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:45.807550 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="offline" containerID="cri-o://6ed23563296681bffe0b8464f6bf57ac65fdbe9bde3483b168bd7fd24a2e4e40" gracePeriod=30 Sep 11 08:25:45.998937 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:45.998889 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j"] Sep 11 08:25:45.999215 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:45.999169 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="registry" containerID="cri-o://0ec607bd908d4028be9f2cc2a573654cfafe442eabb0374f3297e050c487b7bb" gracePeriod=30 Sep 11 08:25:45.999312 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:45.999211 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="offline" containerID="cri-o://a602276dea343185a89eab0b94d135088aec86dd0b02a9032ce30ed783f0fba0" gracePeriod=30 Sep 11 08:25:45.999463 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:45.999360 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="ui" containerID="cri-o://e9b16a2ceb6e114e568541639c461da61f2054b4644ac2d31276c222a219f6c7" gracePeriod=30 Sep 11 08:25:45.999463 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:45.999399 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="online" containerID="cri-o://23480169aafd65c8ceb7185b61b5eb5e5aeab5f68ce9448fd9148c3ee52c35eb" gracePeriod=30 Sep 11 08:25:46.271325 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:46.271274 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Sep 11 08:25:46.486721 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:46.486687 2557 generic.go:356] "Generic (PLEG): container finished" podID="a288e67c-e21e-425f-b961-74d507e80755" containerID="bfbc5852fe0908614814be20a8e94abc175d91f0cee50621cddfdf29c7fa5a43" exitCode=0 Sep 11 08:25:46.486958 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:46.486725 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" event={"ID":"a288e67c-e21e-425f-b961-74d507e80755","Type":"ContainerDied","Data":"bfbc5852fe0908614814be20a8e94abc175d91f0cee50621cddfdf29c7fa5a43"} Sep 11 08:25:47.493759 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:47.493730 2557 generic.go:356] "Generic (PLEG): container finished" podID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerID="e9b16a2ceb6e114e568541639c461da61f2054b4644ac2d31276c222a219f6c7" exitCode=0 Sep 11 08:25:47.493759 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:47.493752 2557 generic.go:356] "Generic (PLEG): container finished" podID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerID="23480169aafd65c8ceb7185b61b5eb5e5aeab5f68ce9448fd9148c3ee52c35eb" exitCode=0 Sep 11 08:25:47.494210 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:47.493808 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" event={"ID":"5ac530b0-45c2-4de2-bed9-3527de2ff310","Type":"ContainerDied","Data":"e9b16a2ceb6e114e568541639c461da61f2054b4644ac2d31276c222a219f6c7"} Sep 11 08:25:47.494210 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:47.493865 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" event={"ID":"5ac530b0-45c2-4de2-bed9-3527de2ff310","Type":"ContainerDied","Data":"23480169aafd65c8ceb7185b61b5eb5e5aeab5f68ce9448fd9148c3ee52c35eb"} Sep 11 08:25:47.496474 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:47.496454 2557 generic.go:356] "Generic (PLEG): container finished" podID="a288e67c-e21e-425f-b961-74d507e80755" containerID="76d453098b9c26a79ee71267b599db8d00e0e5ce06aa3aa7bf46dc49dd903ff1" exitCode=0 Sep 11 08:25:47.496575 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:47.496522 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" event={"ID":"a288e67c-e21e-425f-b961-74d507e80755","Type":"ContainerDied","Data":"76d453098b9c26a79ee71267b599db8d00e0e5ce06aa3aa7bf46dc49dd903ff1"} Sep 11 08:25:50.800485 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:50.800443 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Sep 11 08:25:50.800852 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:50.800447 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Sep 11 08:25:54.200503 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:54.200464 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Sep 11 08:25:56.270723 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:25:56.270676 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Sep 11 08:26:00.800479 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:00.800418 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Sep 11 08:26:00.800930 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:00.800506 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Sep 11 08:26:04.200648 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:04.200595 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Sep 11 08:26:06.271435 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:06.271376 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Sep 11 08:26:06.271934 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:06.271520 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:26:10.799997 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:10.799938 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Sep 11 08:26:10.800480 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:10.799959 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Sep 11 08:26:10.800480 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:10.800096 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:26:10.800480 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:10.800291 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:26:14.199783 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:14.199740 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Sep 11 08:26:14.200170 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:14.199880 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:26:16.271214 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.271180 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.30:8816: connect: connection refused" Sep 11 08:26:16.271524 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.271180 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Sep 11 08:26:16.465157 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.465134 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:26:16.562023 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.561882 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a288e67c-e21e-425f-b961-74d507e80755-feast-data\") pod \"a288e67c-e21e-425f-b961-74d507e80755\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " Sep 11 08:26:16.562023 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.561946 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a288e67c-e21e-425f-b961-74d507e80755-offline-tls\") pod \"a288e67c-e21e-425f-b961-74d507e80755\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " Sep 11 08:26:16.562023 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.561988 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a288e67c-e21e-425f-b961-74d507e80755-registry-tls\") pod \"a288e67c-e21e-425f-b961-74d507e80755\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " Sep 11 08:26:16.562226 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.562029 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a288e67c-e21e-425f-b961-74d507e80755-online-tls\") pod \"a288e67c-e21e-425f-b961-74d507e80755\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " Sep 11 08:26:16.562691 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.562400 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a288e67c-e21e-425f-b961-74d507e80755-feast-data" (OuterVolumeSpecName: "feast-data") pod "a288e67c-e21e-425f-b961-74d507e80755" (UID: "a288e67c-e21e-425f-b961-74d507e80755"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 08:26:16.562691 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.562561 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a288e67c-e21e-425f-b961-74d507e80755-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a288e67c-e21e-425f-b961-74d507e80755" (UID: "a288e67c-e21e-425f-b961-74d507e80755"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:26:16.562691 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.562670 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hh4t\" (UniqueName: \"kubernetes.io/projected/a288e67c-e21e-425f-b961-74d507e80755-kube-api-access-8hh4t\") pod \"a288e67c-e21e-425f-b961-74d507e80755\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " Sep 11 08:26:16.562933 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.562707 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a288e67c-e21e-425f-b961-74d507e80755-ui-tls\") pod \"a288e67c-e21e-425f-b961-74d507e80755\" (UID: \"a288e67c-e21e-425f-b961-74d507e80755\") " Sep 11 08:26:16.564620 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.563097 2557 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a288e67c-e21e-425f-b961-74d507e80755-feast-data\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:26:16.564620 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.563118 2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a288e67c-e21e-425f-b961-74d507e80755-registry-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:26:16.568398 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.568371 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a288e67c-e21e-425f-b961-74d507e80755-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "a288e67c-e21e-425f-b961-74d507e80755" (UID: "a288e67c-e21e-425f-b961-74d507e80755"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:26:16.568492 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.568435 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a288e67c-e21e-425f-b961-74d507e80755-online-tls" (OuterVolumeSpecName: "online-tls") pod "a288e67c-e21e-425f-b961-74d507e80755" (UID: "a288e67c-e21e-425f-b961-74d507e80755"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:26:16.568492 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.568469 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a288e67c-e21e-425f-b961-74d507e80755-kube-api-access-8hh4t" (OuterVolumeSpecName: "kube-api-access-8hh4t") pod "a288e67c-e21e-425f-b961-74d507e80755" (UID: "a288e67c-e21e-425f-b961-74d507e80755"). InnerVolumeSpecName "kube-api-access-8hh4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:26:16.568593 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.568523 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a288e67c-e21e-425f-b961-74d507e80755-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "a288e67c-e21e-425f-b961-74d507e80755" (UID: "a288e67c-e21e-425f-b961-74d507e80755"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:26:16.612317 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.612288 2557 generic.go:356] "Generic (PLEG): container finished" podID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerID="a602276dea343185a89eab0b94d135088aec86dd0b02a9032ce30ed783f0fba0" exitCode=137 Sep 11 08:26:16.612317 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.612316 2557 generic.go:356] "Generic (PLEG): container finished" podID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerID="0ec607bd908d4028be9f2cc2a573654cfafe442eabb0374f3297e050c487b7bb" exitCode=137 Sep 11 08:26:16.612452 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.612362 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" event={"ID":"5ac530b0-45c2-4de2-bed9-3527de2ff310","Type":"ContainerDied","Data":"a602276dea343185a89eab0b94d135088aec86dd0b02a9032ce30ed783f0fba0"} Sep 11 08:26:16.612452 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.612405 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" event={"ID":"5ac530b0-45c2-4de2-bed9-3527de2ff310","Type":"ContainerDied","Data":"0ec607bd908d4028be9f2cc2a573654cfafe442eabb0374f3297e050c487b7bb"} Sep 11 08:26:16.614950 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.614928 2557 generic.go:356] "Generic (PLEG): container finished" podID="a288e67c-e21e-425f-b961-74d507e80755" containerID="6ed23563296681bffe0b8464f6bf57ac65fdbe9bde3483b168bd7fd24a2e4e40" exitCode=137 Sep 11 08:26:16.615031 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.614977 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" event={"ID":"a288e67c-e21e-425f-b961-74d507e80755","Type":"ContainerDied","Data":"6ed23563296681bffe0b8464f6bf57ac65fdbe9bde3483b168bd7fd24a2e4e40"} Sep 11 08:26:16.615031 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.615011 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" event={"ID":"a288e67c-e21e-425f-b961-74d507e80755","Type":"ContainerDied","Data":"73da4a24f2ed41821661dbde382c96fca78637302345e07e11e68cc3fad7f85b"} Sep 11 08:26:16.615121 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.615032 2557 scope.go:117] "RemoveContainer" containerID="bfbc5852fe0908614814be20a8e94abc175d91f0cee50621cddfdf29c7fa5a43" Sep 11 08:26:16.615121 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.615044 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g" Sep 11 08:26:16.623445 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.623424 2557 scope.go:117] "RemoveContainer" containerID="6ed23563296681bffe0b8464f6bf57ac65fdbe9bde3483b168bd7fd24a2e4e40" Sep 11 08:26:16.631034 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.631019 2557 scope.go:117] "RemoveContainer" containerID="76d453098b9c26a79ee71267b599db8d00e0e5ce06aa3aa7bf46dc49dd903ff1" Sep 11 08:26:16.637424 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.637407 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:26:16.638680 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.638663 2557 scope.go:117] "RemoveContainer" containerID="2d44a7418b79da8a9de7218b40e1b9c66b191aff7fe1bdd7776c51755dab9b72" Sep 11 08:26:16.640698 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.640675 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g"] Sep 11 08:26:16.645795 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.645782 2557 scope.go:117] "RemoveContainer" containerID="d409f3d1efd327188875ec1ce254bd38635060035e0f9f0748d027459f2bbcb3" Sep 11 08:26:16.652269 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.652246 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4nc9g"] Sep 11 08:26:16.652680 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.652668 2557 scope.go:117] "RemoveContainer" containerID="bfbc5852fe0908614814be20a8e94abc175d91f0cee50621cddfdf29c7fa5a43" Sep 11 08:26:16.652954 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:26:16.652931 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfbc5852fe0908614814be20a8e94abc175d91f0cee50621cddfdf29c7fa5a43\": container with ID starting with bfbc5852fe0908614814be20a8e94abc175d91f0cee50621cddfdf29c7fa5a43 not found: ID does not exist" containerID="bfbc5852fe0908614814be20a8e94abc175d91f0cee50621cddfdf29c7fa5a43" Sep 11 08:26:16.653007 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.652966 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfbc5852fe0908614814be20a8e94abc175d91f0cee50621cddfdf29c7fa5a43"} err="failed to get container status \"bfbc5852fe0908614814be20a8e94abc175d91f0cee50621cddfdf29c7fa5a43\": rpc error: code = NotFound desc = could not find container \"bfbc5852fe0908614814be20a8e94abc175d91f0cee50621cddfdf29c7fa5a43\": container with ID starting with bfbc5852fe0908614814be20a8e94abc175d91f0cee50621cddfdf29c7fa5a43 not found: ID does not exist" Sep 11 08:26:16.653007 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.652982 2557 scope.go:117] "RemoveContainer" containerID="6ed23563296681bffe0b8464f6bf57ac65fdbe9bde3483b168bd7fd24a2e4e40" Sep 11 08:26:16.653224 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:26:16.653209 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ed23563296681bffe0b8464f6bf57ac65fdbe9bde3483b168bd7fd24a2e4e40\": container with ID starting with 6ed23563296681bffe0b8464f6bf57ac65fdbe9bde3483b168bd7fd24a2e4e40 not found: ID does not exist" containerID="6ed23563296681bffe0b8464f6bf57ac65fdbe9bde3483b168bd7fd24a2e4e40" Sep 11 08:26:16.653263 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.653227 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ed23563296681bffe0b8464f6bf57ac65fdbe9bde3483b168bd7fd24a2e4e40"} err="failed to get container status \"6ed23563296681bffe0b8464f6bf57ac65fdbe9bde3483b168bd7fd24a2e4e40\": rpc error: code = NotFound desc = could not find container \"6ed23563296681bffe0b8464f6bf57ac65fdbe9bde3483b168bd7fd24a2e4e40\": container with ID starting with 6ed23563296681bffe0b8464f6bf57ac65fdbe9bde3483b168bd7fd24a2e4e40 not found: ID does not exist" Sep 11 08:26:16.653263 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.653241 2557 scope.go:117] "RemoveContainer" containerID="76d453098b9c26a79ee71267b599db8d00e0e5ce06aa3aa7bf46dc49dd903ff1" Sep 11 08:26:16.653439 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:26:16.653424 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76d453098b9c26a79ee71267b599db8d00e0e5ce06aa3aa7bf46dc49dd903ff1\": container with ID starting with 76d453098b9c26a79ee71267b599db8d00e0e5ce06aa3aa7bf46dc49dd903ff1 not found: ID does not exist" containerID="76d453098b9c26a79ee71267b599db8d00e0e5ce06aa3aa7bf46dc49dd903ff1" Sep 11 08:26:16.653478 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.653443 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76d453098b9c26a79ee71267b599db8d00e0e5ce06aa3aa7bf46dc49dd903ff1"} err="failed to get container status \"76d453098b9c26a79ee71267b599db8d00e0e5ce06aa3aa7bf46dc49dd903ff1\": rpc error: code = NotFound desc = could not find container \"76d453098b9c26a79ee71267b599db8d00e0e5ce06aa3aa7bf46dc49dd903ff1\": container with ID starting with 76d453098b9c26a79ee71267b599db8d00e0e5ce06aa3aa7bf46dc49dd903ff1 not found: ID does not exist" Sep 11 08:26:16.653478 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.653456 2557 scope.go:117] "RemoveContainer" containerID="2d44a7418b79da8a9de7218b40e1b9c66b191aff7fe1bdd7776c51755dab9b72" Sep 11 08:26:16.653713 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:26:16.653693 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d44a7418b79da8a9de7218b40e1b9c66b191aff7fe1bdd7776c51755dab9b72\": container with ID starting with 2d44a7418b79da8a9de7218b40e1b9c66b191aff7fe1bdd7776c51755dab9b72 not found: ID does not exist" containerID="2d44a7418b79da8a9de7218b40e1b9c66b191aff7fe1bdd7776c51755dab9b72" Sep 11 08:26:16.653760 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.653726 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d44a7418b79da8a9de7218b40e1b9c66b191aff7fe1bdd7776c51755dab9b72"} err="failed to get container status \"2d44a7418b79da8a9de7218b40e1b9c66b191aff7fe1bdd7776c51755dab9b72\": rpc error: code = NotFound desc = could not find container \"2d44a7418b79da8a9de7218b40e1b9c66b191aff7fe1bdd7776c51755dab9b72\": container with ID starting with 2d44a7418b79da8a9de7218b40e1b9c66b191aff7fe1bdd7776c51755dab9b72 not found: ID does not exist" Sep 11 08:26:16.653760 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.653746 2557 scope.go:117] "RemoveContainer" containerID="d409f3d1efd327188875ec1ce254bd38635060035e0f9f0748d027459f2bbcb3" Sep 11 08:26:16.654022 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:26:16.653996 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d409f3d1efd327188875ec1ce254bd38635060035e0f9f0748d027459f2bbcb3\": container with ID starting with d409f3d1efd327188875ec1ce254bd38635060035e0f9f0748d027459f2bbcb3 not found: ID does not exist" containerID="d409f3d1efd327188875ec1ce254bd38635060035e0f9f0748d027459f2bbcb3" Sep 11 08:26:16.654111 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.654030 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d409f3d1efd327188875ec1ce254bd38635060035e0f9f0748d027459f2bbcb3"} err="failed to get container status \"d409f3d1efd327188875ec1ce254bd38635060035e0f9f0748d027459f2bbcb3\": rpc error: code = NotFound desc = could not find container \"d409f3d1efd327188875ec1ce254bd38635060035e0f9f0748d027459f2bbcb3\": container with ID starting with d409f3d1efd327188875ec1ce254bd38635060035e0f9f0748d027459f2bbcb3 not found: ID does not exist" Sep 11 08:26:16.663840 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.663824 2557 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a288e67c-e21e-425f-b961-74d507e80755-ui-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:26:16.663923 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.663841 2557 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a288e67c-e21e-425f-b961-74d507e80755-offline-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:26:16.663923 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.663850 2557 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a288e67c-e21e-425f-b961-74d507e80755-online-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:26:16.663923 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.663859 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hh4t\" (UniqueName: \"kubernetes.io/projected/a288e67c-e21e-425f-b961-74d507e80755-kube-api-access-8hh4t\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:26:16.764757 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.764737 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-online-tls\") pod \"5ac530b0-45c2-4de2-bed9-3527de2ff310\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " Sep 11 08:26:16.764843 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.764768 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-ui-tls\") pod \"5ac530b0-45c2-4de2-bed9-3527de2ff310\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " Sep 11 08:26:16.764843 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.764810 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-registry-tls\") pod \"5ac530b0-45c2-4de2-bed9-3527de2ff310\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " Sep 11 08:26:16.764843 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.764827 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfrzs\" (UniqueName: \"kubernetes.io/projected/5ac530b0-45c2-4de2-bed9-3527de2ff310-kube-api-access-rfrzs\") pod \"5ac530b0-45c2-4de2-bed9-3527de2ff310\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " Sep 11 08:26:16.765009 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.764842 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5ac530b0-45c2-4de2-bed9-3527de2ff310-feast-data\") pod \"5ac530b0-45c2-4de2-bed9-3527de2ff310\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " Sep 11 08:26:16.765009 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.764868 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-offline-tls\") pod \"5ac530b0-45c2-4de2-bed9-3527de2ff310\" (UID: \"5ac530b0-45c2-4de2-bed9-3527de2ff310\") " Sep 11 08:26:16.765404 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.765377 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ac530b0-45c2-4de2-bed9-3527de2ff310-feast-data" (OuterVolumeSpecName: "feast-data") pod "5ac530b0-45c2-4de2-bed9-3527de2ff310" (UID: "5ac530b0-45c2-4de2-bed9-3527de2ff310"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 08:26:16.766735 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.766714 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-online-tls" (OuterVolumeSpecName: "online-tls") pod "5ac530b0-45c2-4de2-bed9-3527de2ff310" (UID: "5ac530b0-45c2-4de2-bed9-3527de2ff310"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:26:16.766807 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.766791 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "5ac530b0-45c2-4de2-bed9-3527de2ff310" (UID: "5ac530b0-45c2-4de2-bed9-3527de2ff310"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:26:16.766887 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.766872 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "5ac530b0-45c2-4de2-bed9-3527de2ff310" (UID: "5ac530b0-45c2-4de2-bed9-3527de2ff310"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:26:16.767254 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.767230 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5ac530b0-45c2-4de2-bed9-3527de2ff310" (UID: "5ac530b0-45c2-4de2-bed9-3527de2ff310"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:26:16.767254 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.767244 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ac530b0-45c2-4de2-bed9-3527de2ff310-kube-api-access-rfrzs" (OuterVolumeSpecName: "kube-api-access-rfrzs") pod "5ac530b0-45c2-4de2-bed9-3527de2ff310" (UID: "5ac530b0-45c2-4de2-bed9-3527de2ff310"). InnerVolumeSpecName "kube-api-access-rfrzs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:26:16.865914 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.865877 2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-registry-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:26:16.865914 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.865911 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfrzs\" (UniqueName: \"kubernetes.io/projected/5ac530b0-45c2-4de2-bed9-3527de2ff310-kube-api-access-rfrzs\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:26:16.866019 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.865921 2557 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5ac530b0-45c2-4de2-bed9-3527de2ff310-feast-data\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:26:16.866019 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.865929 2557 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-offline-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:26:16.866019 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.865937 2557 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-online-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:26:16.866019 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:16.865944 2557 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5ac530b0-45c2-4de2-bed9-3527de2ff310-ui-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:26:17.502572 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:17.502544 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a288e67c-e21e-425f-b961-74d507e80755" path="/var/lib/kubelet/pods/a288e67c-e21e-425f-b961-74d507e80755/volumes" Sep 11 08:26:17.622278 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:17.622243 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" event={"ID":"5ac530b0-45c2-4de2-bed9-3527de2ff310","Type":"ContainerDied","Data":"da57e48292884c5bcd7a67948c229e6a717532edb5c5923e3774742ead1275b4"} Sep 11 08:26:17.622383 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:17.622253 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j" Sep 11 08:26:17.622383 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:17.622301 2557 scope.go:117] "RemoveContainer" containerID="e9b16a2ceb6e114e568541639c461da61f2054b4644ac2d31276c222a219f6c7" Sep 11 08:26:17.629874 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:17.629857 2557 scope.go:117] "RemoveContainer" containerID="a602276dea343185a89eab0b94d135088aec86dd0b02a9032ce30ed783f0fba0" Sep 11 08:26:17.637234 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:17.637218 2557 scope.go:117] "RemoveContainer" containerID="23480169aafd65c8ceb7185b61b5eb5e5aeab5f68ce9448fd9148c3ee52c35eb" Sep 11 08:26:17.640813 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:17.640793 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j"] Sep 11 08:26:17.645219 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:17.645201 2557 scope.go:117] "RemoveContainer" containerID="0ec607bd908d4028be9f2cc2a573654cfafe442eabb0374f3297e050c487b7bb" Sep 11 08:26:17.645284 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:17.645248 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-hrj6j"] Sep 11 08:26:17.652175 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:17.652159 2557 scope.go:117] "RemoveContainer" containerID="00894be7da28186be42596fd5568389572dae7bee8a4d609ac7c017511137681" Sep 11 08:26:17.658754 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:17.658738 2557 scope.go:117] "RemoveContainer" containerID="a604143d9e53e64aca5e367086e8cc9e175801add8efba25225bf289dbbf8aeb" Sep 11 08:26:19.501731 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:19.501693 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" path="/var/lib/kubelet/pods/5ac530b0-45c2-4de2-bed9-3527de2ff310/volumes" Sep 11 08:26:33.485849 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:33.485814 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-fv5tj"] Sep 11 08:26:33.486363 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:33.486048 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fv5tj" podUID="d0728cb4-9ac5-4d37-9415-ba38caa0a3c9" containerName="manager" containerID="cri-o://cb0a994991234f404961d90b80f6156b2b8ee0db7be859de58c9ccfcf9bbb9e2" gracePeriod=10 Sep 11 08:26:33.681545 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:33.681516 2557 generic.go:356] "Generic (PLEG): container finished" podID="d0728cb4-9ac5-4d37-9415-ba38caa0a3c9" containerID="cb0a994991234f404961d90b80f6156b2b8ee0db7be859de58c9ccfcf9bbb9e2" exitCode=0 Sep 11 08:26:33.681656 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:33.681595 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fv5tj" event={"ID":"d0728cb4-9ac5-4d37-9415-ba38caa0a3c9","Type":"ContainerDied","Data":"cb0a994991234f404961d90b80f6156b2b8ee0db7be859de58c9ccfcf9bbb9e2"} Sep 11 08:26:33.730813 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:33.730788 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fv5tj" Sep 11 08:26:33.875736 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:33.875667 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w74gv\" (UniqueName: \"kubernetes.io/projected/d0728cb4-9ac5-4d37-9415-ba38caa0a3c9-kube-api-access-w74gv\") pod \"d0728cb4-9ac5-4d37-9415-ba38caa0a3c9\" (UID: \"d0728cb4-9ac5-4d37-9415-ba38caa0a3c9\") " Sep 11 08:26:33.877653 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:33.877627 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0728cb4-9ac5-4d37-9415-ba38caa0a3c9-kube-api-access-w74gv" (OuterVolumeSpecName: "kube-api-access-w74gv") pod "d0728cb4-9ac5-4d37-9415-ba38caa0a3c9" (UID: "d0728cb4-9ac5-4d37-9415-ba38caa0a3c9"). InnerVolumeSpecName "kube-api-access-w74gv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:26:33.976768 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:33.976733 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w74gv\" (UniqueName: \"kubernetes.io/projected/d0728cb4-9ac5-4d37-9415-ba38caa0a3c9-kube-api-access-w74gv\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:26:34.687163 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:34.687128 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fv5tj" event={"ID":"d0728cb4-9ac5-4d37-9415-ba38caa0a3c9","Type":"ContainerDied","Data":"d39edf3aff1ff188c0fb019dfb3f15e55a8f18d14cfddfb9d0f30aeeabbdd289"} Sep 11 08:26:34.687606 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:34.687141 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fv5tj" Sep 11 08:26:34.687606 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:34.687218 2557 scope.go:117] "RemoveContainer" containerID="cb0a994991234f404961d90b80f6156b2b8ee0db7be859de58c9ccfcf9bbb9e2" Sep 11 08:26:34.715745 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:34.715723 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-fv5tj"] Sep 11 08:26:34.720085 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:34.720066 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-fv5tj"] Sep 11 08:26:35.501649 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:35.501615 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0728cb4-9ac5-4d37-9415-ba38caa0a3c9" path="/var/lib/kubelet/pods/d0728cb4-9ac5-4d37-9415-ba38caa0a3c9/volumes" Sep 11 08:26:38.654139 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654101 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-6qsxb"] Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654363 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="feast-init" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654376 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="feast-init" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654385 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="ui" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654391 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="ui" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654398 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="feast-apply" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654404 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="feast-apply" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654411 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="online" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654416 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="online" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654422 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="offline" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654428 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="offline" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654436 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="online" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654441 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="online" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654447 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="feast-apply" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654452 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="feast-apply" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654488 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="ui" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654494 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="ui" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654502 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0728cb4-9ac5-4d37-9415-ba38caa0a3c9" containerName="manager" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654508 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0728cb4-9ac5-4d37-9415-ba38caa0a3c9" containerName="manager" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654514 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="offline" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654519 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="offline" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654530 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="feast-init" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654537 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="feast-init" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654543 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="registry" Sep 11 08:26:38.654535 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654549 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="registry" Sep 11 08:26:38.655323 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654594 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="offline" Sep 11 08:26:38.655323 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654600 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="ui" Sep 11 08:26:38.655323 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654608 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="registry" Sep 11 08:26:38.655323 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654614 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="online" Sep 11 08:26:38.655323 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654618 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ac530b0-45c2-4de2-bed9-3527de2ff310" containerName="offline" Sep 11 08:26:38.655323 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654623 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0728cb4-9ac5-4d37-9415-ba38caa0a3c9" containerName="manager" Sep 11 08:26:38.655323 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654628 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="online" Sep 11 08:26:38.655323 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.654633 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a288e67c-e21e-425f-b961-74d507e80755" containerName="ui" Sep 11 08:26:38.663602 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.663573 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-6qsxb" Sep 11 08:26:38.668832 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.668758 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 11 08:26:38.669245 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.669214 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-5ndwn\"" Sep 11 08:26:38.669356 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.669254 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 11 08:26:38.671109 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.671083 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-6qsxb"] Sep 11 08:26:38.808591 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.808556 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97wdm\" (UniqueName: \"kubernetes.io/projected/6e21b387-2d09-4fed-aa00-3c7c590472bf-kube-api-access-97wdm\") pod \"feast-operator-controller-manager-689b877b4b-6qsxb\" (UID: \"6e21b387-2d09-4fed-aa00-3c7c590472bf\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-6qsxb" Sep 11 08:26:38.909497 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.909434 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97wdm\" (UniqueName: \"kubernetes.io/projected/6e21b387-2d09-4fed-aa00-3c7c590472bf-kube-api-access-97wdm\") pod \"feast-operator-controller-manager-689b877b4b-6qsxb\" (UID: \"6e21b387-2d09-4fed-aa00-3c7c590472bf\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-6qsxb" Sep 11 08:26:38.919157 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.919134 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97wdm\" (UniqueName: \"kubernetes.io/projected/6e21b387-2d09-4fed-aa00-3c7c590472bf-kube-api-access-97wdm\") pod \"feast-operator-controller-manager-689b877b4b-6qsxb\" (UID: \"6e21b387-2d09-4fed-aa00-3c7c590472bf\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-6qsxb" Sep 11 08:26:38.974755 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:38.974732 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-6qsxb" Sep 11 08:26:39.274979 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:39.274955 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-6qsxb"] Sep 11 08:26:39.277146 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:26:39.277116 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e21b387_2d09_4fed_aa00_3c7c590472bf.slice/crio-55e9d8e7f5bfad2856dfe19d3ea5a946416629a5703157c0be79c0975fa812c7 WatchSource:0}: Error finding container 55e9d8e7f5bfad2856dfe19d3ea5a946416629a5703157c0be79c0975fa812c7: Status 404 returned error can't find the container with id 55e9d8e7f5bfad2856dfe19d3ea5a946416629a5703157c0be79c0975fa812c7 Sep 11 08:26:39.707133 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:39.707098 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-6qsxb" event={"ID":"6e21b387-2d09-4fed-aa00-3c7c590472bf","Type":"ContainerStarted","Data":"d0e28474eeb6d13b7161bf76fb1e830ae2799ca07ed71696380b3fb18219d52f"} Sep 11 08:26:39.707526 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:39.707157 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-6qsxb" event={"ID":"6e21b387-2d09-4fed-aa00-3c7c590472bf","Type":"ContainerStarted","Data":"55e9d8e7f5bfad2856dfe19d3ea5a946416629a5703157c0be79c0975fa812c7"} Sep 11 08:26:39.707526 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:39.707199 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-6qsxb" Sep 11 08:26:50.712816 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:50.712787 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-6qsxb" Sep 11 08:26:50.731249 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:50.731196 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-6qsxb" podStartSLOduration=12.731179299 podStartE2EDuration="12.731179299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:26:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:26:39.731442116 +0000 UTC m=+1636.737159773" watchObservedRunningTime="2026-09-11 08:26:50.731179299 +0000 UTC m=+1647.736896958" Sep 11 08:26:55.716511 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.716479 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn"] Sep 11 08:26:55.719889 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.719874 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:55.725266 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.725247 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 11 08:26:55.725462 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.725442 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 11 08:26:55.725605 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.725584 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-kv2mk\"" Sep 11 08:26:55.725605 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.725599 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 11 08:26:55.725741 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.725596 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 11 08:26:55.726071 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.725873 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 11 08:26:55.726071 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.725880 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 11 08:26:55.738810 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.738784 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn"] Sep 11 08:26:55.830219 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.830192 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-skzdn\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:55.830312 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.830216 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf822\" (UniqueName: \"kubernetes.io/projected/1f9d74ae-f5c1-4e37-a473-94ee838160f3-kube-api-access-sf822\") pod \"feast-simple-feast-setup-57576b8fcf-skzdn\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:55.830312 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.830251 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1f9d74ae-f5c1-4e37-a473-94ee838160f3-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-skzdn\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:55.830391 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.830327 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-skzdn\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:55.830391 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.830348 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-skzdn\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:55.830391 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.830372 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-skzdn\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:55.930755 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.930729 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1f9d74ae-f5c1-4e37-a473-94ee838160f3-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-skzdn\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:55.930848 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.930773 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-skzdn\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:55.930848 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.930790 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-skzdn\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:55.930848 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.930809 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-skzdn\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:55.930848 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.930834 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-skzdn\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:55.931046 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:26:55.930890 2557 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 11 08:26:55.931046 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:26:55.930930 2557 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 11 08:26:55.931046 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:26:55.930949 2557 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 11 08:26:55.931046 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:26:55.930986 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-ui-tls podName:1f9d74ae-f5c1-4e37-a473-94ee838160f3 nodeName:}" failed. No retries permitted until 2026-09-11 08:26:56.430966685 +0000 UTC m=+1653.436684338 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-ui-tls") pod "feast-simple-feast-setup-57576b8fcf-skzdn" (UID: "1f9d74ae-f5c1-4e37-a473-94ee838160f3") : secret "feast-simple-feast-setup-ui-tls" not found Sep 11 08:26:55.931046 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:26:55.931008 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-online-tls podName:1f9d74ae-f5c1-4e37-a473-94ee838160f3 nodeName:}" failed. No retries permitted until 2026-09-11 08:26:56.430998911 +0000 UTC m=+1653.436716547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-online-tls") pod "feast-simple-feast-setup-57576b8fcf-skzdn" (UID: "1f9d74ae-f5c1-4e37-a473-94ee838160f3") : secret "feast-simple-feast-setup-online-tls" not found Sep 11 08:26:55.931046 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:26:55.931025 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-registry-tls podName:1f9d74ae-f5c1-4e37-a473-94ee838160f3 nodeName:}" failed. No retries permitted until 2026-09-11 08:26:56.43101633 +0000 UTC m=+1653.436733967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-registry-tls") pod "feast-simple-feast-setup-57576b8fcf-skzdn" (UID: "1f9d74ae-f5c1-4e37-a473-94ee838160f3") : secret "feast-simple-feast-setup-registry-tls" not found Sep 11 08:26:55.931046 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.930985 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf822\" (UniqueName: \"kubernetes.io/projected/1f9d74ae-f5c1-4e37-a473-94ee838160f3-kube-api-access-sf822\") pod \"feast-simple-feast-setup-57576b8fcf-skzdn\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:55.931311 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.931136 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1f9d74ae-f5c1-4e37-a473-94ee838160f3-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-skzdn\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:55.933129 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.933113 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-skzdn\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:55.940416 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:55.940393 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf822\" (UniqueName: \"kubernetes.io/projected/1f9d74ae-f5c1-4e37-a473-94ee838160f3-kube-api-access-sf822\") pod \"feast-simple-feast-setup-57576b8fcf-skzdn\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:56.435180 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:56.435156 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-skzdn\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:56.435346 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:56.435208 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-skzdn\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:56.435346 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:56.435227 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-skzdn\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:56.437591 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:56.437566 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-skzdn\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:56.437689 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:56.437636 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-skzdn\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:56.437728 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:56.437712 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-skzdn\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:56.635792 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:56.635768 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:26:56.915313 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:56.915277 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn"] Sep 11 08:26:56.917921 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:26:56.917886 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f9d74ae_f5c1_4e37_a473_94ee838160f3.slice/crio-fabd3dc0d39a4ab77c2a917d09968832e2d658bc44875c4cccb4d9f1825826d2 WatchSource:0}: Error finding container fabd3dc0d39a4ab77c2a917d09968832e2d658bc44875c4cccb4d9f1825826d2: Status 404 returned error can't find the container with id fabd3dc0d39a4ab77c2a917d09968832e2d658bc44875c4cccb4d9f1825826d2 Sep 11 08:26:57.773912 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:57.773867 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" event={"ID":"1f9d74ae-f5c1-4e37-a473-94ee838160f3","Type":"ContainerStarted","Data":"858d2980839a95ca7b35192aaaac0d20f5c1a93da7eb816d3612442087582bac"} Sep 11 08:26:57.774074 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:26:57.773949 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" event={"ID":"1f9d74ae-f5c1-4e37-a473-94ee838160f3","Type":"ContainerStarted","Data":"fabd3dc0d39a4ab77c2a917d09968832e2d658bc44875c4cccb4d9f1825826d2"} Sep 11 08:27:01.788949 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:01.788915 2557 generic.go:356] "Generic (PLEG): container finished" podID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerID="858d2980839a95ca7b35192aaaac0d20f5c1a93da7eb816d3612442087582bac" exitCode=0 Sep 11 08:27:01.789254 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:01.788928 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" event={"ID":"1f9d74ae-f5c1-4e37-a473-94ee838160f3","Type":"ContainerDied","Data":"858d2980839a95ca7b35192aaaac0d20f5c1a93da7eb816d3612442087582bac"} Sep 11 08:27:02.795347 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:02.795314 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" event={"ID":"1f9d74ae-f5c1-4e37-a473-94ee838160f3","Type":"ContainerStarted","Data":"f93fd31e4df33f9c320d9d43f1581ab1b295aae6296f4192edd9b04859803dbf"} Sep 11 08:27:07.816556 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:07.816495 2557 generic.go:356] "Generic (PLEG): container finished" podID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerID="f93fd31e4df33f9c320d9d43f1581ab1b295aae6296f4192edd9b04859803dbf" exitCode=0 Sep 11 08:27:07.816939 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:07.816564 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" event={"ID":"1f9d74ae-f5c1-4e37-a473-94ee838160f3","Type":"ContainerDied","Data":"f93fd31e4df33f9c320d9d43f1581ab1b295aae6296f4192edd9b04859803dbf"} Sep 11 08:27:08.826544 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:08.826497 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" event={"ID":"1f9d74ae-f5c1-4e37-a473-94ee838160f3","Type":"ContainerStarted","Data":"bcca4b6aca1d071f7f4181aec43979541efe22f0d4ae0978a27e534ad98d1f7f"} Sep 11 08:27:08.827071 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:08.826562 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" event={"ID":"1f9d74ae-f5c1-4e37-a473-94ee838160f3","Type":"ContainerStarted","Data":"374174abf83b39fce7dc54e44d8c6cb79a7a39d010bff1d97373c93aa7edebd9"} Sep 11 08:27:08.827071 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:08.826584 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" event={"ID":"1f9d74ae-f5c1-4e37-a473-94ee838160f3","Type":"ContainerStarted","Data":"b3f5353758f689ee130104ac3999fada89897226f181e85807afdbde00a510bb"} Sep 11 08:27:08.827071 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:08.826604 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" event={"ID":"1f9d74ae-f5c1-4e37-a473-94ee838160f3","Type":"ContainerStarted","Data":"cdb02aa79485f785712ab81a673d5261e746d953ac8360f64d7cb6f5d8aaef1e"} Sep 11 08:27:08.855173 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:08.855113 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" podStartSLOduration=13.855093689 podStartE2EDuration="13.855093689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:26:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:27:08.853040221 +0000 UTC m=+1665.858757881" watchObservedRunningTime="2026-09-11 08:27:08.855093689 +0000 UTC m=+1665.860811348" Sep 11 08:27:11.637115 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:11.636933 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:27:11.637115 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:11.636990 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:27:11.637115 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:11.637011 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:27:11.637115 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:11.637030 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:27:11.639086 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:11.639051 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused" Sep 11 08:27:11.639310 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:11.639073 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.32:6571: connect: connection refused" Sep 11 08:27:11.639439 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:11.639112 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.32:8816: connect: connection refused" Sep 11 08:27:11.639439 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:11.639190 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Sep 11 08:27:14.637163 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:14.637114 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.32:8816: connect: connection refused" Sep 11 08:27:14.637677 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:14.637167 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused" Sep 11 08:27:14.637677 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:14.637187 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Sep 11 08:27:14.637677 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:14.637406 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.32:6571: connect: connection refused" Sep 11 08:27:16.636191 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:16.636157 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:27:16.636610 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:16.636214 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:27:16.636610 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:16.636235 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:27:16.636610 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:16.636251 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:27:17.637831 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:17.637801 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:27:17.638248 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:17.637958 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:27:17.638767 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:17.638742 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:27:17.639005 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:17.638984 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:27:17.639115 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:17.639103 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:27:17.643449 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:17.643425 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:27:17.868286 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:17.868266 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:27:17.870693 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:27:17.870674 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:28:19.899107 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:19.899077 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn"] Sep 11 08:28:19.899577 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:19.899295 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="registry" containerID="cri-o://cdb02aa79485f785712ab81a673d5261e746d953ac8360f64d7cb6f5d8aaef1e" gracePeriod=30 Sep 11 08:28:19.899577 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:19.899343 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="ui" containerID="cri-o://bcca4b6aca1d071f7f4181aec43979541efe22f0d4ae0978a27e534ad98d1f7f" gracePeriod=30 Sep 11 08:28:19.899577 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:19.899364 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="online" containerID="cri-o://b3f5353758f689ee130104ac3999fada89897226f181e85807afdbde00a510bb" gracePeriod=30 Sep 11 08:28:19.899577 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:19.899369 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="offline" containerID="cri-o://374174abf83b39fce7dc54e44d8c6cb79a7a39d010bff1d97373c93aa7edebd9" gracePeriod=30 Sep 11 08:28:20.702575 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.702512 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll"] Sep 11 08:28:20.706328 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.706309 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:20.711554 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.711534 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-g9kmg\"" Sep 11 08:28:20.734796 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.734774 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll"] Sep 11 08:28:20.817802 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.817779 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-cjzll\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:20.817930 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.817822 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-cjzll\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:20.817930 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.817870 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/95af2f9c-0950-4d31-adda-b15f1dc94db0-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-cjzll\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:20.818026 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.817933 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwfqj\" (UniqueName: \"kubernetes.io/projected/95af2f9c-0950-4d31-adda-b15f1dc94db0-kube-api-access-rwfqj\") pod \"feast-simple-feast-setup-57576b8fcf-cjzll\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:20.818026 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.817968 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-cjzll\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:20.818026 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.818009 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-cjzll\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:20.918271 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.918249 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-cjzll\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:20.918616 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.918280 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/95af2f9c-0950-4d31-adda-b15f1dc94db0-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-cjzll\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:20.918616 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.918299 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwfqj\" (UniqueName: \"kubernetes.io/projected/95af2f9c-0950-4d31-adda-b15f1dc94db0-kube-api-access-rwfqj\") pod \"feast-simple-feast-setup-57576b8fcf-cjzll\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:20.918616 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.918323 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-cjzll\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:20.918616 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.918358 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-cjzll\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:20.918616 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.918395 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-cjzll\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:20.918920 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.918882 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/95af2f9c-0950-4d31-adda-b15f1dc94db0-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-cjzll\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:20.920759 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.920735 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-cjzll\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:20.920913 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.920874 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-cjzll\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:20.920986 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.920954 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-cjzll\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:20.920986 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.920953 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-cjzll\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:20.929911 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:20.929875 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwfqj\" (UniqueName: \"kubernetes.io/projected/95af2f9c-0950-4d31-adda-b15f1dc94db0-kube-api-access-rwfqj\") pod \"feast-simple-feast-setup-57576b8fcf-cjzll\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:21.015063 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:21.015017 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:21.108506 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:21.108474 2557 generic.go:356] "Generic (PLEG): container finished" podID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerID="bcca4b6aca1d071f7f4181aec43979541efe22f0d4ae0978a27e534ad98d1f7f" exitCode=0 Sep 11 08:28:21.108506 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:21.108500 2557 generic.go:356] "Generic (PLEG): container finished" podID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerID="b3f5353758f689ee130104ac3999fada89897226f181e85807afdbde00a510bb" exitCode=0 Sep 11 08:28:21.108660 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:21.108541 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" event={"ID":"1f9d74ae-f5c1-4e37-a473-94ee838160f3","Type":"ContainerDied","Data":"bcca4b6aca1d071f7f4181aec43979541efe22f0d4ae0978a27e534ad98d1f7f"} Sep 11 08:28:21.108660 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:21.108581 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" event={"ID":"1f9d74ae-f5c1-4e37-a473-94ee838160f3","Type":"ContainerDied","Data":"b3f5353758f689ee130104ac3999fada89897226f181e85807afdbde00a510bb"} Sep 11 08:28:21.298991 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:21.298960 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll"] Sep 11 08:28:21.301170 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:28:21.301143 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95af2f9c_0950_4d31_adda_b15f1dc94db0.slice/crio-7a5ee9f636ab404ba03db3da155dedee0e6a276b5932f13374be69aab67f5a09 WatchSource:0}: Error finding container 7a5ee9f636ab404ba03db3da155dedee0e6a276b5932f13374be69aab67f5a09: Status 404 returned error can't find the container with id 7a5ee9f636ab404ba03db3da155dedee0e6a276b5932f13374be69aab67f5a09 Sep 11 08:28:22.113849 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:22.113815 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" event={"ID":"95af2f9c-0950-4d31-adda-b15f1dc94db0","Type":"ContainerStarted","Data":"37ae6d41321b91698f2e0e3db86a78327e482e115a9cf28024bf39fed8fbf5aa"} Sep 11 08:28:22.113849 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:22.113855 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" event={"ID":"95af2f9c-0950-4d31-adda-b15f1dc94db0","Type":"ContainerStarted","Data":"7a5ee9f636ab404ba03db3da155dedee0e6a276b5932f13374be69aab67f5a09"} Sep 11 08:28:26.130930 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:26.130887 2557 generic.go:356] "Generic (PLEG): container finished" podID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerID="37ae6d41321b91698f2e0e3db86a78327e482e115a9cf28024bf39fed8fbf5aa" exitCode=0 Sep 11 08:28:26.131225 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:26.130962 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" event={"ID":"95af2f9c-0950-4d31-adda-b15f1dc94db0","Type":"ContainerDied","Data":"37ae6d41321b91698f2e0e3db86a78327e482e115a9cf28024bf39fed8fbf5aa"} Sep 11 08:28:27.137487 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:27.137454 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" event={"ID":"95af2f9c-0950-4d31-adda-b15f1dc94db0","Type":"ContainerStarted","Data":"dc6fd76134b28a2a1b45566df536978e0f2f62d0110870c453f4a113c249676a"} Sep 11 08:28:27.638737 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:27.638705 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused" Sep 11 08:28:27.867912 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:27.867867 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Sep 11 08:28:32.163781 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:32.163750 2557 generic.go:356] "Generic (PLEG): container finished" podID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerID="dc6fd76134b28a2a1b45566df536978e0f2f62d0110870c453f4a113c249676a" exitCode=0 Sep 11 08:28:32.164134 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:32.163823 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" event={"ID":"95af2f9c-0950-4d31-adda-b15f1dc94db0","Type":"ContainerDied","Data":"dc6fd76134b28a2a1b45566df536978e0f2f62d0110870c453f4a113c249676a"} Sep 11 08:28:33.173495 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:33.173452 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" event={"ID":"95af2f9c-0950-4d31-adda-b15f1dc94db0","Type":"ContainerStarted","Data":"b102810ec48765c8c035b0e50011e539531b1386ffae38a512d646c8cf153a2d"} Sep 11 08:28:33.173988 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:33.173519 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" event={"ID":"95af2f9c-0950-4d31-adda-b15f1dc94db0","Type":"ContainerStarted","Data":"a6f252caf6f1e2a64ef7bc6e1d7310ca03d630e15c18980d7beefa9e7aadfb34"} Sep 11 08:28:33.173988 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:33.173541 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" event={"ID":"95af2f9c-0950-4d31-adda-b15f1dc94db0","Type":"ContainerStarted","Data":"0c84a44f4a59873dea56104d5280debb097815780bf3560ad683efffb0f7750d"} Sep 11 08:28:33.173988 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:33.173561 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" event={"ID":"95af2f9c-0950-4d31-adda-b15f1dc94db0","Type":"ContainerStarted","Data":"7280aa2447c0b0380315fa177ed739233095a5e9e30383b564f4dabb619a93f2"} Sep 11 08:28:33.207718 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:33.207658 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" podStartSLOduration=13.207634236 podStartE2EDuration="13.207634236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:28:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:28:33.205418819 +0000 UTC m=+1750.211136490" watchObservedRunningTime="2026-09-11 08:28:33.207634236 +0000 UTC m=+1750.213351895" Sep 11 08:28:36.016694 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:36.015763 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:36.016694 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:36.015822 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:36.016694 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:36.015842 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:36.016694 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:36.015860 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:36.017760 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:36.017720 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.33:8443: connect: connection refused" Sep 11 08:28:36.017881 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:36.017803 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="online" probeResult="failure" output="Get \"https://10.132.0.33:6567/health\": dial tcp 10.132.0.33:6567: connect: connection refused" Sep 11 08:28:36.017881 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:36.017809 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.33:6571: connect: connection refused" Sep 11 08:28:36.018120 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:36.018084 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.33:8816: connect: connection refused" Sep 11 08:28:37.638687 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:37.638640 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused" Sep 11 08:28:37.868698 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:37.868650 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Sep 11 08:28:39.015863 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:39.015807 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.33:8443: connect: connection refused" Sep 11 08:28:39.016369 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:39.015819 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.33:6571: connect: connection refused" Sep 11 08:28:39.016369 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:39.016011 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="online" probeResult="failure" output="Get \"https://10.132.0.33:6567/health\": dial tcp 10.132.0.33:6567: connect: connection refused" Sep 11 08:28:39.016369 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:39.015871 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.33:8816: connect: connection refused" Sep 11 08:28:41.015875 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:41.015839 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:41.016247 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:41.015933 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:41.016247 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:41.015950 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:41.016247 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:41.015960 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:42.016607 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:42.016571 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:42.017076 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:42.016827 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:42.017405 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:42.017382 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:42.017524 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:42.017459 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:42.017524 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:42.017514 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:42.022487 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:42.022463 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:42.219765 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:42.219744 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:42.222438 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:42.222422 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:28:47.638932 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:47.638855 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused" Sep 11 08:28:47.639398 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:47.639016 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:28:47.868158 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:47.868120 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Sep 11 08:28:47.868312 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:47.868260 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:28:50.253389 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.253322 2557 generic.go:356] "Generic (PLEG): container finished" podID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerID="374174abf83b39fce7dc54e44d8c6cb79a7a39d010bff1d97373c93aa7edebd9" exitCode=137 Sep 11 08:28:50.253389 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.253347 2557 generic.go:356] "Generic (PLEG): container finished" podID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerID="cdb02aa79485f785712ab81a673d5261e746d953ac8360f64d7cb6f5d8aaef1e" exitCode=137 Sep 11 08:28:50.253721 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.253394 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" event={"ID":"1f9d74ae-f5c1-4e37-a473-94ee838160f3","Type":"ContainerDied","Data":"374174abf83b39fce7dc54e44d8c6cb79a7a39d010bff1d97373c93aa7edebd9"} Sep 11 08:28:50.253721 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.253438 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" event={"ID":"1f9d74ae-f5c1-4e37-a473-94ee838160f3","Type":"ContainerDied","Data":"cdb02aa79485f785712ab81a673d5261e746d953ac8360f64d7cb6f5d8aaef1e"} Sep 11 08:28:50.541234 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.541210 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:28:50.640479 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.640456 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-offline-tls\") pod \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " Sep 11 08:28:50.640591 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.640494 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sf822\" (UniqueName: \"kubernetes.io/projected/1f9d74ae-f5c1-4e37-a473-94ee838160f3-kube-api-access-sf822\") pod \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " Sep 11 08:28:50.640591 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.640522 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-registry-tls\") pod \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " Sep 11 08:28:50.640591 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.640579 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1f9d74ae-f5c1-4e37-a473-94ee838160f3-feast-data\") pod \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " Sep 11 08:28:50.640761 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.640626 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-online-tls\") pod \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " Sep 11 08:28:50.640761 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.640692 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-ui-tls\") pod \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\" (UID: \"1f9d74ae-f5c1-4e37-a473-94ee838160f3\") " Sep 11 08:28:50.641221 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.641169 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9d74ae-f5c1-4e37-a473-94ee838160f3-feast-data" (OuterVolumeSpecName: "feast-data") pod "1f9d74ae-f5c1-4e37-a473-94ee838160f3" (UID: "1f9d74ae-f5c1-4e37-a473-94ee838160f3"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 08:28:50.642976 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.642930 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f9d74ae-f5c1-4e37-a473-94ee838160f3-kube-api-access-sf822" (OuterVolumeSpecName: "kube-api-access-sf822") pod "1f9d74ae-f5c1-4e37-a473-94ee838160f3" (UID: "1f9d74ae-f5c1-4e37-a473-94ee838160f3"). InnerVolumeSpecName "kube-api-access-sf822". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:28:50.643106 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.643031 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "1f9d74ae-f5c1-4e37-a473-94ee838160f3" (UID: "1f9d74ae-f5c1-4e37-a473-94ee838160f3"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:28:50.643106 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.643056 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1f9d74ae-f5c1-4e37-a473-94ee838160f3" (UID: "1f9d74ae-f5c1-4e37-a473-94ee838160f3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:28:50.643220 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.643120 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-online-tls" (OuterVolumeSpecName: "online-tls") pod "1f9d74ae-f5c1-4e37-a473-94ee838160f3" (UID: "1f9d74ae-f5c1-4e37-a473-94ee838160f3"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:28:50.643220 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.643142 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "1f9d74ae-f5c1-4e37-a473-94ee838160f3" (UID: "1f9d74ae-f5c1-4e37-a473-94ee838160f3"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:28:50.741779 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.741753 2557 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1f9d74ae-f5c1-4e37-a473-94ee838160f3-feast-data\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:28:50.741779 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.741775 2557 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-online-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:28:50.741779 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.741784 2557 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-ui-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:28:50.741977 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.741791 2557 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-offline-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:28:50.741977 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.741800 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sf822\" (UniqueName: \"kubernetes.io/projected/1f9d74ae-f5c1-4e37-a473-94ee838160f3-kube-api-access-sf822\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:28:50.741977 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:50.741809 2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1f9d74ae-f5c1-4e37-a473-94ee838160f3-registry-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:28:51.260625 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:51.260594 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" event={"ID":"1f9d74ae-f5c1-4e37-a473-94ee838160f3","Type":"ContainerDied","Data":"fabd3dc0d39a4ab77c2a917d09968832e2d658bc44875c4cccb4d9f1825826d2"} Sep 11 08:28:51.261051 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:51.260643 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn" Sep 11 08:28:51.261051 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:51.260648 2557 scope.go:117] "RemoveContainer" containerID="bcca4b6aca1d071f7f4181aec43979541efe22f0d4ae0978a27e534ad98d1f7f" Sep 11 08:28:51.269876 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:51.269855 2557 scope.go:117] "RemoveContainer" containerID="374174abf83b39fce7dc54e44d8c6cb79a7a39d010bff1d97373c93aa7edebd9" Sep 11 08:28:51.277317 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:51.277302 2557 scope.go:117] "RemoveContainer" containerID="b3f5353758f689ee130104ac3999fada89897226f181e85807afdbde00a510bb" Sep 11 08:28:51.284752 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:51.284727 2557 scope.go:117] "RemoveContainer" containerID="cdb02aa79485f785712ab81a673d5261e746d953ac8360f64d7cb6f5d8aaef1e" Sep 11 08:28:51.285819 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:51.285756 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn"] Sep 11 08:28:51.291650 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:51.291632 2557 scope.go:117] "RemoveContainer" containerID="f93fd31e4df33f9c320d9d43f1581ab1b295aae6296f4192edd9b04859803dbf" Sep 11 08:28:51.297357 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:51.297334 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-skzdn"] Sep 11 08:28:51.298837 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:51.298821 2557 scope.go:117] "RemoveContainer" containerID="858d2980839a95ca7b35192aaaac0d20f5c1a93da7eb816d3612442087582bac" Sep 11 08:28:51.502311 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:28:51.502286 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" path="/var/lib/kubelet/pods/1f9d74ae-f5c1-4e37-a473-94ee838160f3/volumes" Sep 11 08:29:45.070826 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.070790 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr"] Sep 11 08:29:45.071226 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.071060 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="registry" Sep 11 08:29:45.071226 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.071072 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="registry" Sep 11 08:29:45.071226 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.071084 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="feast-init" Sep 11 08:29:45.071226 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.071089 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="feast-init" Sep 11 08:29:45.071226 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.071102 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="feast-apply" Sep 11 08:29:45.071226 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.071107 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="feast-apply" Sep 11 08:29:45.071226 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.071114 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="online" Sep 11 08:29:45.071226 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.071119 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="online" Sep 11 08:29:45.071226 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.071126 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="ui" Sep 11 08:29:45.071226 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.071130 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="ui" Sep 11 08:29:45.071226 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.071138 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="offline" Sep 11 08:29:45.071226 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.071143 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="offline" Sep 11 08:29:45.071226 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.071185 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="offline" Sep 11 08:29:45.071226 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.071194 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="online" Sep 11 08:29:45.071226 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.071200 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="registry" Sep 11 08:29:45.071226 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.071206 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f9d74ae-f5c1-4e37-a473-94ee838160f3" containerName="ui" Sep 11 08:29:45.074367 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.074351 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:45.077223 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.077172 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 11 08:29:45.077223 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.077191 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-r42r8\"" Sep 11 08:29:45.077454 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.077174 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 11 08:29:45.077454 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.077180 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 11 08:29:45.077454 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.077182 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 11 08:29:45.077454 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.077174 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 11 08:29:45.077619 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.077468 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 11 08:29:45.092695 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.092674 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr"] Sep 11 08:29:45.111281 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.111260 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-j46rr\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:45.111401 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.111320 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/5d1b50ed-fd5f-4923-9863-10194e9e35c6-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-j46rr\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:45.111401 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.111369 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5d1b50ed-fd5f-4923-9863-10194e9e35c6-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-j46rr\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:45.111559 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.111420 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-j46rr\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:45.111622 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.111566 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdxk6\" (UniqueName: \"kubernetes.io/projected/5d1b50ed-fd5f-4923-9863-10194e9e35c6-kube-api-access-xdxk6\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-j46rr\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:45.111622 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.111595 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-j46rr\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:45.212102 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.212071 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdxk6\" (UniqueName: \"kubernetes.io/projected/5d1b50ed-fd5f-4923-9863-10194e9e35c6-kube-api-access-xdxk6\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-j46rr\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:45.212228 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.212113 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-j46rr\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:45.212228 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.212145 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-j46rr\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:45.212228 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.212188 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/5d1b50ed-fd5f-4923-9863-10194e9e35c6-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-j46rr\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:45.212228 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.212222 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5d1b50ed-fd5f-4923-9863-10194e9e35c6-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-j46rr\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:45.212438 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.212247 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-j46rr\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:45.212438 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:29:45.212292 2557 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 11 08:29:45.212438 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:29:45.212347 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-ui-tls podName:5d1b50ed-fd5f-4923-9863-10194e9e35c6 nodeName:}" failed. No retries permitted until 2026-09-11 08:29:45.712328053 +0000 UTC m=+1822.718045708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-ui-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-j46rr" (UID: "5d1b50ed-fd5f-4923-9863-10194e9e35c6") : secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 11 08:29:45.212438 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:29:45.212390 2557 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found Sep 11 08:29:45.212438 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:29:45.212432 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-offline-tls podName:5d1b50ed-fd5f-4923-9863-10194e9e35c6 nodeName:}" failed. No retries permitted until 2026-09-11 08:29:45.712418586 +0000 UTC m=+1822.718136232 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-offline-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-j46rr" (UID: "5d1b50ed-fd5f-4923-9863-10194e9e35c6") : secret "feast-simple-feast-remote-setup-offline-tls" not found Sep 11 08:29:45.212626 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.212586 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5d1b50ed-fd5f-4923-9863-10194e9e35c6-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-j46rr\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:45.212787 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.212771 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/5d1b50ed-fd5f-4923-9863-10194e9e35c6-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-j46rr\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:45.214635 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.214614 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-j46rr\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:45.221668 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.221642 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdxk6\" (UniqueName: \"kubernetes.io/projected/5d1b50ed-fd5f-4923-9863-10194e9e35c6-kube-api-access-xdxk6\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-j46rr\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:45.715150 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.715120 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-j46rr\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:45.715306 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.715189 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-j46rr\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:45.717457 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.717431 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-j46rr\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:45.717557 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.717467 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-j46rr\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:45.983978 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:45.983924 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:29:46.262759 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:46.262731 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr"] Sep 11 08:29:46.264677 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:29:46.264652 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d1b50ed_fd5f_4923_9863_10194e9e35c6.slice/crio-c39a104eaba2d39980530b67da8c20f501b5b4824cf5895c9e8c6a6f723aed33 WatchSource:0}: Error finding container c39a104eaba2d39980530b67da8c20f501b5b4824cf5895c9e8c6a6f723aed33: Status 404 returned error can't find the container with id c39a104eaba2d39980530b67da8c20f501b5b4824cf5895c9e8c6a6f723aed33 Sep 11 08:29:46.475354 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:46.475320 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" event={"ID":"5d1b50ed-fd5f-4923-9863-10194e9e35c6","Type":"ContainerStarted","Data":"8baab2cad49cdd0bce192289bf647f02e523d04ef50566f1fdabd603997e7c4e"} Sep 11 08:29:46.475503 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:46.475362 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" event={"ID":"5d1b50ed-fd5f-4923-9863-10194e9e35c6","Type":"ContainerStarted","Data":"c39a104eaba2d39980530b67da8c20f501b5b4824cf5895c9e8c6a6f723aed33"} Sep 11 08:29:51.495904 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:51.495876 2557 generic.go:356] "Generic (PLEG): container finished" podID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerID="8baab2cad49cdd0bce192289bf647f02e523d04ef50566f1fdabd603997e7c4e" exitCode=0 Sep 11 08:29:51.496227 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:51.495953 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" event={"ID":"5d1b50ed-fd5f-4923-9863-10194e9e35c6","Type":"ContainerDied","Data":"8baab2cad49cdd0bce192289bf647f02e523d04ef50566f1fdabd603997e7c4e"} Sep 11 08:29:52.501770 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:29:52.501738 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" event={"ID":"5d1b50ed-fd5f-4923-9863-10194e9e35c6","Type":"ContainerStarted","Data":"24bafec7c19b3abfda87874c13428a3446e0b72c7b8397829d50acb4c32c4e5f"} Sep 11 08:30:22.615393 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:22.615356 2557 generic.go:356] "Generic (PLEG): container finished" podID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerID="24bafec7c19b3abfda87874c13428a3446e0b72c7b8397829d50acb4c32c4e5f" exitCode=0 Sep 11 08:30:22.615805 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:22.615428 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" event={"ID":"5d1b50ed-fd5f-4923-9863-10194e9e35c6","Type":"ContainerDied","Data":"24bafec7c19b3abfda87874c13428a3446e0b72c7b8397829d50acb4c32c4e5f"} Sep 11 08:30:23.624273 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:23.624236 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" event={"ID":"5d1b50ed-fd5f-4923-9863-10194e9e35c6","Type":"ContainerStarted","Data":"ce15261fd36439ae4d6512b30a1ac820b1c8c70e681e455ec32965ec3b91a9e0"} Sep 11 08:30:23.624619 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:23.624296 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" event={"ID":"5d1b50ed-fd5f-4923-9863-10194e9e35c6","Type":"ContainerStarted","Data":"5f9ba577aceaf4b1260aba148a60dcb2e5b2a21be6862474e52e7957d668585d"} Sep 11 08:30:23.624619 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:23.624316 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" event={"ID":"5d1b50ed-fd5f-4923-9863-10194e9e35c6","Type":"ContainerStarted","Data":"73ebd6bd179b7b1cce29b75d178493c1fded32dcab12e8706493978f56384609"} Sep 11 08:30:23.649844 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:23.649790 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podStartSLOduration=38.649771134 podStartE2EDuration="38.649771134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:29:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:30:23.648040551 +0000 UTC m=+1860.653758209" watchObservedRunningTime="2026-09-11 08:30:23.649771134 +0000 UTC m=+1860.655488825" Sep 11 08:30:24.984651 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:24.984608 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:30:24.985126 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:24.984747 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:30:24.985126 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:24.984770 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:30:24.986559 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:24.986519 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.34:8443: connect: connection refused" Sep 11 08:30:24.986753 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:24.986576 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": dial tcp 10.132.0.34:6567: connect: connection refused" Sep 11 08:30:24.986753 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:24.986532 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.34:8816: connect: connection refused" Sep 11 08:30:25.984790 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:25.984753 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:30:25.985366 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:25.984823 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:30:25.985366 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:25.984843 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:30:27.984573 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:27.984507 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.34:8443: connect: connection refused" Sep 11 08:30:27.984573 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:27.984546 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": dial tcp 10.132.0.34:6567: connect: connection refused" Sep 11 08:30:27.985021 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:27.984507 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.34:8816: connect: connection refused" Sep 11 08:30:30.984615 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:30.984574 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": dial tcp 10.132.0.34:6567: connect: connection refused" Sep 11 08:30:30.985111 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:30.985090 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:30:30.985242 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:30.985228 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:30:30.985837 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:30.985817 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:30:30.985958 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:30.985947 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:30:34.990011 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:34.989975 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 11 08:30:37.988020 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:37.987985 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 11 08:30:40.990035 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:40.989994 2557 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 11 08:30:43.939144 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:43.939115 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:30:44.921106 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:30:44.921076 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:31:04.944066 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:04.944023 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 11 08:31:06.988008 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:06.987969 2557 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 11 08:31:14.945036 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:14.944991 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 11 08:31:34.944079 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:34.944033 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 11 08:31:46.989017 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:46.988982 2557 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 11 08:31:47.239874 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:47.239792 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr"] Sep 11 08:31:47.240118 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:47.240081 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="online" containerID="cri-o://73ebd6bd179b7b1cce29b75d178493c1fded32dcab12e8706493978f56384609" gracePeriod=30 Sep 11 08:31:47.240261 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:47.240110 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="ui" containerID="cri-o://ce15261fd36439ae4d6512b30a1ac820b1c8c70e681e455ec32965ec3b91a9e0" gracePeriod=30 Sep 11 08:31:47.240261 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:47.240168 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="offline" containerID="cri-o://5f9ba577aceaf4b1260aba148a60dcb2e5b2a21be6862474e52e7957d668585d" gracePeriod=30 Sep 11 08:31:47.433243 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:47.433209 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll"] Sep 11 08:31:47.433521 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:47.433493 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="registry" containerID="cri-o://7280aa2447c0b0380315fa177ed739233095a5e9e30383b564f4dabb619a93f2" gracePeriod=30 Sep 11 08:31:47.433623 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:47.433590 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="offline" containerID="cri-o://a6f252caf6f1e2a64ef7bc6e1d7310ca03d630e15c18980d7beefa9e7aadfb34" gracePeriod=30 Sep 11 08:31:47.433686 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:47.433635 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="online" containerID="cri-o://0c84a44f4a59873dea56104d5280debb097815780bf3560ad683efffb0f7750d" gracePeriod=30 Sep 11 08:31:47.433746 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:47.433602 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="ui" containerID="cri-o://b102810ec48765c8c035b0e50011e539531b1386ffae38a512d646c8cf153a2d" gracePeriod=30 Sep 11 08:31:47.952665 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:47.952619 2557 generic.go:356] "Generic (PLEG): container finished" podID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerID="ce15261fd36439ae4d6512b30a1ac820b1c8c70e681e455ec32965ec3b91a9e0" exitCode=0 Sep 11 08:31:47.952832 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:47.952674 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" event={"ID":"5d1b50ed-fd5f-4923-9863-10194e9e35c6","Type":"ContainerDied","Data":"ce15261fd36439ae4d6512b30a1ac820b1c8c70e681e455ec32965ec3b91a9e0"} Sep 11 08:31:48.960025 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:48.959990 2557 generic.go:356] "Generic (PLEG): container finished" podID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerID="73ebd6bd179b7b1cce29b75d178493c1fded32dcab12e8706493978f56384609" exitCode=0 Sep 11 08:31:48.960405 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:48.960063 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" event={"ID":"5d1b50ed-fd5f-4923-9863-10194e9e35c6","Type":"ContainerDied","Data":"73ebd6bd179b7b1cce29b75d178493c1fded32dcab12e8706493978f56384609"} Sep 11 08:31:48.963174 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:48.963152 2557 generic.go:356] "Generic (PLEG): container finished" podID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerID="b102810ec48765c8c035b0e50011e539531b1386ffae38a512d646c8cf153a2d" exitCode=0 Sep 11 08:31:48.963174 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:48.963171 2557 generic.go:356] "Generic (PLEG): container finished" podID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerID="0c84a44f4a59873dea56104d5280debb097815780bf3560ad683efffb0f7750d" exitCode=0 Sep 11 08:31:48.963313 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:48.963202 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" event={"ID":"95af2f9c-0950-4d31-adda-b15f1dc94db0","Type":"ContainerDied","Data":"b102810ec48765c8c035b0e50011e539531b1386ffae38a512d646c8cf153a2d"} Sep 11 08:31:48.963313 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:48.963227 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" event={"ID":"95af2f9c-0950-4d31-adda-b15f1dc94db0","Type":"ContainerDied","Data":"0c84a44f4a59873dea56104d5280debb097815780bf3560ad683efffb0f7750d"} Sep 11 08:31:50.986579 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:50.986535 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.34:8443: connect: connection refused" Sep 11 08:31:52.017783 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:52.017742 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.33:8443: connect: connection refused" Sep 11 08:31:52.220194 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:52.220154 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="online" probeResult="failure" output="Get \"https://10.132.0.33:6567/health\": dial tcp 10.132.0.33:6567: connect: connection refused" Sep 11 08:31:53.939817 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:31:53.939781 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": dial tcp 10.132.0.34:6567: connect: connection refused" Sep 11 08:32:00.986055 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:00.986006 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.34:8443: connect: connection refused" Sep 11 08:32:02.017122 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:02.017079 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.33:8443: connect: connection refused" Sep 11 08:32:02.220375 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:02.220333 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="online" probeResult="failure" output="Get \"https://10.132.0.33:6567/health\": dial tcp 10.132.0.33:6567: connect: connection refused" Sep 11 08:32:03.939651 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:03.939607 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": dial tcp 10.132.0.34:6567: connect: connection refused" Sep 11 08:32:10.986152 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:10.986099 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.34:8443: connect: connection refused" Sep 11 08:32:10.986634 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:10.986277 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:32:12.017926 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:12.017872 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.33:8443: connect: connection refused" Sep 11 08:32:12.018415 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:12.018036 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:32:12.220378 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:12.220334 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="online" probeResult="failure" output="Get \"https://10.132.0.33:6567/health\": dial tcp 10.132.0.33:6567: connect: connection refused" Sep 11 08:32:12.220555 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:12.220479 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:32:13.939977 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:13.939938 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": dial tcp 10.132.0.34:6567: connect: connection refused" Sep 11 08:32:13.940334 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:13.940067 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:32:17.881337 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:17.881310 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:32:17.972890 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:17.972864 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-ui-tls\") pod \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " Sep 11 08:32:17.973022 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:17.972918 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdxk6\" (UniqueName: \"kubernetes.io/projected/5d1b50ed-fd5f-4923-9863-10194e9e35c6-kube-api-access-xdxk6\") pod \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " Sep 11 08:32:17.973022 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:17.972954 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5d1b50ed-fd5f-4923-9863-10194e9e35c6-feast-data\") pod \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " Sep 11 08:32:17.973022 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:17.972977 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/5d1b50ed-fd5f-4923-9863-10194e9e35c6-registry-tls\") pod \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " Sep 11 08:32:17.973173 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:17.973036 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-offline-tls\") pod \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " Sep 11 08:32:17.973173 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:17.973057 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-online-tls\") pod \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\" (UID: \"5d1b50ed-fd5f-4923-9863-10194e9e35c6\") " Sep 11 08:32:17.973480 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:17.973446 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d1b50ed-fd5f-4923-9863-10194e9e35c6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5d1b50ed-fd5f-4923-9863-10194e9e35c6" (UID: "5d1b50ed-fd5f-4923-9863-10194e9e35c6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:32:17.973588 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:17.973493 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d1b50ed-fd5f-4923-9863-10194e9e35c6-feast-data" (OuterVolumeSpecName: "feast-data") pod "5d1b50ed-fd5f-4923-9863-10194e9e35c6" (UID: "5d1b50ed-fd5f-4923-9863-10194e9e35c6"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 08:32:17.975146 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:17.975123 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "5d1b50ed-fd5f-4923-9863-10194e9e35c6" (UID: "5d1b50ed-fd5f-4923-9863-10194e9e35c6"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:32:17.975230 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:17.975124 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d1b50ed-fd5f-4923-9863-10194e9e35c6-kube-api-access-xdxk6" (OuterVolumeSpecName: "kube-api-access-xdxk6") pod "5d1b50ed-fd5f-4923-9863-10194e9e35c6" (UID: "5d1b50ed-fd5f-4923-9863-10194e9e35c6"). InnerVolumeSpecName "kube-api-access-xdxk6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:32:17.975352 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:17.975330 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-online-tls" (OuterVolumeSpecName: "online-tls") pod "5d1b50ed-fd5f-4923-9863-10194e9e35c6" (UID: "5d1b50ed-fd5f-4923-9863-10194e9e35c6"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:32:17.975554 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:17.975531 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "5d1b50ed-fd5f-4923-9863-10194e9e35c6" (UID: "5d1b50ed-fd5f-4923-9863-10194e9e35c6"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:32:18.065330 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.065303 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:32:18.073520 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.073503 2557 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5d1b50ed-fd5f-4923-9863-10194e9e35c6-feast-data\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:32:18.073520 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.073522 2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/5d1b50ed-fd5f-4923-9863-10194e9e35c6-registry-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:32:18.073641 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.073530 2557 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-offline-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:32:18.073641 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.073538 2557 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-online-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:32:18.073641 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.073546 2557 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5d1b50ed-fd5f-4923-9863-10194e9e35c6-ui-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:32:18.073641 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.073553 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdxk6\" (UniqueName: \"kubernetes.io/projected/5d1b50ed-fd5f-4923-9863-10194e9e35c6-kube-api-access-xdxk6\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:32:18.078351 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.078327 2557 generic.go:356] "Generic (PLEG): container finished" podID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerID="a6f252caf6f1e2a64ef7bc6e1d7310ca03d630e15c18980d7beefa9e7aadfb34" exitCode=137 Sep 11 08:32:18.078351 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.078351 2557 generic.go:356] "Generic (PLEG): container finished" podID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerID="7280aa2447c0b0380315fa177ed739233095a5e9e30383b564f4dabb619a93f2" exitCode=137 Sep 11 08:32:18.078471 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.078371 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" event={"ID":"95af2f9c-0950-4d31-adda-b15f1dc94db0","Type":"ContainerDied","Data":"a6f252caf6f1e2a64ef7bc6e1d7310ca03d630e15c18980d7beefa9e7aadfb34"} Sep 11 08:32:18.078471 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.078391 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" Sep 11 08:32:18.078471 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.078420 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" event={"ID":"95af2f9c-0950-4d31-adda-b15f1dc94db0","Type":"ContainerDied","Data":"7280aa2447c0b0380315fa177ed739233095a5e9e30383b564f4dabb619a93f2"} Sep 11 08:32:18.078471 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.078437 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll" event={"ID":"95af2f9c-0950-4d31-adda-b15f1dc94db0","Type":"ContainerDied","Data":"7a5ee9f636ab404ba03db3da155dedee0e6a276b5932f13374be69aab67f5a09"} Sep 11 08:32:18.078471 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.078463 2557 scope.go:117] "RemoveContainer" containerID="b102810ec48765c8c035b0e50011e539531b1386ffae38a512d646c8cf153a2d" Sep 11 08:32:18.080952 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.080933 2557 generic.go:356] "Generic (PLEG): container finished" podID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerID="5f9ba577aceaf4b1260aba148a60dcb2e5b2a21be6862474e52e7957d668585d" exitCode=137 Sep 11 08:32:18.081058 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.081002 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" Sep 11 08:32:18.081127 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.081002 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" event={"ID":"5d1b50ed-fd5f-4923-9863-10194e9e35c6","Type":"ContainerDied","Data":"5f9ba577aceaf4b1260aba148a60dcb2e5b2a21be6862474e52e7957d668585d"} Sep 11 08:32:18.081127 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.081103 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr" event={"ID":"5d1b50ed-fd5f-4923-9863-10194e9e35c6","Type":"ContainerDied","Data":"c39a104eaba2d39980530b67da8c20f501b5b4824cf5895c9e8c6a6f723aed33"} Sep 11 08:32:18.088171 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.088141 2557 scope.go:117] "RemoveContainer" containerID="a6f252caf6f1e2a64ef7bc6e1d7310ca03d630e15c18980d7beefa9e7aadfb34" Sep 11 08:32:18.096915 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.096876 2557 scope.go:117] "RemoveContainer" containerID="0c84a44f4a59873dea56104d5280debb097815780bf3560ad683efffb0f7750d" Sep 11 08:32:18.105369 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.105352 2557 scope.go:117] "RemoveContainer" containerID="7280aa2447c0b0380315fa177ed739233095a5e9e30383b564f4dabb619a93f2" Sep 11 08:32:18.109129 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.109110 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr"] Sep 11 08:32:18.112788 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.112771 2557 scope.go:117] "RemoveContainer" containerID="dc6fd76134b28a2a1b45566df536978e0f2f62d0110870c453f4a113c249676a" Sep 11 08:32:18.115943 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.115923 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-j46rr"] Sep 11 08:32:18.119881 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.119864 2557 scope.go:117] "RemoveContainer" containerID="37ae6d41321b91698f2e0e3db86a78327e482e115a9cf28024bf39fed8fbf5aa" Sep 11 08:32:18.126413 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.126397 2557 scope.go:117] "RemoveContainer" containerID="b102810ec48765c8c035b0e50011e539531b1386ffae38a512d646c8cf153a2d" Sep 11 08:32:18.126653 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:32:18.126634 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b102810ec48765c8c035b0e50011e539531b1386ffae38a512d646c8cf153a2d\": container with ID starting with b102810ec48765c8c035b0e50011e539531b1386ffae38a512d646c8cf153a2d not found: ID does not exist" containerID="b102810ec48765c8c035b0e50011e539531b1386ffae38a512d646c8cf153a2d" Sep 11 08:32:18.126712 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.126660 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b102810ec48765c8c035b0e50011e539531b1386ffae38a512d646c8cf153a2d"} err="failed to get container status \"b102810ec48765c8c035b0e50011e539531b1386ffae38a512d646c8cf153a2d\": rpc error: code = NotFound desc = could not find container \"b102810ec48765c8c035b0e50011e539531b1386ffae38a512d646c8cf153a2d\": container with ID starting with b102810ec48765c8c035b0e50011e539531b1386ffae38a512d646c8cf153a2d not found: ID does not exist" Sep 11 08:32:18.126712 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.126675 2557 scope.go:117] "RemoveContainer" containerID="a6f252caf6f1e2a64ef7bc6e1d7310ca03d630e15c18980d7beefa9e7aadfb34" Sep 11 08:32:18.126968 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:32:18.126950 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6f252caf6f1e2a64ef7bc6e1d7310ca03d630e15c18980d7beefa9e7aadfb34\": container with ID starting with a6f252caf6f1e2a64ef7bc6e1d7310ca03d630e15c18980d7beefa9e7aadfb34 not found: ID does not exist" containerID="a6f252caf6f1e2a64ef7bc6e1d7310ca03d630e15c18980d7beefa9e7aadfb34" Sep 11 08:32:18.127035 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.126971 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6f252caf6f1e2a64ef7bc6e1d7310ca03d630e15c18980d7beefa9e7aadfb34"} err="failed to get container status \"a6f252caf6f1e2a64ef7bc6e1d7310ca03d630e15c18980d7beefa9e7aadfb34\": rpc error: code = NotFound desc = could not find container \"a6f252caf6f1e2a64ef7bc6e1d7310ca03d630e15c18980d7beefa9e7aadfb34\": container with ID starting with a6f252caf6f1e2a64ef7bc6e1d7310ca03d630e15c18980d7beefa9e7aadfb34 not found: ID does not exist" Sep 11 08:32:18.127035 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.126985 2557 scope.go:117] "RemoveContainer" containerID="0c84a44f4a59873dea56104d5280debb097815780bf3560ad683efffb0f7750d" Sep 11 08:32:18.127198 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:32:18.127180 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c84a44f4a59873dea56104d5280debb097815780bf3560ad683efffb0f7750d\": container with ID starting with 0c84a44f4a59873dea56104d5280debb097815780bf3560ad683efffb0f7750d not found: ID does not exist" containerID="0c84a44f4a59873dea56104d5280debb097815780bf3560ad683efffb0f7750d" Sep 11 08:32:18.127246 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.127203 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c84a44f4a59873dea56104d5280debb097815780bf3560ad683efffb0f7750d"} err="failed to get container status \"0c84a44f4a59873dea56104d5280debb097815780bf3560ad683efffb0f7750d\": rpc error: code = NotFound desc = could not find container \"0c84a44f4a59873dea56104d5280debb097815780bf3560ad683efffb0f7750d\": container with ID starting with 0c84a44f4a59873dea56104d5280debb097815780bf3560ad683efffb0f7750d not found: ID does not exist" Sep 11 08:32:18.127246 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.127214 2557 scope.go:117] "RemoveContainer" containerID="7280aa2447c0b0380315fa177ed739233095a5e9e30383b564f4dabb619a93f2" Sep 11 08:32:18.127463 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:32:18.127436 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7280aa2447c0b0380315fa177ed739233095a5e9e30383b564f4dabb619a93f2\": container with ID starting with 7280aa2447c0b0380315fa177ed739233095a5e9e30383b564f4dabb619a93f2 not found: ID does not exist" containerID="7280aa2447c0b0380315fa177ed739233095a5e9e30383b564f4dabb619a93f2" Sep 11 08:32:18.127509 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.127471 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7280aa2447c0b0380315fa177ed739233095a5e9e30383b564f4dabb619a93f2"} err="failed to get container status \"7280aa2447c0b0380315fa177ed739233095a5e9e30383b564f4dabb619a93f2\": rpc error: code = NotFound desc = could not find container \"7280aa2447c0b0380315fa177ed739233095a5e9e30383b564f4dabb619a93f2\": container with ID starting with 7280aa2447c0b0380315fa177ed739233095a5e9e30383b564f4dabb619a93f2 not found: ID does not exist" Sep 11 08:32:18.127509 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.127487 2557 scope.go:117] "RemoveContainer" containerID="dc6fd76134b28a2a1b45566df536978e0f2f62d0110870c453f4a113c249676a" Sep 11 08:32:18.127756 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:32:18.127737 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc6fd76134b28a2a1b45566df536978e0f2f62d0110870c453f4a113c249676a\": container with ID starting with dc6fd76134b28a2a1b45566df536978e0f2f62d0110870c453f4a113c249676a not found: ID does not exist" containerID="dc6fd76134b28a2a1b45566df536978e0f2f62d0110870c453f4a113c249676a" Sep 11 08:32:18.127822 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.127760 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc6fd76134b28a2a1b45566df536978e0f2f62d0110870c453f4a113c249676a"} err="failed to get container status \"dc6fd76134b28a2a1b45566df536978e0f2f62d0110870c453f4a113c249676a\": rpc error: code = NotFound desc = could not find container \"dc6fd76134b28a2a1b45566df536978e0f2f62d0110870c453f4a113c249676a\": container with ID starting with dc6fd76134b28a2a1b45566df536978e0f2f62d0110870c453f4a113c249676a not found: ID does not exist" Sep 11 08:32:18.127822 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.127775 2557 scope.go:117] "RemoveContainer" containerID="37ae6d41321b91698f2e0e3db86a78327e482e115a9cf28024bf39fed8fbf5aa" Sep 11 08:32:18.128060 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:32:18.128044 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37ae6d41321b91698f2e0e3db86a78327e482e115a9cf28024bf39fed8fbf5aa\": container with ID starting with 37ae6d41321b91698f2e0e3db86a78327e482e115a9cf28024bf39fed8fbf5aa not found: ID does not exist" containerID="37ae6d41321b91698f2e0e3db86a78327e482e115a9cf28024bf39fed8fbf5aa" Sep 11 08:32:18.128107 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.128062 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37ae6d41321b91698f2e0e3db86a78327e482e115a9cf28024bf39fed8fbf5aa"} err="failed to get container status \"37ae6d41321b91698f2e0e3db86a78327e482e115a9cf28024bf39fed8fbf5aa\": rpc error: code = NotFound desc = could not find container \"37ae6d41321b91698f2e0e3db86a78327e482e115a9cf28024bf39fed8fbf5aa\": container with ID starting with 37ae6d41321b91698f2e0e3db86a78327e482e115a9cf28024bf39fed8fbf5aa not found: ID does not exist" Sep 11 08:32:18.128107 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.128076 2557 scope.go:117] "RemoveContainer" containerID="b102810ec48765c8c035b0e50011e539531b1386ffae38a512d646c8cf153a2d" Sep 11 08:32:18.128288 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.128272 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b102810ec48765c8c035b0e50011e539531b1386ffae38a512d646c8cf153a2d"} err="failed to get container status \"b102810ec48765c8c035b0e50011e539531b1386ffae38a512d646c8cf153a2d\": rpc error: code = NotFound desc = could not find container \"b102810ec48765c8c035b0e50011e539531b1386ffae38a512d646c8cf153a2d\": container with ID starting with b102810ec48765c8c035b0e50011e539531b1386ffae38a512d646c8cf153a2d not found: ID does not exist" Sep 11 08:32:18.128333 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.128288 2557 scope.go:117] "RemoveContainer" containerID="a6f252caf6f1e2a64ef7bc6e1d7310ca03d630e15c18980d7beefa9e7aadfb34" Sep 11 08:32:18.128509 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.128490 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6f252caf6f1e2a64ef7bc6e1d7310ca03d630e15c18980d7beefa9e7aadfb34"} err="failed to get container status \"a6f252caf6f1e2a64ef7bc6e1d7310ca03d630e15c18980d7beefa9e7aadfb34\": rpc error: code = NotFound desc = could not find container \"a6f252caf6f1e2a64ef7bc6e1d7310ca03d630e15c18980d7beefa9e7aadfb34\": container with ID starting with a6f252caf6f1e2a64ef7bc6e1d7310ca03d630e15c18980d7beefa9e7aadfb34 not found: ID does not exist" Sep 11 08:32:18.128509 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.128507 2557 scope.go:117] "RemoveContainer" containerID="0c84a44f4a59873dea56104d5280debb097815780bf3560ad683efffb0f7750d" Sep 11 08:32:18.128737 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.128720 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c84a44f4a59873dea56104d5280debb097815780bf3560ad683efffb0f7750d"} err="failed to get container status \"0c84a44f4a59873dea56104d5280debb097815780bf3560ad683efffb0f7750d\": rpc error: code = NotFound desc = could not find container \"0c84a44f4a59873dea56104d5280debb097815780bf3560ad683efffb0f7750d\": container with ID starting with 0c84a44f4a59873dea56104d5280debb097815780bf3560ad683efffb0f7750d not found: ID does not exist" Sep 11 08:32:18.128799 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.128738 2557 scope.go:117] "RemoveContainer" containerID="7280aa2447c0b0380315fa177ed739233095a5e9e30383b564f4dabb619a93f2" Sep 11 08:32:18.128978 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.128961 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7280aa2447c0b0380315fa177ed739233095a5e9e30383b564f4dabb619a93f2"} err="failed to get container status \"7280aa2447c0b0380315fa177ed739233095a5e9e30383b564f4dabb619a93f2\": rpc error: code = NotFound desc = could not find container \"7280aa2447c0b0380315fa177ed739233095a5e9e30383b564f4dabb619a93f2\": container with ID starting with 7280aa2447c0b0380315fa177ed739233095a5e9e30383b564f4dabb619a93f2 not found: ID does not exist" Sep 11 08:32:18.129032 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.128979 2557 scope.go:117] "RemoveContainer" containerID="dc6fd76134b28a2a1b45566df536978e0f2f62d0110870c453f4a113c249676a" Sep 11 08:32:18.129180 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.129164 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc6fd76134b28a2a1b45566df536978e0f2f62d0110870c453f4a113c249676a"} err="failed to get container status \"dc6fd76134b28a2a1b45566df536978e0f2f62d0110870c453f4a113c249676a\": rpc error: code = NotFound desc = could not find container \"dc6fd76134b28a2a1b45566df536978e0f2f62d0110870c453f4a113c249676a\": container with ID starting with dc6fd76134b28a2a1b45566df536978e0f2f62d0110870c453f4a113c249676a not found: ID does not exist" Sep 11 08:32:18.129221 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.129181 2557 scope.go:117] "RemoveContainer" containerID="37ae6d41321b91698f2e0e3db86a78327e482e115a9cf28024bf39fed8fbf5aa" Sep 11 08:32:18.129368 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.129353 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37ae6d41321b91698f2e0e3db86a78327e482e115a9cf28024bf39fed8fbf5aa"} err="failed to get container status \"37ae6d41321b91698f2e0e3db86a78327e482e115a9cf28024bf39fed8fbf5aa\": rpc error: code = NotFound desc = could not find container \"37ae6d41321b91698f2e0e3db86a78327e482e115a9cf28024bf39fed8fbf5aa\": container with ID starting with 37ae6d41321b91698f2e0e3db86a78327e482e115a9cf28024bf39fed8fbf5aa not found: ID does not exist" Sep 11 08:32:18.129368 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.129367 2557 scope.go:117] "RemoveContainer" containerID="ce15261fd36439ae4d6512b30a1ac820b1c8c70e681e455ec32965ec3b91a9e0" Sep 11 08:32:18.136073 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.136059 2557 scope.go:117] "RemoveContainer" containerID="5f9ba577aceaf4b1260aba148a60dcb2e5b2a21be6862474e52e7957d668585d" Sep 11 08:32:18.142774 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.142757 2557 scope.go:117] "RemoveContainer" containerID="73ebd6bd179b7b1cce29b75d178493c1fded32dcab12e8706493978f56384609" Sep 11 08:32:18.149366 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.149331 2557 scope.go:117] "RemoveContainer" containerID="24bafec7c19b3abfda87874c13428a3446e0b72c7b8397829d50acb4c32c4e5f" Sep 11 08:32:18.156603 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.156562 2557 scope.go:117] "RemoveContainer" containerID="8baab2cad49cdd0bce192289bf647f02e523d04ef50566f1fdabd603997e7c4e" Sep 11 08:32:18.163205 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.163187 2557 scope.go:117] "RemoveContainer" containerID="ce15261fd36439ae4d6512b30a1ac820b1c8c70e681e455ec32965ec3b91a9e0" Sep 11 08:32:18.163416 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:32:18.163397 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce15261fd36439ae4d6512b30a1ac820b1c8c70e681e455ec32965ec3b91a9e0\": container with ID starting with ce15261fd36439ae4d6512b30a1ac820b1c8c70e681e455ec32965ec3b91a9e0 not found: ID does not exist" containerID="ce15261fd36439ae4d6512b30a1ac820b1c8c70e681e455ec32965ec3b91a9e0" Sep 11 08:32:18.163569 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.163423 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce15261fd36439ae4d6512b30a1ac820b1c8c70e681e455ec32965ec3b91a9e0"} err="failed to get container status \"ce15261fd36439ae4d6512b30a1ac820b1c8c70e681e455ec32965ec3b91a9e0\": rpc error: code = NotFound desc = could not find container \"ce15261fd36439ae4d6512b30a1ac820b1c8c70e681e455ec32965ec3b91a9e0\": container with ID starting with ce15261fd36439ae4d6512b30a1ac820b1c8c70e681e455ec32965ec3b91a9e0 not found: ID does not exist" Sep 11 08:32:18.163569 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.163443 2557 scope.go:117] "RemoveContainer" containerID="5f9ba577aceaf4b1260aba148a60dcb2e5b2a21be6862474e52e7957d668585d" Sep 11 08:32:18.163720 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:32:18.163704 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f9ba577aceaf4b1260aba148a60dcb2e5b2a21be6862474e52e7957d668585d\": container with ID starting with 5f9ba577aceaf4b1260aba148a60dcb2e5b2a21be6862474e52e7957d668585d not found: ID does not exist" containerID="5f9ba577aceaf4b1260aba148a60dcb2e5b2a21be6862474e52e7957d668585d" Sep 11 08:32:18.163769 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.163734 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f9ba577aceaf4b1260aba148a60dcb2e5b2a21be6862474e52e7957d668585d"} err="failed to get container status \"5f9ba577aceaf4b1260aba148a60dcb2e5b2a21be6862474e52e7957d668585d\": rpc error: code = NotFound desc = could not find container \"5f9ba577aceaf4b1260aba148a60dcb2e5b2a21be6862474e52e7957d668585d\": container with ID starting with 5f9ba577aceaf4b1260aba148a60dcb2e5b2a21be6862474e52e7957d668585d not found: ID does not exist" Sep 11 08:32:18.163769 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.163750 2557 scope.go:117] "RemoveContainer" containerID="73ebd6bd179b7b1cce29b75d178493c1fded32dcab12e8706493978f56384609" Sep 11 08:32:18.164002 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:32:18.163976 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73ebd6bd179b7b1cce29b75d178493c1fded32dcab12e8706493978f56384609\": container with ID starting with 73ebd6bd179b7b1cce29b75d178493c1fded32dcab12e8706493978f56384609 not found: ID does not exist" containerID="73ebd6bd179b7b1cce29b75d178493c1fded32dcab12e8706493978f56384609" Sep 11 08:32:18.164059 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.164006 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73ebd6bd179b7b1cce29b75d178493c1fded32dcab12e8706493978f56384609"} err="failed to get container status \"73ebd6bd179b7b1cce29b75d178493c1fded32dcab12e8706493978f56384609\": rpc error: code = NotFound desc = could not find container \"73ebd6bd179b7b1cce29b75d178493c1fded32dcab12e8706493978f56384609\": container with ID starting with 73ebd6bd179b7b1cce29b75d178493c1fded32dcab12e8706493978f56384609 not found: ID does not exist" Sep 11 08:32:18.164059 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.164020 2557 scope.go:117] "RemoveContainer" containerID="24bafec7c19b3abfda87874c13428a3446e0b72c7b8397829d50acb4c32c4e5f" Sep 11 08:32:18.164240 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:32:18.164221 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24bafec7c19b3abfda87874c13428a3446e0b72c7b8397829d50acb4c32c4e5f\": container with ID starting with 24bafec7c19b3abfda87874c13428a3446e0b72c7b8397829d50acb4c32c4e5f not found: ID does not exist" containerID="24bafec7c19b3abfda87874c13428a3446e0b72c7b8397829d50acb4c32c4e5f" Sep 11 08:32:18.164295 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.164243 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24bafec7c19b3abfda87874c13428a3446e0b72c7b8397829d50acb4c32c4e5f"} err="failed to get container status \"24bafec7c19b3abfda87874c13428a3446e0b72c7b8397829d50acb4c32c4e5f\": rpc error: code = NotFound desc = could not find container \"24bafec7c19b3abfda87874c13428a3446e0b72c7b8397829d50acb4c32c4e5f\": container with ID starting with 24bafec7c19b3abfda87874c13428a3446e0b72c7b8397829d50acb4c32c4e5f not found: ID does not exist" Sep 11 08:32:18.164295 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.164262 2557 scope.go:117] "RemoveContainer" containerID="8baab2cad49cdd0bce192289bf647f02e523d04ef50566f1fdabd603997e7c4e" Sep 11 08:32:18.164483 ip-10-0-136-245 kubenswrapper[2557]: E0911 08:32:18.164466 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8baab2cad49cdd0bce192289bf647f02e523d04ef50566f1fdabd603997e7c4e\": container with ID starting with 8baab2cad49cdd0bce192289bf647f02e523d04ef50566f1fdabd603997e7c4e not found: ID does not exist" containerID="8baab2cad49cdd0bce192289bf647f02e523d04ef50566f1fdabd603997e7c4e" Sep 11 08:32:18.164549 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.164490 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8baab2cad49cdd0bce192289bf647f02e523d04ef50566f1fdabd603997e7c4e"} err="failed to get container status \"8baab2cad49cdd0bce192289bf647f02e523d04ef50566f1fdabd603997e7c4e\": rpc error: code = NotFound desc = could not find container \"8baab2cad49cdd0bce192289bf647f02e523d04ef50566f1fdabd603997e7c4e\": container with ID starting with 8baab2cad49cdd0bce192289bf647f02e523d04ef50566f1fdabd603997e7c4e not found: ID does not exist" Sep 11 08:32:18.174721 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.174703 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwfqj\" (UniqueName: \"kubernetes.io/projected/95af2f9c-0950-4d31-adda-b15f1dc94db0-kube-api-access-rwfqj\") pod \"95af2f9c-0950-4d31-adda-b15f1dc94db0\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " Sep 11 08:32:18.174932 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.174740 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-offline-tls\") pod \"95af2f9c-0950-4d31-adda-b15f1dc94db0\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " Sep 11 08:32:18.174932 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.174764 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/95af2f9c-0950-4d31-adda-b15f1dc94db0-feast-data\") pod \"95af2f9c-0950-4d31-adda-b15f1dc94db0\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " Sep 11 08:32:18.174932 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.174786 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-online-tls\") pod \"95af2f9c-0950-4d31-adda-b15f1dc94db0\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " Sep 11 08:32:18.174932 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.174811 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-registry-tls\") pod \"95af2f9c-0950-4d31-adda-b15f1dc94db0\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " Sep 11 08:32:18.174932 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.174830 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-ui-tls\") pod \"95af2f9c-0950-4d31-adda-b15f1dc94db0\" (UID: \"95af2f9c-0950-4d31-adda-b15f1dc94db0\") " Sep 11 08:32:18.175308 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.175284 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95af2f9c-0950-4d31-adda-b15f1dc94db0-feast-data" (OuterVolumeSpecName: "feast-data") pod "95af2f9c-0950-4d31-adda-b15f1dc94db0" (UID: "95af2f9c-0950-4d31-adda-b15f1dc94db0"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 08:32:18.176740 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.176718 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "95af2f9c-0950-4d31-adda-b15f1dc94db0" (UID: "95af2f9c-0950-4d31-adda-b15f1dc94db0"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:32:18.176830 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.176759 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-online-tls" (OuterVolumeSpecName: "online-tls") pod "95af2f9c-0950-4d31-adda-b15f1dc94db0" (UID: "95af2f9c-0950-4d31-adda-b15f1dc94db0"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:32:18.176830 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.176776 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "95af2f9c-0950-4d31-adda-b15f1dc94db0" (UID: "95af2f9c-0950-4d31-adda-b15f1dc94db0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:32:18.177161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.177145 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95af2f9c-0950-4d31-adda-b15f1dc94db0-kube-api-access-rwfqj" (OuterVolumeSpecName: "kube-api-access-rwfqj") pod "95af2f9c-0950-4d31-adda-b15f1dc94db0" (UID: "95af2f9c-0950-4d31-adda-b15f1dc94db0"). InnerVolumeSpecName "kube-api-access-rwfqj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:32:18.177161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.177146 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "95af2f9c-0950-4d31-adda-b15f1dc94db0" (UID: "95af2f9c-0950-4d31-adda-b15f1dc94db0"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:32:18.276073 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.276046 2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-registry-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:32:18.276073 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.276070 2557 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-ui-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:32:18.276197 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.276083 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwfqj\" (UniqueName: \"kubernetes.io/projected/95af2f9c-0950-4d31-adda-b15f1dc94db0-kube-api-access-rwfqj\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:32:18.276197 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.276096 2557 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-offline-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:32:18.276197 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.276109 2557 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/95af2f9c-0950-4d31-adda-b15f1dc94db0-feast-data\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:32:18.276197 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.276121 2557 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/95af2f9c-0950-4d31-adda-b15f1dc94db0-online-tls\") on node \"ip-10-0-136-245.ec2.internal\" DevicePath \"\"" Sep 11 08:32:18.408619 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.408597 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll"] Sep 11 08:32:18.413978 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:18.413958 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-cjzll"] Sep 11 08:32:19.501285 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:19.501253 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" path="/var/lib/kubelet/pods/5d1b50ed-fd5f-4923-9863-10194e9e35c6/volumes" Sep 11 08:32:19.501775 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:19.501761 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" path="/var/lib/kubelet/pods/95af2f9c-0950-4d31-adda-b15f1dc94db0/volumes" Sep 11 08:32:55.389563 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.389526 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9dvm5/must-gather-8rck4"] Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.389877 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="feast-init" Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.389895 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="feast-init" Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.389926 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="online" Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.389934 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="online" Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.389947 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="ui" Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.389954 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="ui" Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.389971 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="feast-init" Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.389978 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="feast-init" Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.389989 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="offline" Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.389997 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="offline" Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.390004 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="online" Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.390012 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="online" Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.390025 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="registry" Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.390033 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="registry" Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.390047 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="feast-apply" Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.390055 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="feast-apply" Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.390064 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="ui" Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.390073 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="ui" Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.390088 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="feast-apply" Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.390097 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="feast-apply" Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.390106 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="offline" Sep 11 08:32:55.390161 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.390116 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="offline" Sep 11 08:32:55.391237 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.390185 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="online" Sep 11 08:32:55.391237 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.390194 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="online" Sep 11 08:32:55.391237 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.390204 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="ui" Sep 11 08:32:55.391237 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.390212 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="ui" Sep 11 08:32:55.391237 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.390222 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d1b50ed-fd5f-4923-9863-10194e9e35c6" containerName="offline" Sep 11 08:32:55.391237 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.390233 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="offline" Sep 11 08:32:55.391237 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.390241 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="95af2f9c-0950-4d31-adda-b15f1dc94db0" containerName="registry" Sep 11 08:32:55.393492 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.393473 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9dvm5/must-gather-8rck4" Sep 11 08:32:55.395803 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.395785 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9dvm5\"/\"default-dockercfg-mt8rk\"" Sep 11 08:32:55.395915 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.395816 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9dvm5\"/\"kube-root-ca.crt\"" Sep 11 08:32:55.396036 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.396021 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9dvm5\"/\"openshift-service-ca.crt\"" Sep 11 08:32:55.402415 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.402395 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9dvm5/must-gather-8rck4"] Sep 11 08:32:55.528211 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.528180 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmlqh\" (UniqueName: \"kubernetes.io/projected/1072c413-33d5-4f13-89c5-a184a085aacd-kube-api-access-vmlqh\") pod \"must-gather-8rck4\" (UID: \"1072c413-33d5-4f13-89c5-a184a085aacd\") " pod="openshift-must-gather-9dvm5/must-gather-8rck4" Sep 11 08:32:55.528337 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.528213 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1072c413-33d5-4f13-89c5-a184a085aacd-must-gather-output\") pod \"must-gather-8rck4\" (UID: \"1072c413-33d5-4f13-89c5-a184a085aacd\") " pod="openshift-must-gather-9dvm5/must-gather-8rck4" Sep 11 08:32:55.628724 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.628694 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmlqh\" (UniqueName: \"kubernetes.io/projected/1072c413-33d5-4f13-89c5-a184a085aacd-kube-api-access-vmlqh\") pod \"must-gather-8rck4\" (UID: \"1072c413-33d5-4f13-89c5-a184a085aacd\") " pod="openshift-must-gather-9dvm5/must-gather-8rck4" Sep 11 08:32:55.628724 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.628730 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1072c413-33d5-4f13-89c5-a184a085aacd-must-gather-output\") pod \"must-gather-8rck4\" (UID: \"1072c413-33d5-4f13-89c5-a184a085aacd\") " pod="openshift-must-gather-9dvm5/must-gather-8rck4" Sep 11 08:32:55.629040 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.629023 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1072c413-33d5-4f13-89c5-a184a085aacd-must-gather-output\") pod \"must-gather-8rck4\" (UID: \"1072c413-33d5-4f13-89c5-a184a085aacd\") " pod="openshift-must-gather-9dvm5/must-gather-8rck4" Sep 11 08:32:55.639731 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.639681 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmlqh\" (UniqueName: \"kubernetes.io/projected/1072c413-33d5-4f13-89c5-a184a085aacd-kube-api-access-vmlqh\") pod \"must-gather-8rck4\" (UID: \"1072c413-33d5-4f13-89c5-a184a085aacd\") " pod="openshift-must-gather-9dvm5/must-gather-8rck4" Sep 11 08:32:55.718146 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.718125 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9dvm5/must-gather-8rck4" Sep 11 08:32:55.989754 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.989608 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9dvm5/must-gather-8rck4"] Sep 11 08:32:55.992501 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:32:55.992472 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1072c413_33d5_4f13_89c5_a184a085aacd.slice/crio-71346af807b1f1d96d3225c1249692d76cdb1971b6ba07ca67385981d63721c5 WatchSource:0}: Error finding container 71346af807b1f1d96d3225c1249692d76cdb1971b6ba07ca67385981d63721c5: Status 404 returned error can't find the container with id 71346af807b1f1d96d3225c1249692d76cdb1971b6ba07ca67385981d63721c5 Sep 11 08:32:55.994505 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:55.994475 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 11 08:32:56.227385 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:56.227360 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9dvm5/must-gather-8rck4" event={"ID":"1072c413-33d5-4f13-89c5-a184a085aacd","Type":"ContainerStarted","Data":"71346af807b1f1d96d3225c1249692d76cdb1971b6ba07ca67385981d63721c5"} Sep 11 08:32:57.234253 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:57.234219 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9dvm5/must-gather-8rck4" event={"ID":"1072c413-33d5-4f13-89c5-a184a085aacd","Type":"ContainerStarted","Data":"52b3c21007729c937c83cb62dcc8c95d17a977fce26921a4dbe71295860b0a72"} Sep 11 08:32:57.234650 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:57.234265 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9dvm5/must-gather-8rck4" event={"ID":"1072c413-33d5-4f13-89c5-a184a085aacd","Type":"ContainerStarted","Data":"b647690f31b02064c550fc8de10adbffa7fec2496bd4aa7380b316d558db14b9"} Sep 11 08:32:57.264784 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:57.264735 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9dvm5/must-gather-8rck4" podStartSLOduration=1.523654976 podStartE2EDuration="2.264719358s" totalImagesPullingTime="741.064382ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:32:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:32:57.263577519 +0000 UTC m=+2014.269295178" watchObservedRunningTime="2026-09-11 08:32:57.264719358 +0000 UTC m=+2014.270437016" Sep 11 08:32:58.437113 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:58.437076 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-f8ptc_98d2f5ae-5780-4110-935d-e3781cc271f7/global-pull-secret-syncer/0.log" Sep 11 08:32:58.684409 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:58.684367 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ddqh8_967e46d4-81f0-49ce-a6ee-d8737cc7ad96/konnectivity-agent/0.log" Sep 11 08:32:58.741080 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:32:58.740996 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-245.ec2.internal_e11886251d79d45e61a827aee7470670/haproxy/0.log" Sep 11 08:33:02.258373 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:02.258284 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-p8lmw_c391e6df-9648-424a-bb7e-2fca75511930/monitoring-plugin/0.log" Sep 11 08:33:02.442183 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:02.442152 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5n7lg_e910d42e-6726-46ca-978a-e687d028af81/node-exporter/0.log" Sep 11 08:33:02.481412 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:02.481374 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5n7lg_e910d42e-6726-46ca-978a-e687d028af81/kube-rbac-proxy/0.log" Sep 11 08:33:02.517392 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:02.517315 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5n7lg_e910d42e-6726-46ca-978a-e687d028af81/init-textfile/0.log" Sep 11 08:33:02.690429 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:02.690403 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-wn6ct_6dcaa449-e314-4d02-ac8f-de897a090057/kube-rbac-proxy-main/0.log" Sep 11 08:33:02.722258 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:02.722226 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-wn6ct_6dcaa449-e314-4d02-ac8f-de897a090057/kube-rbac-proxy-self/0.log" Sep 11 08:33:02.760634 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:02.760603 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-wn6ct_6dcaa449-e314-4d02-ac8f-de897a090057/openshift-state-metrics/0.log" Sep 11 08:33:03.046634 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:03.046597 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-hkdql_68645e9e-5833-4cdc-896d-86ee7f234140/prometheus-operator/0.log" Sep 11 08:33:03.073765 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:03.073732 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-hkdql_68645e9e-5833-4cdc-896d-86ee7f234140/kube-rbac-proxy/0.log" Sep 11 08:33:03.106671 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:03.106650 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-d5jvv_931723ec-ab59-48c7-8d99-881c5144da26/prometheus-operator-admission-webhook/0.log" Sep 11 08:33:04.746809 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:04.746767 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-gczkp_6eafaa84-dc03-4fef-a4c6-ecb80dd2172e/networking-console-plugin/0.log" Sep 11 08:33:05.297447 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:05.297416 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm"] Sep 11 08:33:05.303316 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:05.303286 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" Sep 11 08:33:05.315702 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:05.315678 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm"] Sep 11 08:33:05.410782 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:05.410751 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7ef1fa2-de81-426e-957d-1102f8c50e04-sys\") pod \"perf-node-gather-daemonset-bfstm\" (UID: \"c7ef1fa2-de81-426e-957d-1102f8c50e04\") " pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" Sep 11 08:33:05.410989 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:05.410799 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhmjm\" (UniqueName: \"kubernetes.io/projected/c7ef1fa2-de81-426e-957d-1102f8c50e04-kube-api-access-hhmjm\") pod \"perf-node-gather-daemonset-bfstm\" (UID: \"c7ef1fa2-de81-426e-957d-1102f8c50e04\") " pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" Sep 11 08:33:05.410989 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:05.410883 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c7ef1fa2-de81-426e-957d-1102f8c50e04-proc\") pod \"perf-node-gather-daemonset-bfstm\" (UID: \"c7ef1fa2-de81-426e-957d-1102f8c50e04\") " pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" Sep 11 08:33:05.410989 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:05.410927 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c7ef1fa2-de81-426e-957d-1102f8c50e04-podres\") pod \"perf-node-gather-daemonset-bfstm\" (UID: \"c7ef1fa2-de81-426e-957d-1102f8c50e04\") " pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" Sep 11 08:33:05.411151 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:05.410996 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c7ef1fa2-de81-426e-957d-1102f8c50e04-lib-modules\") pod \"perf-node-gather-daemonset-bfstm\" (UID: \"c7ef1fa2-de81-426e-957d-1102f8c50e04\") " pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" Sep 11 08:33:05.512400 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:05.512343 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhmjm\" (UniqueName: \"kubernetes.io/projected/c7ef1fa2-de81-426e-957d-1102f8c50e04-kube-api-access-hhmjm\") pod \"perf-node-gather-daemonset-bfstm\" (UID: \"c7ef1fa2-de81-426e-957d-1102f8c50e04\") " pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" Sep 11 08:33:05.512597 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:05.512461 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c7ef1fa2-de81-426e-957d-1102f8c50e04-proc\") pod \"perf-node-gather-daemonset-bfstm\" (UID: \"c7ef1fa2-de81-426e-957d-1102f8c50e04\") " pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" Sep 11 08:33:05.512597 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:05.512492 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c7ef1fa2-de81-426e-957d-1102f8c50e04-podres\") pod \"perf-node-gather-daemonset-bfstm\" (UID: \"c7ef1fa2-de81-426e-957d-1102f8c50e04\") " pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" Sep 11 08:33:05.512597 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:05.512569 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c7ef1fa2-de81-426e-957d-1102f8c50e04-lib-modules\") pod \"perf-node-gather-daemonset-bfstm\" (UID: \"c7ef1fa2-de81-426e-957d-1102f8c50e04\") " pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" Sep 11 08:33:05.512779 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:05.512632 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7ef1fa2-de81-426e-957d-1102f8c50e04-sys\") pod \"perf-node-gather-daemonset-bfstm\" (UID: \"c7ef1fa2-de81-426e-957d-1102f8c50e04\") " pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" Sep 11 08:33:05.512779 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:05.512737 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7ef1fa2-de81-426e-957d-1102f8c50e04-sys\") pod \"perf-node-gather-daemonset-bfstm\" (UID: \"c7ef1fa2-de81-426e-957d-1102f8c50e04\") " pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" Sep 11 08:33:05.513076 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:05.513048 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c7ef1fa2-de81-426e-957d-1102f8c50e04-podres\") pod \"perf-node-gather-daemonset-bfstm\" (UID: \"c7ef1fa2-de81-426e-957d-1102f8c50e04\") " pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" Sep 11 08:33:05.513266 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:05.513224 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c7ef1fa2-de81-426e-957d-1102f8c50e04-proc\") pod \"perf-node-gather-daemonset-bfstm\" (UID: \"c7ef1fa2-de81-426e-957d-1102f8c50e04\") " pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" Sep 11 08:33:05.513385 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:05.513356 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c7ef1fa2-de81-426e-957d-1102f8c50e04-lib-modules\") pod \"perf-node-gather-daemonset-bfstm\" (UID: \"c7ef1fa2-de81-426e-957d-1102f8c50e04\") " pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" Sep 11 08:33:05.526621 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:05.526591 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhmjm\" (UniqueName: \"kubernetes.io/projected/c7ef1fa2-de81-426e-957d-1102f8c50e04-kube-api-access-hhmjm\") pod \"perf-node-gather-daemonset-bfstm\" (UID: \"c7ef1fa2-de81-426e-957d-1102f8c50e04\") " pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" Sep 11 08:33:05.619060 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:05.618980 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" Sep 11 08:33:05.706427 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:05.706400 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7bb4f78d5c-gbt9v_34628316-b370-492d-a638-797607438036/console/0.log" Sep 11 08:33:05.900265 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:05.900235 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm"] Sep 11 08:33:05.903146 ip-10-0-136-245 kubenswrapper[2557]: W0911 08:33:05.903114 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc7ef1fa2_de81_426e_957d_1102f8c50e04.slice/crio-1eb8c7c2318d7c2a312dbdffae2fe4efd11baef99f320c151731d93ca690593a WatchSource:0}: Error finding container 1eb8c7c2318d7c2a312dbdffae2fe4efd11baef99f320c151731d93ca690593a: Status 404 returned error can't find the container with id 1eb8c7c2318d7c2a312dbdffae2fe4efd11baef99f320c151731d93ca690593a Sep 11 08:33:06.282965 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:06.282862 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" event={"ID":"c7ef1fa2-de81-426e-957d-1102f8c50e04","Type":"ContainerStarted","Data":"f39c1fd11e741b2f74ad166b1cbc4476467c8ed32f7e791510d1acae3e09e956"} Sep 11 08:33:06.282965 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:06.282958 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" Sep 11 08:33:06.283139 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:06.282984 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" event={"ID":"c7ef1fa2-de81-426e-957d-1102f8c50e04","Type":"ContainerStarted","Data":"1eb8c7c2318d7c2a312dbdffae2fe4efd11baef99f320c151731d93ca690593a"} Sep 11 08:33:06.303730 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:06.303685 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" podStartSLOduration=1.303665324 podStartE2EDuration="1.303665324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 08:33:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:33:06.302784087 +0000 UTC m=+2023.308501758" watchObservedRunningTime="2026-09-11 08:33:06.303665324 +0000 UTC m=+2023.309382982" Sep 11 08:33:07.295592 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:07.295558 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xl9mm_022e9b3f-8f4d-40c1-989a-140d1d00f5d0/dns/0.log" Sep 11 08:33:07.319003 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:07.318981 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xl9mm_022e9b3f-8f4d-40c1-989a-140d1d00f5d0/kube-rbac-proxy/0.log" Sep 11 08:33:07.402881 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:07.402859 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vqjrt_c61a8ab5-9ed9-4f3d-9e32-935de3901e01/dns-node-resolver/0.log" Sep 11 08:33:07.931019 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:07.930976 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6skrk_88bb78bf-db17-415d-b3e8-b88236c84350/node-ca/0.log" Sep 11 08:33:08.807884 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:08.807848 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5f8bf9f44-82dnf_47a69b8b-6b79-426e-bf18-963fe1046241/router/0.log" Sep 11 08:33:09.282852 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:09.282824 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qwg5f_8fc4eb55-74fd-4d56-bc7f-6812a50165a4/serve-healthcheck-canary/0.log" Sep 11 08:33:09.802363 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:09.802330 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fssdv_0aad543f-d277-4552-8909-de30a3820f5d/kube-rbac-proxy/0.log" Sep 11 08:33:09.831121 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:09.831096 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fssdv_0aad543f-d277-4552-8909-de30a3820f5d/exporter/0.log" Sep 11 08:33:09.865189 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:09.865158 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fssdv_0aad543f-d277-4552-8909-de30a3820f5d/extractor/0.log" Sep 11 08:33:12.296923 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:12.296881 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9dvm5/perf-node-gather-daemonset-bfstm" Sep 11 08:33:16.764726 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:16.764645 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-797fj_06a3e034-7768-4e68-8ff4-4678f3cd69e7/kube-multus/0.log" Sep 11 08:33:16.794946 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:16.794920 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8hfsw_2cb363cc-5b2b-47df-a9a4-29c33b6a67f5/kube-multus-additional-cni-plugins/0.log" Sep 11 08:33:16.828947 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:16.828926 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8hfsw_2cb363cc-5b2b-47df-a9a4-29c33b6a67f5/egress-router-binary-copy/0.log" Sep 11 08:33:16.855480 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:16.855463 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8hfsw_2cb363cc-5b2b-47df-a9a4-29c33b6a67f5/cni-plugins/0.log" Sep 11 08:33:16.887706 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:16.887683 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8hfsw_2cb363cc-5b2b-47df-a9a4-29c33b6a67f5/bond-cni-plugin/0.log" Sep 11 08:33:16.919822 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:16.919802 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8hfsw_2cb363cc-5b2b-47df-a9a4-29c33b6a67f5/routeoverride-cni/0.log" Sep 11 08:33:16.956615 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:16.956591 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8hfsw_2cb363cc-5b2b-47df-a9a4-29c33b6a67f5/whereabouts-cni-bincopy/0.log" Sep 11 08:33:17.020158 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:17.020099 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8hfsw_2cb363cc-5b2b-47df-a9a4-29c33b6a67f5/whereabouts-cni/0.log" Sep 11 08:33:17.869067 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:17.869040 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mkcn8_c5234baf-5b8f-46ff-9fa3-f217c0087ded/network-metrics-daemon/0.log" Sep 11 08:33:17.907404 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:17.907380 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mkcn8_c5234baf-5b8f-46ff-9fa3-f217c0087ded/kube-rbac-proxy/0.log" Sep 11 08:33:19.068222 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:19.068193 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j5fx_0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08/ovn-controller/0.log" Sep 11 08:33:19.092113 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:19.092087 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j5fx_0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08/ovn-acl-logging/0.log" Sep 11 08:33:19.123940 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:19.123913 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j5fx_0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08/kube-rbac-proxy-node/0.log" Sep 11 08:33:19.157818 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:19.157799 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j5fx_0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08/kube-rbac-proxy-ovn-metrics/0.log" Sep 11 08:33:19.187884 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:19.187866 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j5fx_0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08/northd/0.log" Sep 11 08:33:19.609565 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:19.609536 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j5fx_0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08/nbdb/0.log" Sep 11 08:33:19.633952 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:19.633931 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j5fx_0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08/sbdb/0.log" Sep 11 08:33:19.692033 ip-10-0-136-245 kubenswrapper[2557]: I0911 08:33:19.691997 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9j5fx_0c06f0f2-b5b9-40c2-bd00-438a9e0c0f08/ovnkube-controller/0.log"