Sep 17 07:27:59.718084 ip-10-0-129-217 systemd[1]: Starting Kubernetes Kubelet... Sep 17 07:28:00.115564 ip-10-0-129-217 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 07:28:00.115564 ip-10-0-129-217 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 17 07:28:00.115564 ip-10-0-129-217 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 07:28:00.115564 ip-10-0-129-217 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 17 07:28:00.116245 ip-10-0-129-217 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 07:28:00.116245 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.115655 2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 17 07:28:00.120616 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120578 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:28:00.120616 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120604 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:28:00.120616 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120609 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:28:00.120616 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120619 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:28:00.120826 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120761 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:28:00.120826 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120770 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:28:00.120826 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120774 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:28:00.120826 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120777 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:28:00.120826 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120780 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:28:00.120826 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120783 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:28:00.120826 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120786 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:28:00.120826 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120789 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:28:00.120826 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120792 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:28:00.120826 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120795 2579 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:28:00.120826 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120798 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:28:00.120826 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120801 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:28:00.120826 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120803 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:28:00.120826 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120806 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:28:00.120826 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120809 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:28:00.120826 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120812 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:28:00.120826 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120814 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:28:00.120826 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120817 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:28:00.120826 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120819 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:28:00.120826 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120826 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:28:00.121296 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120829 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:28:00.121296 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120832 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:28:00.121296 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120835 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:28:00.121296 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120838 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:28:00.121296 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120841 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:28:00.121296 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120844 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:28:00.121296 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120846 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:28:00.121296 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120849 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:28:00.121296 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120852 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:28:00.121296 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120854 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:28:00.121296 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120857 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:28:00.121296 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120860 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:28:00.121296 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120863 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:28:00.121296 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120865 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:28:00.121296 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120868 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:28:00.121296 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120870 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:28:00.121296 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120875 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:28:00.121296 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120878 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:28:00.121296 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120880 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:28:00.121829 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120884 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:28:00.121829 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120888 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:28:00.121829 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120890 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:28:00.121829 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120893 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:28:00.121829 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120896 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:28:00.121829 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120898 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:28:00.121829 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120900 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:28:00.121829 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120904 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:28:00.121829 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120907 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:28:00.121829 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120909 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:28:00.121829 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120912 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:28:00.121829 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120914 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:28:00.121829 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120917 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:28:00.121829 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120920 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:28:00.121829 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120923 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:28:00.121829 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120926 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:28:00.121829 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120928 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:28:00.121829 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120931 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:28:00.121829 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120934 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:28:00.121829 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120937 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:28:00.122312 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120940 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:28:00.122312 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120944 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:28:00.122312 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120948 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:28:00.122312 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120951 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:28:00.122312 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120954 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:28:00.122312 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120959 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:28:00.122312 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120962 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:28:00.122312 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120965 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:28:00.122312 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120967 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:28:00.122312 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120971 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:28:00.122312 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120973 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:28:00.122312 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120976 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:28:00.122312 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120979 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:28:00.122312 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120982 2579 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:28:00.122312 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120984 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:28:00.122312 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120987 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:28:00.122312 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120989 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:28:00.122312 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120992 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:28:00.122312 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120995 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:28:00.122780 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.120999 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:28:00.122780 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121004 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:28:00.122780 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121007 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:28:00.122780 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121010 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:28:00.122780 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121013 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:28:00.122780 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121015 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:28:00.122780 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121020 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:28:00.122780 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121023 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:28:00.122780 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121026 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:28:00.122780 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121029 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:28:00.122780 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121032 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:28:00.122780 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121034 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:28:00.122780 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121037 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:28:00.122780 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121040 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:28:00.122780 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121043 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:28:00.122780 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121045 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:28:00.122780 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121048 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:28:00.122780 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121786 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:28:00.122780 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121793 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:28:00.122780 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121796 2579 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:28:00.123242 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121799 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:28:00.123242 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121802 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:28:00.123242 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121805 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:28:00.123242 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121807 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:28:00.123242 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121810 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:28:00.123242 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121813 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:28:00.123242 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121816 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:28:00.123242 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121819 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:28:00.123242 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121821 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:28:00.123242 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121823 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:28:00.123242 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121826 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:28:00.123242 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121829 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:28:00.123242 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121832 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:28:00.123242 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121834 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:28:00.123242 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121840 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:28:00.123242 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121844 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:28:00.123242 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121848 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:28:00.123242 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121851 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:28:00.123242 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121854 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:28:00.123735 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121857 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:28:00.123735 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121859 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:28:00.123735 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121862 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:28:00.123735 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121865 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:28:00.123735 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121867 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:28:00.123735 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121870 2579 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:28:00.123735 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121872 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:28:00.123735 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121875 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:28:00.123735 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121878 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:28:00.123735 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121881 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:28:00.123735 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121883 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:28:00.123735 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121886 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:28:00.123735 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121888 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:28:00.123735 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121891 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:28:00.123735 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121894 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:28:00.123735 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121896 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:28:00.123735 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121899 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:28:00.123735 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121901 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:28:00.123735 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121904 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:28:00.124200 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121906 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:28:00.124200 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121909 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:28:00.124200 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121911 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:28:00.124200 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121914 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:28:00.124200 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121917 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:28:00.124200 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121919 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:28:00.124200 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121922 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:28:00.124200 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121924 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:28:00.124200 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121927 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:28:00.124200 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121931 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:28:00.124200 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121935 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:28:00.124200 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121939 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:28:00.124200 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121942 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:28:00.124200 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121944 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:28:00.124200 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121947 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:28:00.124200 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121950 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:28:00.124200 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121952 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:28:00.124200 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121955 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:28:00.124200 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121957 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:28:00.124200 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121960 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:28:00.124712 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121964 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:28:00.124712 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121967 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:28:00.124712 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121970 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:28:00.124712 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121973 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:28:00.124712 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121976 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:28:00.124712 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121979 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:28:00.124712 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121982 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:28:00.124712 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121985 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:28:00.124712 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121988 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:28:00.124712 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121990 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:28:00.124712 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121993 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:28:00.124712 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121996 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:28:00.124712 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.121999 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:28:00.124712 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122001 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:28:00.124712 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122003 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:28:00.124712 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122006 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:28:00.124712 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122009 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:28:00.124712 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122011 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:28:00.124712 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122014 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:28:00.125184 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122016 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:28:00.125184 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122018 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:28:00.125184 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122021 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:28:00.125184 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122024 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:28:00.125184 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122027 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:28:00.125184 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122030 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:28:00.125184 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122034 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:28:00.125184 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122037 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:28:00.125184 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122040 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:28:00.125184 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122043 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:28:00.125184 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122045 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:28:00.125184 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122048 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:28:00.125184 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122050 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:28:00.125184 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122053 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:28:00.125184 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122056 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:28:00.125184 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122058 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:28:00.125184 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122061 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:28:00.125184 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122063 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:28:00.125184 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122066 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:28:00.125184 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122463 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:28:00.125707 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122467 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:28:00.125707 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122470 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:28:00.125707 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122473 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:28:00.125707 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122476 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:28:00.125707 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122478 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:28:00.125707 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122481 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:28:00.125707 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122484 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:28:00.125707 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122487 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:28:00.125707 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122489 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:28:00.125707 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122492 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:28:00.125707 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122495 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:28:00.125707 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122497 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:28:00.125707 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122500 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:28:00.125707 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122502 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:28:00.125707 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122505 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:28:00.125707 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122508 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:28:00.125707 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122512 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:28:00.125707 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122515 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:28:00.125707 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122518 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:28:00.126205 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122521 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:28:00.126205 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122523 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:28:00.126205 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122526 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:28:00.126205 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122529 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:28:00.126205 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122531 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:28:00.126205 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122534 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:28:00.126205 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122536 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:28:00.126205 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122539 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:28:00.126205 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122542 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:28:00.126205 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122544 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:28:00.126205 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122547 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:28:00.126205 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122549 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:28:00.126205 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122551 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:28:00.126205 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122554 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:28:00.126205 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122556 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:28:00.126205 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122559 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:28:00.126205 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122562 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:28:00.126205 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122565 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:28:00.126205 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122567 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:28:00.126205 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122570 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:28:00.126711 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122572 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:28:00.126711 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122575 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:28:00.126711 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122577 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:28:00.126711 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122579 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:28:00.126711 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122582 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:28:00.126711 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122584 2579 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:28:00.126711 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122587 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:28:00.126711 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122590 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:28:00.126711 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122593 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:28:00.126711 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122596 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:28:00.126711 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122599 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:28:00.126711 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122602 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:28:00.126711 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122604 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:28:00.126711 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122607 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:28:00.126711 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122609 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:28:00.126711 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122612 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:28:00.126711 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122615 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:28:00.126711 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122617 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:28:00.126711 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122634 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:28:00.126711 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122638 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:28:00.127243 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122641 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:28:00.127243 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122643 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:28:00.127243 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122646 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:28:00.127243 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122649 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:28:00.127243 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122651 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:28:00.127243 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122654 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:28:00.127243 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122656 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:28:00.127243 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122659 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:28:00.127243 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122661 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:28:00.127243 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122666 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:28:00.127243 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122669 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:28:00.127243 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122672 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:28:00.127243 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122676 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:28:00.127243 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122679 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:28:00.127243 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122682 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:28:00.127243 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122684 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:28:00.127243 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122688 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:28:00.127243 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122691 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:28:00.127243 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122693 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:28:00.127727 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122697 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:28:00.127727 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122700 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:28:00.127727 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122704 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:28:00.127727 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122708 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:28:00.127727 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122710 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:28:00.127727 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122713 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:28:00.127727 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122715 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:28:00.127727 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122718 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:28:00.127727 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122722 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:28:00.127727 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122725 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:28:00.127727 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122728 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:28:00.127727 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122731 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:28:00.127727 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122734 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:28:00.127727 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122736 2579 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:28:00.127727 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122739 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:28:00.127727 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122742 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:28:00.127727 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122744 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:28:00.127727 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122747 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:28:00.127727 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122749 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.122752 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122825 2579 flags.go:64] FLAG: --address="0.0.0.0" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122873 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122881 2579 flags.go:64] FLAG: --anonymous-auth="true" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122886 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122890 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122894 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122898 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122903 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122906 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122909 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122915 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122919 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122922 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122926 2579 flags.go:64] FLAG: --cgroup-root="" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122929 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122932 2579 flags.go:64] FLAG: --client-ca-file="" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122935 2579 flags.go:64] FLAG: --cloud-provider="external" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122938 2579 flags.go:64] FLAG: --cluster-dns="[]" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122943 2579 flags.go:64] FLAG: --cluster-domain="" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122945 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122949 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122952 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 17 07:28:00.128227 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122955 2579 flags.go:64] FLAG: --container-log-max-files="5" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122959 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122962 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122965 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122969 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122972 2579 flags.go:64] FLAG: --contention-profiling="false" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122975 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122979 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122982 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122985 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122989 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122992 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122995 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.122998 2579 flags.go:64] FLAG: --enable-load-reader="false" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123001 2579 flags.go:64] FLAG: --enable-server="true" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123004 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123009 2579 flags.go:64] FLAG: --event-burst="100" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123012 2579 flags.go:64] FLAG: --event-qps="50" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123015 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123018 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123023 2579 flags.go:64] FLAG: --eviction-hard="" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123027 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123030 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123035 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123038 2579 flags.go:64] FLAG: --eviction-soft="" Sep 17 07:28:00.128838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123041 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123044 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123048 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123051 2579 flags.go:64] FLAG: --experimental-mounter-path="" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123054 2579 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123057 2579 flags.go:64] FLAG: --fail-swap-on="true" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123059 2579 flags.go:64] FLAG: --feature-gates="" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123063 2579 flags.go:64] FLAG: --file-check-frequency="20s" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123066 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123070 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123073 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123076 2579 flags.go:64] FLAG: --healthz-port="10248" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123079 2579 flags.go:64] FLAG: --help="false" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123082 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-129-217.ec2.internal" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123085 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123088 2579 flags.go:64] FLAG: --http-check-frequency="20s" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123091 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123094 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123098 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123101 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123104 2579 flags.go:64] FLAG: --image-service-endpoint="" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123107 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123110 2579 flags.go:64] FLAG: --kube-api-burst="100" Sep 17 07:28:00.129429 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123112 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123116 2579 flags.go:64] FLAG: --kube-api-qps="50" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123119 2579 flags.go:64] FLAG: --kube-reserved="" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123122 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123126 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123129 2579 flags.go:64] FLAG: --kubelet-cgroups="" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123132 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123136 2579 flags.go:64] FLAG: --lock-file="" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123140 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123143 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123145 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123151 2579 flags.go:64] FLAG: --log-json-split-stream="false" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123156 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123159 2579 flags.go:64] FLAG: --log-text-split-stream="false" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123162 2579 flags.go:64] FLAG: --logging-format="text" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123165 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123168 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123171 2579 flags.go:64] FLAG: --manifest-url="" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123174 2579 flags.go:64] FLAG: --manifest-url-header="" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123179 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123182 2579 flags.go:64] FLAG: --max-open-files="1000000" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123186 2579 flags.go:64] FLAG: --max-pods="110" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123189 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123192 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123195 2579 flags.go:64] FLAG: --memory-manager-policy="None" Sep 17 07:28:00.130047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123198 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123201 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123204 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123207 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123215 2579 flags.go:64] FLAG: --node-status-max-images="50" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123218 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123221 2579 flags.go:64] FLAG: --oom-score-adj="-999" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123224 2579 flags.go:64] FLAG: --pod-cidr="" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123227 2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123233 2579 flags.go:64] FLAG: --pod-manifest-path="" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123236 2579 flags.go:64] FLAG: --pod-max-pids="-1" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123239 2579 flags.go:64] FLAG: --pods-per-core="0" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123243 2579 flags.go:64] FLAG: --port="10250" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123246 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123251 2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d26d28ac1058b868" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123255 2579 flags.go:64] FLAG: --qos-reserved="" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123258 2579 flags.go:64] FLAG: --read-only-port="10255" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123261 2579 flags.go:64] FLAG: --register-node="true" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123264 2579 flags.go:64] FLAG: --register-with-taints="" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123267 2579 flags.go:64] FLAG: --registry-burst="10" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123271 2579 flags.go:64] FLAG: --registry-qps="5" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123274 2579 flags.go:64] FLAG: --reserved-cpus="" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123277 2579 flags.go:64] FLAG: --reserved-memory="" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123280 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123284 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 17 07:28:00.131041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123286 2579 flags.go:64] FLAG: --rotate-certificates="false" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123289 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123292 2579 flags.go:64] FLAG: --runonce="false" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123295 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123303 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123306 2579 flags.go:64] FLAG: --seccomp-default="false" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123309 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123311 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123314 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123317 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123321 2579 flags.go:64] FLAG: --storage-driver-password="root" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123324 2579 flags.go:64] FLAG: --storage-driver-secure="false" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123327 2579 flags.go:64] FLAG: --storage-driver-table="stats" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123329 2579 flags.go:64] FLAG: --storage-driver-user="root" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123332 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123335 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123339 2579 flags.go:64] FLAG: --system-cgroups="" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123341 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123347 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123351 2579 flags.go:64] FLAG: --tls-cert-file="" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123354 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123360 2579 flags.go:64] FLAG: --tls-min-version="" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123362 2579 flags.go:64] FLAG: --tls-private-key-file="" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123365 2579 flags.go:64] FLAG: --topology-manager-policy="none" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123368 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 17 07:28:00.132119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123371 2579 flags.go:64] FLAG: --topology-manager-scope="container" Sep 17 07:28:00.133016 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123374 2579 flags.go:64] FLAG: --v="2" Sep 17 07:28:00.133016 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123378 2579 flags.go:64] FLAG: --version="false" Sep 17 07:28:00.133016 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123383 2579 flags.go:64] FLAG: --vmodule="" Sep 17 07:28:00.133016 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123387 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 17 07:28:00.133016 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.123390 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 17 07:28:00.133016 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123510 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:28:00.133016 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123513 2579 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:28:00.133016 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123516 2579 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:28:00.133016 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123519 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:28:00.133016 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123523 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:28:00.133016 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123525 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:28:00.133016 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123528 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:28:00.133016 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123531 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:28:00.133016 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123533 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:28:00.133016 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123536 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:28:00.133016 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123539 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:28:00.133016 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123541 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:28:00.133016 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123544 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:28:00.133016 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123546 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:28:00.133016 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123549 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:28:00.133016 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123552 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:28:00.133726 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123554 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:28:00.133726 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123557 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:28:00.133726 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123560 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:28:00.133726 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123562 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:28:00.133726 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123566 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:28:00.133726 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123569 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:28:00.133726 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123573 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:28:00.133726 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123576 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:28:00.133726 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123578 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:28:00.133726 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123581 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:28:00.133726 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123584 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:28:00.133726 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123586 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:28:00.133726 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123589 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:28:00.133726 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123591 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:28:00.133726 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123594 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:28:00.133726 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123597 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:28:00.133726 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123600 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:28:00.133726 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123602 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:28:00.133726 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123605 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:28:00.134568 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123609 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:28:00.134568 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123612 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:28:00.134568 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123615 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:28:00.134568 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123618 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:28:00.134568 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123635 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:28:00.134568 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123639 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:28:00.134568 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123643 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:28:00.134568 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123647 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:28:00.134568 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123650 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:28:00.134568 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123653 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:28:00.134568 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123656 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:28:00.134568 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123659 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:28:00.134568 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123662 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:28:00.134568 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123665 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:28:00.134568 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123667 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:28:00.134568 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123670 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:28:00.134568 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123674 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:28:00.134568 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123678 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:28:00.134568 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123681 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:28:00.135201 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123685 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:28:00.135201 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123687 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:28:00.135201 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123690 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:28:00.135201 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123693 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:28:00.135201 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123696 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:28:00.135201 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123699 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:28:00.135201 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123702 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:28:00.135201 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123704 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:28:00.135201 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123707 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:28:00.135201 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123711 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:28:00.135201 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123714 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:28:00.135201 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123717 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:28:00.135201 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123720 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:28:00.135201 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123722 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:28:00.135201 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123725 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:28:00.135201 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123728 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:28:00.135201 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123731 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:28:00.135201 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123733 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:28:00.135201 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123736 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:28:00.135201 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123739 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:28:00.135911 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123742 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:28:00.135911 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123746 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:28:00.135911 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123749 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:28:00.135911 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123752 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:28:00.135911 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123755 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:28:00.135911 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123757 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:28:00.135911 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123760 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:28:00.135911 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123763 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:28:00.135911 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123766 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:28:00.135911 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123769 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:28:00.135911 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123773 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:28:00.135911 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123775 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:28:00.135911 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123780 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:28:00.135911 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123783 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:28:00.135911 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123785 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:28:00.135911 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123788 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:28:00.135911 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123790 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:28:00.135911 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123793 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:28:00.135911 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123796 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:28:00.135911 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123798 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:28:00.136775 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123801 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:28:00.136775 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123803 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:28:00.136775 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123806 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:28:00.136775 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123809 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:28:00.136775 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.123812 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:28:00.136775 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.124363 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 07:28:00.136775 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.132567 2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 17 07:28:00.136775 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.132588 2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 17 07:28:00.136775 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.133354 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 17 07:28:00.136775 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.133369 2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 17 07:28:00.136775 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133458 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:28:00.136775 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133466 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:28:00.136775 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133470 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:28:00.136775 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133475 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:28:00.136775 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133480 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:28:00.137257 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133484 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:28:00.137257 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133489 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:28:00.137257 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133493 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:28:00.137257 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133498 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:28:00.137257 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133502 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:28:00.137257 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133507 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:28:00.137257 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133511 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:28:00.137257 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133516 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:28:00.137257 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133520 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:28:00.137257 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133525 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:28:00.137257 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133529 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:28:00.137257 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133533 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:28:00.137257 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133537 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:28:00.137257 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133541 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:28:00.137257 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133546 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:28:00.137257 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133550 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:28:00.137257 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133555 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:28:00.137257 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133559 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:28:00.137257 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133563 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:28:00.137257 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133569 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:28:00.137950 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133577 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:28:00.137950 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133583 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:28:00.137950 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133587 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:28:00.137950 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133592 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:28:00.137950 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133597 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:28:00.137950 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133601 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:28:00.137950 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133606 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:28:00.137950 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133610 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:28:00.137950 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133614 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:28:00.137950 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133618 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:28:00.137950 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133637 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:28:00.137950 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133642 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:28:00.137950 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133646 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:28:00.137950 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133650 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:28:00.137950 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133654 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:28:00.137950 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133661 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:28:00.137950 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133667 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:28:00.137950 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133672 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:28:00.137950 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133676 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:28:00.138445 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133681 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:28:00.138445 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133685 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:28:00.138445 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133690 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:28:00.138445 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133694 2579 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:28:00.138445 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133698 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:28:00.138445 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133703 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:28:00.138445 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133707 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:28:00.138445 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133711 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:28:00.138445 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133716 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:28:00.138445 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133720 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:28:00.138445 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133725 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:28:00.138445 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133729 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:28:00.138445 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133733 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:28:00.138445 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133739 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:28:00.138445 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133743 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:28:00.138445 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133748 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:28:00.138445 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133752 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:28:00.138445 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133756 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:28:00.138445 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133761 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:28:00.138445 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133765 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:28:00.139011 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133770 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:28:00.139011 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133774 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:28:00.139011 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133778 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:28:00.139011 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133782 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:28:00.139011 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133787 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:28:00.139011 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133791 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:28:00.139011 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133795 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:28:00.139011 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133799 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:28:00.139011 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133804 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:28:00.139011 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133808 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:28:00.139011 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133813 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:28:00.139011 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133817 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:28:00.139011 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133821 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:28:00.139011 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133825 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:28:00.139011 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133830 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:28:00.139011 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133834 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:28:00.139011 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133838 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:28:00.139011 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133842 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:28:00.139011 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133846 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:28:00.139011 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133850 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:28:00.139566 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133854 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:28:00.139566 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133858 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:28:00.139566 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133863 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:28:00.139566 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133870 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:28:00.139566 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133875 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:28:00.139566 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133880 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:28:00.139566 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133886 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:28:00.139566 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133892 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:28:00.139566 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133896 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:28:00.139566 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133901 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:28:00.139566 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133906 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:28:00.139566 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133911 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:28:00.139566 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133915 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:28:00.139566 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133919 2579 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:28:00.139566 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.133923 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:28:00.139959 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.133932 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 07:28:00.139959 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134121 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:28:00.139959 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134128 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:28:00.139959 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134133 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:28:00.139959 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134138 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:28:00.139959 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134143 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:28:00.139959 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134147 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:28:00.139959 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134151 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:28:00.139959 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134156 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:28:00.139959 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134160 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:28:00.139959 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134164 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:28:00.139959 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134168 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:28:00.139959 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134172 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:28:00.139959 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134176 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:28:00.139959 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134181 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:28:00.140386 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134186 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:28:00.140386 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134190 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:28:00.140386 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134194 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:28:00.140386 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134198 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:28:00.140386 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134203 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:28:00.140386 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134208 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:28:00.140386 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134214 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:28:00.140386 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134219 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:28:00.140386 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134224 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:28:00.140386 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134235 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:28:00.140386 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134240 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:28:00.140386 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134245 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:28:00.140386 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134250 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:28:00.140386 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134255 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:28:00.140386 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134259 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:28:00.140386 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134263 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:28:00.140386 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134268 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:28:00.140386 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134272 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:28:00.140386 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134276 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:28:00.141074 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134280 2579 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:28:00.141074 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134285 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:28:00.141074 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134289 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:28:00.141074 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134294 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:28:00.141074 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134298 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:28:00.141074 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134302 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:28:00.141074 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134306 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:28:00.141074 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134310 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:28:00.141074 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134315 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:28:00.141074 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134320 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:28:00.141074 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134324 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:28:00.141074 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134328 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:28:00.141074 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134333 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:28:00.141074 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134337 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:28:00.141074 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134341 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:28:00.141074 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134345 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:28:00.141074 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134350 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:28:00.141074 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134354 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:28:00.141074 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134358 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:28:00.141074 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134363 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:28:00.141847 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134367 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:28:00.141847 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134371 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:28:00.141847 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134375 2579 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:28:00.141847 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134380 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:28:00.141847 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134384 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:28:00.141847 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134389 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:28:00.141847 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134393 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:28:00.141847 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134397 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:28:00.141847 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134402 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:28:00.141847 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134406 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:28:00.141847 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134410 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:28:00.141847 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134416 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:28:00.141847 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134421 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:28:00.141847 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134426 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:28:00.141847 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134431 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:28:00.141847 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134435 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:28:00.141847 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134441 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:28:00.141847 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134446 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:28:00.141847 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134451 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:28:00.142325 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134455 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:28:00.142325 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134459 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:28:00.142325 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134464 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:28:00.142325 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134469 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:28:00.142325 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134473 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:28:00.142325 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134478 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:28:00.142325 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134483 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:28:00.142325 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134487 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:28:00.142325 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134491 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:28:00.142325 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134496 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:28:00.142325 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134501 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:28:00.142325 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134505 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:28:00.142325 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134509 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:28:00.142325 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134513 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:28:00.142325 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134517 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:28:00.142325 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134522 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:28:00.142325 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134525 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:28:00.142325 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134530 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:28:00.142325 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134535 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:28:00.142325 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134539 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:28:00.142969 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134543 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:28:00.142969 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134547 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:28:00.142969 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134551 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:28:00.142969 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134555 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:28:00.142969 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134560 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:28:00.142969 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134564 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:28:00.142969 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:00.134568 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:28:00.142969 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.134576 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 07:28:00.142969 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.134773 2579 server.go:962] "Client rotation is on, will bootstrap in background" Sep 17 07:28:00.142969 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.137571 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 17 07:28:00.142969 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.138667 2579 server.go:1019] "Starting client certificate rotation" Sep 17 07:28:00.142969 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.138774 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 07:28:00.142969 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.139659 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 07:28:00.143327 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.141244 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 07:28:00.145634 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.145594 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 07:28:00.156538 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.156516 2579 log.go:25] "Validated CRI v1 runtime API" Sep 17 07:28:00.160910 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.160893 2579 log.go:25] "Validated CRI v1 image API" Sep 17 07:28:00.161779 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.161757 2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 17 07:28:00.164715 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.164691 2579 fs.go:135] Filesystem UUIDs: map[470deaa8-a4f6-45ea-8c4d-8e1b49863a8c:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9f90e6f8-b554-4e5d-ae56-686eaf5c4d71:/dev/nvme0n1p4] Sep 17 07:28:00.164835 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.164714 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 17 07:28:00.170041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.169911 2579 manager.go:216] Machine: {Timestamp:2026-09-17 07:28:00.168545856 +0000 UTC m=+0.327892081 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100529 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec230b51837c2eb0cc1f3bf1c12af55d SystemUUID:ec230b51-837c-2eb0-cc1f-3bf1c12af55d BootID:45c790c4-2416-4bcd-a305-d83a2663217b Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:fe:c0:7c:0b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:fe:c0:7c:0b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:42:ac:28:9b:08:60 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 17 07:28:00.170041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.170031 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 17 07:28:00.170220 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.170153 2579 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 17 07:28:00.173265 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.173236 2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 17 07:28:00.173428 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.173269 2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-217.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 17 07:28:00.174571 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.174559 2579 topology_manager.go:138] "Creating topology manager with none policy" Sep 17 07:28:00.174612 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.174572 2579 container_manager_linux.go:309] "Creating device plugin manager" Sep 17 07:28:00.174612 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.174580 2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 07:28:00.175282 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.175271 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 07:28:00.176197 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.176187 2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 17 07:28:00.180681 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.180664 2579 state_mem.go:36] "Initialized new in-memory state store" Sep 17 07:28:00.180830 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.180818 2579 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 17 07:28:00.180864 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.180845 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 07:28:00.180920 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.180854 2579 kubelet.go:486] "Attempting to sync node with API server" Sep 17 07:28:00.180967 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.180937 2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 17 07:28:00.180967 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.180959 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 17 07:28:00.181036 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.180970 2579 kubelet.go:398] "Adding apiserver pod source" Sep 17 07:28:00.181036 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.180979 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 17 07:28:00.181396 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.181382 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 07:28:00.181435 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.181409 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 07:28:00.184107 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.184088 2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 17 07:28:00.187114 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.187097 2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 17 07:28:00.187183 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.187129 2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 17 07:28:00.188998 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.188983 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 17 07:28:00.188998 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.189000 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 17 07:28:00.189126 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.189005 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 17 07:28:00.189126 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.189011 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 17 07:28:00.189126 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.189019 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 17 07:28:00.189126 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.189024 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 17 07:28:00.189126 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.189042 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 17 07:28:00.189126 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.189048 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 17 07:28:00.189126 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.189056 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 17 07:28:00.189126 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.189061 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 17 07:28:00.189126 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.189079 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 17 07:28:00.189126 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.189086 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 17 07:28:00.189828 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.189815 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 17 07:28:00.189828 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.189828 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 17 07:28:00.193875 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.193848 2579 server.go:1268] "Started kubelet" Sep 17 07:28:00.194020 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.193966 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 17 07:28:00.194087 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.194048 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 17 07:28:00.194173 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.194153 2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 17 07:28:00.194749 ip-10-0-129-217 systemd[1]: Started Kubernetes Kubelet. Sep 17 07:28:00.197876 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.197841 2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 17 07:28:00.198641 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.198574 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-217.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 17 07:28:00.199414 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.199388 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-217.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 17 07:28:00.199507 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.199391 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 17 07:28:00.199582 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.199556 2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 17 07:28:00.199854 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.199837 2579 server.go:310] "Adding debug handlers to kubelet server" Sep 17 07:28:00.203739 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.203721 2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 17 07:28:00.203837 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.203739 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 17 07:28:00.203837 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.202822 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-217.ec2.internal.18d60b13794e6596 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-217.ec2.internal,UID:ip-10-0-129-217.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-217.ec2.internal,},FirstTimestamp:2026-09-17 07:28:00.193815958 +0000 UTC m=+0.353162181,LastTimestamp:2026-09-17 07:28:00.193815958 +0000 UTC m=+0.353162181,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-217.ec2.internal,}" Sep 17 07:28:00.203837 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.203791 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Sep 17 07:28:00.204712 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.204695 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 17 07:28:00.204712 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.204711 2579 factory.go:55] Registering systemd factory Sep 17 07:28:00.204839 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.204719 2579 factory.go:223] Registration of the systemd container factory successfully Sep 17 07:28:00.204905 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.204877 2579 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 17 07:28:00.204952 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.204915 2579 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 17 07:28:00.205004 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.204952 2579 factory.go:153] Registering CRI-O factory Sep 17 07:28:00.205004 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.204966 2579 factory.go:223] Registration of the crio container factory successfully Sep 17 07:28:00.205004 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.204985 2579 factory.go:103] Registering Raw factory Sep 17 07:28:00.205004 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.204998 2579 manager.go:1196] Started watching for new ooms in manager Sep 17 07:28:00.205158 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.205055 2579 reconstruct.go:97] "Volume reconstruction finished" Sep 17 07:28:00.205158 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.205084 2579 reconciler.go:29] "Reconciler: start to sync state" Sep 17 07:28:00.205158 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.205063 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 17 07:28:00.205976 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.205947 2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-217.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 17 07:28:00.206050 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.205992 2579 manager.go:349] Starting recovery of all containers Sep 17 07:28:00.212814 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.212778 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 17 07:28:00.216280 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.215682 2579 manager.go:354] Recovery completed Sep 17 07:28:00.219888 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.219866 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 17 07:28:00.221811 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.221794 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:28:00.224040 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.224018 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:28:00.224105 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.224059 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:28:00.224105 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.224074 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:28:00.224797 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.224779 2579 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 17 07:28:00.224797 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.224795 2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 17 07:28:00.224894 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.224813 2579 state_mem.go:36] "Initialized new in-memory state store" Sep 17 07:28:00.224894 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.224786 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:28:00.225549 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.225534 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:28:00.225601 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.225563 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:28:00.225601 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.225577 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:28:00.227025 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.227010 2579 policy_none.go:49] "None policy: Start" Sep 17 07:28:00.227025 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.227026 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 17 07:28:00.227130 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.227037 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 17 07:28:00.229016 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.229004 2579 policy_none.go:47] "Start" Sep 17 07:28:00.236163 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.236045 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-217.ec2.internal.18d60b137b1b9280 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-217.ec2.internal,UID:ip-10-0-129-217.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-217.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-217.ec2.internal,},FirstTimestamp:2026-09-17 07:28:00.224039552 +0000 UTC m=+0.383385775,LastTimestamp:2026-09-17 07:28:00.224039552 +0000 UTC m=+0.383385775,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-217.ec2.internal,}" Sep 17 07:28:00.245127 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.245059 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-217.ec2.internal.18d60b137b1bf772 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-217.ec2.internal,UID:ip-10-0-129-217.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-217.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-217.ec2.internal,},FirstTimestamp:2026-09-17 07:28:00.224065394 +0000 UTC m=+0.383411618,LastTimestamp:2026-09-17 07:28:00.224065394 +0000 UTC m=+0.383411618,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-217.ec2.internal,}" Sep 17 07:28:00.248915 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.248783 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pvkhc" Sep 17 07:28:00.283912 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.255485 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-217.ec2.internal.18d60b137b1c2d51 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-217.ec2.internal,UID:ip-10-0-129-217.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-217.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-217.ec2.internal,},FirstTimestamp:2026-09-17 07:28:00.224079185 +0000 UTC m=+0.383425408,LastTimestamp:2026-09-17 07:28:00.224079185 +0000 UTC m=+0.383425408,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-217.ec2.internal,}" Sep 17 07:28:00.283912 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.257365 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pvkhc" Sep 17 07:28:00.283912 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.265449 2579 manager.go:337] "Starting Device Plugin manager" Sep 17 07:28:00.283912 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.265490 2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 17 07:28:00.283912 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.265500 2579 server.go:87] "Starting device plugin registration server" Sep 17 07:28:00.283912 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.265832 2579 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 17 07:28:00.283912 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.265845 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 17 07:28:00.283912 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.265939 2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 17 07:28:00.283912 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.266046 2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 17 07:28:00.283912 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.266054 2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 17 07:28:00.283912 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.266661 2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 17 07:28:00.283912 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.266700 2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-217.ec2.internal\" not found" Sep 17 07:28:00.335501 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.335467 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 17 07:28:00.335699 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.335539 2579 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 17 07:28:00.335699 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.335568 2579 kubelet.go:2448] "Starting kubelet main sync loop" Sep 17 07:28:00.335793 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.335703 2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 17 07:28:00.338356 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.338334 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:28:00.367004 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.366928 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:28:00.369152 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.369126 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:28:00.369302 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.369169 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:28:00.369302 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.369180 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:28:00.369302 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.369208 2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-217.ec2.internal" Sep 17 07:28:00.381212 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.381192 2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-217.ec2.internal" Sep 17 07:28:00.381272 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.381216 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-217.ec2.internal\": node \"ip-10-0-129-217.ec2.internal\" not found" Sep 17 07:28:00.416968 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.416940 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Sep 17 07:28:00.436367 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.436333 2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal"] Sep 17 07:28:00.436545 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.436431 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:28:00.437943 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.437925 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:28:00.438014 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.437961 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:28:00.438014 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.437974 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:28:00.440277 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.440263 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:28:00.440405 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.440390 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" Sep 17 07:28:00.440467 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.440427 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:28:00.441545 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.441528 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:28:00.441634 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.441557 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:28:00.441634 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.441594 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:28:00.441704 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.441532 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:28:00.441704 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.441662 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:28:00.441704 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.441673 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:28:00.443932 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.443920 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal" Sep 17 07:28:00.443979 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.443944 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:28:00.444928 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.444911 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:28:00.445022 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.444940 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:28:00.445022 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.444951 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:28:00.466539 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.466517 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-217.ec2.internal\" not found" node="ip-10-0-129-217.ec2.internal" Sep 17 07:28:00.471181 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.471166 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-217.ec2.internal\" not found" node="ip-10-0-129-217.ec2.internal" Sep 17 07:28:00.507182 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.507152 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b580332feb733df4d6e54bb54860651-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal\" (UID: \"2b580332feb733df4d6e54bb54860651\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" Sep 17 07:28:00.507182 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.507186 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/24d443373f3eb9148c2f63dcb77fe917-config\") pod \"kube-apiserver-proxy-ip-10-0-129-217.ec2.internal\" (UID: \"24d443373f3eb9148c2f63dcb77fe917\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal" Sep 17 07:28:00.507371 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.507202 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2b580332feb733df4d6e54bb54860651-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal\" (UID: \"2b580332feb733df4d6e54bb54860651\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" Sep 17 07:28:00.517533 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.517513 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Sep 17 07:28:00.608208 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.608176 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/24d443373f3eb9148c2f63dcb77fe917-config\") pod \"kube-apiserver-proxy-ip-10-0-129-217.ec2.internal\" (UID: \"24d443373f3eb9148c2f63dcb77fe917\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal" Sep 17 07:28:00.608208 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.608206 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2b580332feb733df4d6e54bb54860651-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal\" (UID: \"2b580332feb733df4d6e54bb54860651\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" Sep 17 07:28:00.608439 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.608229 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b580332feb733df4d6e54bb54860651-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal\" (UID: \"2b580332feb733df4d6e54bb54860651\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" Sep 17 07:28:00.608439 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.608310 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/24d443373f3eb9148c2f63dcb77fe917-config\") pod \"kube-apiserver-proxy-ip-10-0-129-217.ec2.internal\" (UID: \"24d443373f3eb9148c2f63dcb77fe917\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal" Sep 17 07:28:00.608439 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.608308 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b580332feb733df4d6e54bb54860651-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal\" (UID: \"2b580332feb733df4d6e54bb54860651\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" Sep 17 07:28:00.608439 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.608321 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2b580332feb733df4d6e54bb54860651-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal\" (UID: \"2b580332feb733df4d6e54bb54860651\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" Sep 17 07:28:00.618376 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.618297 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Sep 17 07:28:00.719191 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.719150 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Sep 17 07:28:00.769351 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.769312 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" Sep 17 07:28:00.774896 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:00.774444 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal" Sep 17 07:28:00.819937 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.819901 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Sep 17 07:28:00.920678 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:00.920543 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Sep 17 07:28:01.021175 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:01.021136 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Sep 17 07:28:01.121855 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:01.121812 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Sep 17 07:28:01.139016 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:01.138986 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 17 07:28:01.139181 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:01.139163 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 17 07:28:01.210536 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:01.210469 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:28:01.221922 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:01.221894 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Sep 17 07:28:01.262273 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:01.262213 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 07:23:02 +0000 UTC" deadline="2028-05-01 11:59:20.194876062 +0000 UTC" Sep 17 07:28:01.262273 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:01.262268 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14212h31m18.932611508s" Sep 17 07:28:01.291296 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:01.291268 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:28:01.292450 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:01.292432 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:28:01.292569 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:01.292462 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:28:01.292569 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:01.292474 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:28:01.323002 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:01.322966 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Sep 17 07:28:01.348474 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:01.348447 2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:28:01.404214 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:01.404186 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" Sep 17 07:28:01.405675 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:01.405642 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24d443373f3eb9148c2f63dcb77fe917.slice/crio-2b1f2c4020ad9c3b30b9afa13dc01d08cc3529b0d49b3074f3e92dcfea6d1907 WatchSource:0}: Error finding container 2b1f2c4020ad9c3b30b9afa13dc01d08cc3529b0d49b3074f3e92dcfea6d1907: Status 404 returned error can't find the container with id 2b1f2c4020ad9c3b30b9afa13dc01d08cc3529b0d49b3074f3e92dcfea6d1907 Sep 17 07:28:01.405925 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:01.405898 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b580332feb733df4d6e54bb54860651.slice/crio-8897536c63e43f526b0d0d8ecc994c71fe6b8bc3a9600bf8da1c9cd4a5be1034 WatchSource:0}: Error finding container 8897536c63e43f526b0d0d8ecc994c71fe6b8bc3a9600bf8da1c9cd4a5be1034: Status 404 returned error can't find the container with id 8897536c63e43f526b0d0d8ecc994c71fe6b8bc3a9600bf8da1c9cd4a5be1034 Sep 17 07:28:01.409431 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:01.409415 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 07:28:01.420238 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:01.420220 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 17 07:28:01.422264 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:01.422245 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal" Sep 17 07:28:01.431930 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:01.431913 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 17 07:28:01.516671 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:01.516585 2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:28:02.181740 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.181704 2579 apiserver.go:52] "Watching apiserver" Sep 17 07:28:02.191098 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.191065 2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 17 07:28:02.192345 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.192309 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-mmcvp","kube-system/konnectivity-agent-v46tc","kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal","openshift-multus/multus-x7r9x","openshift-network-diagnostics/network-check-target-njpsg","openshift-ovn-kubernetes/ovnkube-node-nsjvb","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng","openshift-cluster-node-tuning-operator/tuned-xfn69","openshift-dns/node-resolver-dml6l","openshift-image-registry/node-ca-gmkxm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal","openshift-multus/multus-additional-cni-plugins-rvnph","openshift-multus/network-metrics-daemon-zjz9v","openshift-network-operator/iptables-alerter-pm5vm"] Sep 17 07:28:02.194922 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.194900 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.197334 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.197282 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:02.197441 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:02.197389 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjz9v" podUID="5eb544bb-f51e-4755-891e-ed3e8f031f0d" Sep 17 07:28:02.197543 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.197520 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 17 07:28:02.197682 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.197649 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 17 07:28:02.197682 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.197663 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 17 07:28:02.197803 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.197770 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-f8d27\"" Sep 17 07:28:02.199709 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.199448 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pm5vm" Sep 17 07:28:02.201749 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.201725 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.202902 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.202673 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 17 07:28:02.202902 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.202744 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 17 07:28:02.202902 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.202764 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 17 07:28:02.203094 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.202922 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mg29x\"" Sep 17 07:28:02.203995 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.203976 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 17 07:28:02.203995 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.203987 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 17 07:28:02.204889 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.204452 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-62wxf\"" Sep 17 07:28:02.204889 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.204487 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 17 07:28:02.204889 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.204716 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 17 07:28:02.206188 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.205594 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 17 07:28:02.206188 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.205847 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:02.206188 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:02.205901 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njpsg" podUID="e7339f6e-91b9-4b00-b7aa-915f9ea79232" Sep 17 07:28:02.207861 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.207837 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.210224 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.210015 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dml6l" Sep 17 07:28:02.211323 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.211305 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 17 07:28:02.212517 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.212496 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:02.212609 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:02.212572 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mmcvp" podUID="d4580c7f-1c6c-420d-962c-0ba1a6a222fa" Sep 17 07:28:02.213252 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.212888 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 17 07:28:02.213252 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.212906 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 17 07:28:02.213252 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.213059 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 17 07:28:02.213439 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.213359 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 17 07:28:02.213439 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.213403 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 17 07:28:02.214184 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.213734 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 17 07:28:02.214184 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.213965 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 17 07:28:02.214444 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.214252 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-w8cjt\"" Sep 17 07:28:02.214444 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.214380 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2rx4j\"" Sep 17 07:28:02.216814 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.216793 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/25122293-6ff9-4681-ac0f-3283e6de9a5d-registration-dir\") pod \"aws-ebs-csi-driver-node-p2bng\" (UID: \"25122293-6ff9-4681-ac0f-3283e6de9a5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.216911 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.216830 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-hostroot\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.216911 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.216901 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-host-run-multus-certs\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.217020 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.216934 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-run-systemd\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.217020 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.216969 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-etc-openvswitch\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.217020 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.216999 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-host-cni-bin\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.217165 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217023 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/25122293-6ff9-4681-ac0f-3283e6de9a5d-device-dir\") pod \"aws-ebs-csi-driver-node-p2bng\" (UID: \"25122293-6ff9-4681-ac0f-3283e6de9a5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.217165 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217055 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zczwg\" (UniqueName: \"kubernetes.io/projected/1c554023-f031-4773-b6ef-ef864215ba80-kube-api-access-zczwg\") pod \"iptables-alerter-pm5vm\" (UID: \"1c554023-f031-4773-b6ef-ef864215ba80\") " pod="openshift-network-operator/iptables-alerter-pm5vm" Sep 17 07:28:02.217165 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217087 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-system-cni-dir\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.217165 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217117 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-host-run-k8s-cni-cncf-io\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.217165 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217146 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-host-var-lib-cni-bin\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.217413 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217171 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-multus-conf-dir\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.217413 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217185 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v46tc" Sep 17 07:28:02.217413 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217203 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-etc-kubernetes\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.217413 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217235 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-host-cni-netd\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.217413 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217269 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/25122293-6ff9-4681-ac0f-3283e6de9a5d-etc-selinux\") pod \"aws-ebs-csi-driver-node-p2bng\" (UID: \"25122293-6ff9-4681-ac0f-3283e6de9a5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.217413 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217301 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/25122293-6ff9-4681-ac0f-3283e6de9a5d-sys-fs\") pod \"aws-ebs-csi-driver-node-p2bng\" (UID: \"25122293-6ff9-4681-ac0f-3283e6de9a5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.217413 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217328 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwplc\" (UniqueName: \"kubernetes.io/projected/5eb544bb-f51e-4755-891e-ed3e8f031f0d-kube-api-access-wwplc\") pod \"network-metrics-daemon-zjz9v\" (UID: \"5eb544bb-f51e-4755-891e-ed3e8f031f0d\") " pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:02.217413 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217361 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-log-socket\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.217413 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217389 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/42e9a882-f084-4752-af08-6911b3237ae2-ovnkube-config\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.217858 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217419 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/42e9a882-f084-4752-af08-6911b3237ae2-env-overrides\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.217858 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217451 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/25122293-6ff9-4681-ac0f-3283e6de9a5d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p2bng\" (UID: \"25122293-6ff9-4681-ac0f-3283e6de9a5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.217858 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217480 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-cnibin\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.217858 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217511 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6fb6478a-4047-42e0-a66f-336860a6867c-cni-binary-copy\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.217858 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217541 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6fb6478a-4047-42e0-a66f-336860a6867c-multus-daemon-config\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.217858 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217575 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-systemd-units\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.217858 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217603 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-host-run-netns\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.217858 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217644 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-var-lib-openvswitch\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.217858 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217675 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs\") pod \"network-metrics-daemon-zjz9v\" (UID: \"5eb544bb-f51e-4755-891e-ed3e8f031f0d\") " pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:02.217858 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217705 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1c554023-f031-4773-b6ef-ef864215ba80-host-slash\") pod \"iptables-alerter-pm5vm\" (UID: \"1c554023-f031-4773-b6ef-ef864215ba80\") " pod="openshift-network-operator/iptables-alerter-pm5vm" Sep 17 07:28:02.217858 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217735 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-host-var-lib-kubelet\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.217858 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217766 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.217858 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217792 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/42e9a882-f084-4752-af08-6911b3237ae2-ovnkube-script-lib\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.217858 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217825 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/25122293-6ff9-4681-ac0f-3283e6de9a5d-socket-dir\") pod \"aws-ebs-csi-driver-node-p2bng\" (UID: \"25122293-6ff9-4681-ac0f-3283e6de9a5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.218498 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.217871 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1c554023-f031-4773-b6ef-ef864215ba80-iptables-alerter-script\") pod \"iptables-alerter-pm5vm\" (UID: \"1c554023-f031-4773-b6ef-ef864215ba80\") " pod="openshift-network-operator/iptables-alerter-pm5vm" Sep 17 07:28:02.218498 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.218029 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-multus-cni-dir\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.218498 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.218068 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-multus-socket-dir-parent\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.218498 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.218095 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9cvc\" (UniqueName: \"kubernetes.io/projected/6fb6478a-4047-42e0-a66f-336860a6867c-kube-api-access-c9cvc\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.218498 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.218353 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-host-slash\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.218498 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.218384 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-run-openvswitch\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.218498 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.218441 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-run-ovn\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.218858 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.218519 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-host-var-lib-cni-multus\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.218858 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.218553 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-node-log\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.218858 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.218633 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/42e9a882-f084-4752-af08-6911b3237ae2-ovn-node-metrics-cert\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.218858 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.218725 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgj6n\" (UniqueName: \"kubernetes.io/projected/42e9a882-f084-4752-af08-6911b3237ae2-kube-api-access-hgj6n\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.219047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.218778 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v9dg\" (UniqueName: \"kubernetes.io/projected/25122293-6ff9-4681-ac0f-3283e6de9a5d-kube-api-access-9v9dg\") pod \"aws-ebs-csi-driver-node-p2bng\" (UID: \"25122293-6ff9-4681-ac0f-3283e6de9a5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.219047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.219006 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-host-run-netns\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.219139 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.219056 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-422lx\" (UniqueName: \"kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx\") pod \"network-check-target-njpsg\" (UID: \"e7339f6e-91b9-4b00-b7aa-915f9ea79232\") " pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:02.219139 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.219093 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-host-kubelet\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.219235 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.219202 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-host-run-ovn-kubernetes\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.220934 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.220757 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 17 07:28:02.220934 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.220786 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jrjlc\"" Sep 17 07:28:02.220934 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.220815 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 17 07:28:02.221144 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.221061 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gmkxm" Sep 17 07:28:02.222553 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.222532 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 07:28:02.223403 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.223271 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 17 07:28:02.223502 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.223448 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 17 07:28:02.223502 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.223486 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 17 07:28:02.223606 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.223530 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:02.223724 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.223687 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xkmph\"" Sep 17 07:28:02.226097 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.225786 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-t9xqk\"" Sep 17 07:28:02.226097 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.225897 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:02.226097 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.226062 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 17 07:28:02.226287 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.226264 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 17 07:28:02.228374 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.228353 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 17 07:28:02.228932 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.228757 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-558ds\"" Sep 17 07:28:02.228932 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.228787 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 17 07:28:02.261127 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.261105 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-w4pd6" Sep 17 07:28:02.270406 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.270372 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-w4pd6" Sep 17 07:28:02.306222 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.306193 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 17 07:28:02.319583 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.319545 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-multus-socket-dir-parent\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.319583 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.319576 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9cvc\" (UniqueName: \"kubernetes.io/projected/6fb6478a-4047-42e0-a66f-336860a6867c-kube-api-access-c9cvc\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.319832 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.319594 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-host-slash\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.319832 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.319616 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-run-ovn\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.319832 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.319675 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/42e9a882-f084-4752-af08-6911b3237ae2-ovn-node-metrics-cert\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.319832 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.319701 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-host-var-lib-cni-multus\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.319832 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.319701 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-host-slash\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.319832 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.319716 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-run-ovn\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.319832 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.319702 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-multus-socket-dir-parent\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.319832 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.319723 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-node-log\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.319832 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.319766 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-node-log\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.319832 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.319775 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-host-var-lib-cni-multus\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.319832 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.319778 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/073304c8-0913-4ef9-8388-920b0f7dd353-konnectivity-ca\") pod \"konnectivity-agent-v46tc\" (UID: \"073304c8-0913-4ef9-8388-920b0f7dd353\") " pod="kube-system/konnectivity-agent-v46tc" Sep 17 07:28:02.319832 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.319815 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-etc-modprobe-d\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:02.319832 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.319846 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-sys\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:02.320456 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.319897 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-lib-modules\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:02.320456 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.319945 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-host-run-netns\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.320456 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.319974 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-422lx\" (UniqueName: \"kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx\") pod \"network-check-target-njpsg\" (UID: \"e7339f6e-91b9-4b00-b7aa-915f9ea79232\") " pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:02.320456 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320002 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-host-run-ovn-kubernetes\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.320456 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320053 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-host-run-netns\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.320456 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320092 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98b322c1-0c60-4b8d-b79b-32c529a63ccd-cni-binary-copy\") pod \"multus-additional-cni-plugins-rvnph\" (UID: \"98b322c1-0c60-4b8d-b79b-32c529a63ccd\") " pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:02.320456 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320101 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-host-run-ovn-kubernetes\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.320456 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320118 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c327a988-59e8-48d2-a9fd-ff4668b66a22-tmp-dir\") pod \"node-resolver-dml6l\" (UID: \"c327a988-59e8-48d2-a9fd-ff4668b66a22\") " pod="openshift-dns/node-resolver-dml6l" Sep 17 07:28:02.320456 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320151 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/25122293-6ff9-4681-ac0f-3283e6de9a5d-registration-dir\") pod \"aws-ebs-csi-driver-node-p2bng\" (UID: \"25122293-6ff9-4681-ac0f-3283e6de9a5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.320456 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320220 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-host-run-multus-certs\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.320456 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320251 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c327a988-59e8-48d2-a9fd-ff4668b66a22-hosts-file\") pod \"node-resolver-dml6l\" (UID: \"c327a988-59e8-48d2-a9fd-ff4668b66a22\") " pod="openshift-dns/node-resolver-dml6l" Sep 17 07:28:02.320456 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320277 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-etc-sysctl-conf\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:02.320456 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320300 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-etc-systemd\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:02.320456 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320303 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/25122293-6ff9-4681-ac0f-3283e6de9a5d-registration-dir\") pod \"aws-ebs-csi-driver-node-p2bng\" (UID: \"25122293-6ff9-4681-ac0f-3283e6de9a5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.320456 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320296 2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 17 07:28:02.320456 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320343 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-host-run-k8s-cni-cncf-io\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.320456 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320382 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-host-var-lib-cni-bin\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.321212 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320300 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-host-run-multus-certs\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.321212 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320403 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-host-run-k8s-cni-cncf-io\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.321212 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320443 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-etc-kubernetes\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.321212 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320481 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-host-var-lib-cni-bin\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.321212 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320525 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/42e9a882-f084-4752-af08-6911b3237ae2-env-overrides\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.321212 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320543 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-etc-kubernetes\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.321212 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320580 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/25122293-6ff9-4681-ac0f-3283e6de9a5d-sys-fs\") pod \"aws-ebs-csi-driver-node-p2bng\" (UID: \"25122293-6ff9-4681-ac0f-3283e6de9a5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.321212 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320655 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/42e9a882-f084-4752-af08-6911b3237ae2-ovnkube-config\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.321212 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320685 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/25122293-6ff9-4681-ac0f-3283e6de9a5d-sys-fs\") pod \"aws-ebs-csi-driver-node-p2bng\" (UID: \"25122293-6ff9-4681-ac0f-3283e6de9a5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.321212 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320731 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c538774b-3450-4093-84a6-7cd9f9a0167e-etc-tuned\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:02.321212 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320832 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/25122293-6ff9-4681-ac0f-3283e6de9a5d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p2bng\" (UID: \"25122293-6ff9-4681-ac0f-3283e6de9a5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.321212 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320884 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-cnibin\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.321212 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320921 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6fb6478a-4047-42e0-a66f-336860a6867c-cni-binary-copy\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.321212 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320945 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-systemd-units\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.321212 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.320935 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/25122293-6ff9-4681-ac0f-3283e6de9a5d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p2bng\" (UID: \"25122293-6ff9-4681-ac0f-3283e6de9a5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.321212 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321002 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-host-run-netns\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.321212 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321035 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret\") pod \"global-pull-secret-syncer-mmcvp\" (UID: \"d4580c7f-1c6c-420d-962c-0ba1a6a222fa\") " pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:02.321934 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321055 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/42e9a882-f084-4752-af08-6911b3237ae2-env-overrides\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.321934 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321064 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-etc-sysconfig\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:02.321934 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321096 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-cnibin\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.321934 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321136 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-systemd-units\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.321934 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321172 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxnfp\" (UniqueName: \"kubernetes.io/projected/c538774b-3450-4093-84a6-7cd9f9a0167e-kube-api-access-vxnfp\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:02.321934 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321222 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs\") pod \"network-metrics-daemon-zjz9v\" (UID: \"5eb544bb-f51e-4755-891e-ed3e8f031f0d\") " pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:02.321934 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321231 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/42e9a882-f084-4752-af08-6911b3237ae2-ovnkube-config\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.321934 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321248 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-etc-sysctl-d\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:02.321934 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321306 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-run-openvswitch\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.321934 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321338 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgj6n\" (UniqueName: \"kubernetes.io/projected/42e9a882-f084-4752-af08-6911b3237ae2-kube-api-access-hgj6n\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.321934 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:02.321340 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:02.321934 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321372 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8kqz\" (UniqueName: \"kubernetes.io/projected/c327a988-59e8-48d2-a9fd-ff4668b66a22-kube-api-access-h8kqz\") pod \"node-resolver-dml6l\" (UID: \"c327a988-59e8-48d2-a9fd-ff4668b66a22\") " pod="openshift-dns/node-resolver-dml6l" Sep 17 07:28:02.321934 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321402 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-dbus\") pod \"global-pull-secret-syncer-mmcvp\" (UID: \"d4580c7f-1c6c-420d-962c-0ba1a6a222fa\") " pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:02.321934 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321416 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-run-openvswitch\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.321934 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321433 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-etc-kubernetes\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:02.321934 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321456 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-host-run-netns\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.321934 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321463 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-host\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:02.321934 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321530 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98b322c1-0c60-4b8d-b79b-32c529a63ccd-cnibin\") pod \"multus-additional-cni-plugins-rvnph\" (UID: \"98b322c1-0c60-4b8d-b79b-32c529a63ccd\") " pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:02.322795 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:02.321547 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs podName:5eb544bb-f51e-4755-891e-ed3e8f031f0d nodeName:}" failed. No retries permitted until 2026-09-17 07:28:02.821526774 +0000 UTC m=+2.980872991 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs") pod "network-metrics-daemon-zjz9v" (UID: "5eb544bb-f51e-4755-891e-ed3e8f031f0d") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:02.322795 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321582 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6fb6478a-4047-42e0-a66f-336860a6867c-cni-binary-copy\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.322795 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321594 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v9dg\" (UniqueName: \"kubernetes.io/projected/25122293-6ff9-4681-ac0f-3283e6de9a5d-kube-api-access-9v9dg\") pod \"aws-ebs-csi-driver-node-p2bng\" (UID: \"25122293-6ff9-4681-ac0f-3283e6de9a5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.322795 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321618 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-host-kubelet\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.322795 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321664 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-kubelet-config\") pod \"global-pull-secret-syncer-mmcvp\" (UID: \"d4580c7f-1c6c-420d-962c-0ba1a6a222fa\") " pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:02.322795 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321671 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-host-kubelet\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.322795 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321687 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c538774b-3450-4093-84a6-7cd9f9a0167e-tmp\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:02.322795 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321739 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d59a6e2b-8dd8-45ff-be5f-01ad01f02124-serviceca\") pod \"node-ca-gmkxm\" (UID: \"d59a6e2b-8dd8-45ff-be5f-01ad01f02124\") " pod="openshift-image-registry/node-ca-gmkxm" Sep 17 07:28:02.322795 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321782 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-hostroot\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.322795 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321806 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-run-systemd\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.322795 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321830 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-etc-openvswitch\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.322795 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321856 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-host-cni-bin\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.322795 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321861 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-hostroot\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.322795 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321878 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/25122293-6ff9-4681-ac0f-3283e6de9a5d-device-dir\") pod \"aws-ebs-csi-driver-node-p2bng\" (UID: \"25122293-6ff9-4681-ac0f-3283e6de9a5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.322795 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321882 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-run-systemd\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.322795 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321886 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-etc-openvswitch\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.322795 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321906 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-host-cni-bin\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.323570 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321906 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zczwg\" (UniqueName: \"kubernetes.io/projected/1c554023-f031-4773-b6ef-ef864215ba80-kube-api-access-zczwg\") pod \"iptables-alerter-pm5vm\" (UID: \"1c554023-f031-4773-b6ef-ef864215ba80\") " pod="openshift-network-operator/iptables-alerter-pm5vm" Sep 17 07:28:02.323570 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321941 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-system-cni-dir\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.323570 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321958 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/25122293-6ff9-4681-ac0f-3283e6de9a5d-device-dir\") pod \"aws-ebs-csi-driver-node-p2bng\" (UID: \"25122293-6ff9-4681-ac0f-3283e6de9a5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.323570 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321968 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-multus-conf-dir\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.323570 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.321991 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-host-cni-netd\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.323570 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322016 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7w5b\" (UniqueName: \"kubernetes.io/projected/d59a6e2b-8dd8-45ff-be5f-01ad01f02124-kube-api-access-b7w5b\") pod \"node-ca-gmkxm\" (UID: \"d59a6e2b-8dd8-45ff-be5f-01ad01f02124\") " pod="openshift-image-registry/node-ca-gmkxm" Sep 17 07:28:02.323570 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322036 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-system-cni-dir\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.323570 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322035 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-multus-conf-dir\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.323570 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322043 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/98b322c1-0c60-4b8d-b79b-32c529a63ccd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rvnph\" (UID: \"98b322c1-0c60-4b8d-b79b-32c529a63ccd\") " pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:02.323570 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322063 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-host-cni-netd\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.323570 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322098 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/25122293-6ff9-4681-ac0f-3283e6de9a5d-etc-selinux\") pod \"aws-ebs-csi-driver-node-p2bng\" (UID: \"25122293-6ff9-4681-ac0f-3283e6de9a5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.323570 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322124 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwplc\" (UniqueName: \"kubernetes.io/projected/5eb544bb-f51e-4755-891e-ed3e8f031f0d-kube-api-access-wwplc\") pod \"network-metrics-daemon-zjz9v\" (UID: \"5eb544bb-f51e-4755-891e-ed3e8f031f0d\") " pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:02.323570 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322147 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-log-socket\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.323570 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322172 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-run\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:02.323570 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322175 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/25122293-6ff9-4681-ac0f-3283e6de9a5d-etc-selinux\") pod \"aws-ebs-csi-driver-node-p2bng\" (UID: \"25122293-6ff9-4681-ac0f-3283e6de9a5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.323570 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322196 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-var-lib-kubelet\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:02.323570 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322219 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-log-socket\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.324341 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322225 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98b322c1-0c60-4b8d-b79b-32c529a63ccd-system-cni-dir\") pod \"multus-additional-cni-plugins-rvnph\" (UID: \"98b322c1-0c60-4b8d-b79b-32c529a63ccd\") " pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:02.324341 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322259 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/98b322c1-0c60-4b8d-b79b-32c529a63ccd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rvnph\" (UID: \"98b322c1-0c60-4b8d-b79b-32c529a63ccd\") " pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:02.324341 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322297 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtz29\" (UniqueName: \"kubernetes.io/projected/98b322c1-0c60-4b8d-b79b-32c529a63ccd-kube-api-access-rtz29\") pod \"multus-additional-cni-plugins-rvnph\" (UID: \"98b322c1-0c60-4b8d-b79b-32c529a63ccd\") " pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:02.324341 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322329 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6fb6478a-4047-42e0-a66f-336860a6867c-multus-daemon-config\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.324341 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322354 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-var-lib-openvswitch\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.324341 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322376 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98b322c1-0c60-4b8d-b79b-32c529a63ccd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rvnph\" (UID: \"98b322c1-0c60-4b8d-b79b-32c529a63ccd\") " pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:02.324341 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322401 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1c554023-f031-4773-b6ef-ef864215ba80-host-slash\") pod \"iptables-alerter-pm5vm\" (UID: \"1c554023-f031-4773-b6ef-ef864215ba80\") " pod="openshift-network-operator/iptables-alerter-pm5vm" Sep 17 07:28:02.324341 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322414 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-var-lib-openvswitch\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.324341 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322426 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-host-var-lib-kubelet\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.324341 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322452 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.324341 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322471 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1c554023-f031-4773-b6ef-ef864215ba80-host-slash\") pod \"iptables-alerter-pm5vm\" (UID: \"1c554023-f031-4773-b6ef-ef864215ba80\") " pod="openshift-network-operator/iptables-alerter-pm5vm" Sep 17 07:28:02.324341 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322483 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/42e9a882-f084-4752-af08-6911b3237ae2-ovnkube-script-lib\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.324341 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322508 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/073304c8-0913-4ef9-8388-920b0f7dd353-agent-certs\") pod \"konnectivity-agent-v46tc\" (UID: \"073304c8-0913-4ef9-8388-920b0f7dd353\") " pod="kube-system/konnectivity-agent-v46tc" Sep 17 07:28:02.324341 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322515 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-host-var-lib-kubelet\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.324341 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322530 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d59a6e2b-8dd8-45ff-be5f-01ad01f02124-host\") pod \"node-ca-gmkxm\" (UID: \"d59a6e2b-8dd8-45ff-be5f-01ad01f02124\") " pod="openshift-image-registry/node-ca-gmkxm" Sep 17 07:28:02.324341 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322547 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42e9a882-f084-4752-af08-6911b3237ae2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.324955 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322553 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/25122293-6ff9-4681-ac0f-3283e6de9a5d-socket-dir\") pod \"aws-ebs-csi-driver-node-p2bng\" (UID: \"25122293-6ff9-4681-ac0f-3283e6de9a5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.324955 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322605 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1c554023-f031-4773-b6ef-ef864215ba80-iptables-alerter-script\") pod \"iptables-alerter-pm5vm\" (UID: \"1c554023-f031-4773-b6ef-ef864215ba80\") " pod="openshift-network-operator/iptables-alerter-pm5vm" Sep 17 07:28:02.324955 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322646 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-multus-cni-dir\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.324955 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322658 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/25122293-6ff9-4681-ac0f-3283e6de9a5d-socket-dir\") pod \"aws-ebs-csi-driver-node-p2bng\" (UID: \"25122293-6ff9-4681-ac0f-3283e6de9a5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.324955 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322747 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6fb6478a-4047-42e0-a66f-336860a6867c-multus-cni-dir\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.324955 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.322816 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6fb6478a-4047-42e0-a66f-336860a6867c-multus-daemon-config\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.324955 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.323046 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/42e9a882-f084-4752-af08-6911b3237ae2-ovnkube-script-lib\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.324955 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.323216 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1c554023-f031-4773-b6ef-ef864215ba80-iptables-alerter-script\") pod \"iptables-alerter-pm5vm\" (UID: \"1c554023-f031-4773-b6ef-ef864215ba80\") " pod="openshift-network-operator/iptables-alerter-pm5vm" Sep 17 07:28:02.324955 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.324498 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/42e9a882-f084-4752-af08-6911b3237ae2-ovn-node-metrics-cert\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.329463 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.329431 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9cvc\" (UniqueName: \"kubernetes.io/projected/6fb6478a-4047-42e0-a66f-336860a6867c-kube-api-access-c9cvc\") pod \"multus-x7r9x\" (UID: \"6fb6478a-4047-42e0-a66f-336860a6867c\") " pod="openshift-multus/multus-x7r9x" Sep 17 07:28:02.330055 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.330020 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v9dg\" (UniqueName: \"kubernetes.io/projected/25122293-6ff9-4681-ac0f-3283e6de9a5d-kube-api-access-9v9dg\") pod \"aws-ebs-csi-driver-node-p2bng\" (UID: \"25122293-6ff9-4681-ac0f-3283e6de9a5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:02.331692 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.331665 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgj6n\" (UniqueName: \"kubernetes.io/projected/42e9a882-f084-4752-af08-6911b3237ae2-kube-api-access-hgj6n\") pod \"ovnkube-node-nsjvb\" (UID: \"42e9a882-f084-4752-af08-6911b3237ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:02.332199 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.331907 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwplc\" (UniqueName: \"kubernetes.io/projected/5eb544bb-f51e-4755-891e-ed3e8f031f0d-kube-api-access-wwplc\") pod \"network-metrics-daemon-zjz9v\" (UID: \"5eb544bb-f51e-4755-891e-ed3e8f031f0d\") " pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:02.332199 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.332157 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zczwg\" (UniqueName: \"kubernetes.io/projected/1c554023-f031-4773-b6ef-ef864215ba80-kube-api-access-zczwg\") pod \"iptables-alerter-pm5vm\" (UID: \"1c554023-f031-4773-b6ef-ef864215ba80\") " pod="openshift-network-operator/iptables-alerter-pm5vm" Sep 17 07:28:02.332993 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:02.332970 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:28:02.332993 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:02.332998 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:28:02.333126 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:02.333012 2579 projected.go:196] Error preparing data for projected volume kube-api-access-422lx for pod openshift-network-diagnostics/network-check-target-njpsg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:02.333126 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:02.333077 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx podName:e7339f6e-91b9-4b00-b7aa-915f9ea79232 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:02.833061971 +0000 UTC m=+2.992408182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-422lx" (UniqueName: "kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx") pod "network-check-target-njpsg" (UID: "e7339f6e-91b9-4b00-b7aa-915f9ea79232") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:02.342462 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.342382 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal" event={"ID":"24d443373f3eb9148c2f63dcb77fe917","Type":"ContainerStarted","Data":"2b1f2c4020ad9c3b30b9afa13dc01d08cc3529b0d49b3074f3e92dcfea6d1907"} Sep 17 07:28:02.343527 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.343502 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" event={"ID":"2b580332feb733df4d6e54bb54860651","Type":"ContainerStarted","Data":"8897536c63e43f526b0d0d8ecc994c71fe6b8bc3a9600bf8da1c9cd4a5be1034"} Sep 17 07:28:02.423320 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.423287 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98b322c1-0c60-4b8d-b79b-32c529a63ccd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rvnph\" (UID: \"98b322c1-0c60-4b8d-b79b-32c529a63ccd\") " pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:02.423320 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.423325 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/073304c8-0913-4ef9-8388-920b0f7dd353-agent-certs\") pod \"konnectivity-agent-v46tc\" (UID: \"073304c8-0913-4ef9-8388-920b0f7dd353\") " pod="kube-system/konnectivity-agent-v46tc" Sep 17 07:28:02.423596 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.423345 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d59a6e2b-8dd8-45ff-be5f-01ad01f02124-host\") pod \"node-ca-gmkxm\" (UID: \"d59a6e2b-8dd8-45ff-be5f-01ad01f02124\") " pod="openshift-image-registry/node-ca-gmkxm" Sep 17 07:28:02.423596 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.423416 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d59a6e2b-8dd8-45ff-be5f-01ad01f02124-host\") pod \"node-ca-gmkxm\" (UID: \"d59a6e2b-8dd8-45ff-be5f-01ad01f02124\") " pod="openshift-image-registry/node-ca-gmkxm" Sep 17 07:28:02.423596 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.423449 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/073304c8-0913-4ef9-8388-920b0f7dd353-konnectivity-ca\") pod \"konnectivity-agent-v46tc\" (UID: \"073304c8-0913-4ef9-8388-920b0f7dd353\") " pod="kube-system/konnectivity-agent-v46tc" Sep 17 07:28:02.423596 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.423467 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-etc-modprobe-d\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:02.423596 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.423567 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98b322c1-0c60-4b8d-b79b-32c529a63ccd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rvnph\" (UID: \"98b322c1-0c60-4b8d-b79b-32c529a63ccd\") " pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:02.423596 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.423575 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-etc-modprobe-d\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.393384 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.423592 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-sys\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.393384 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.423657 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-lib-modules\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.393384 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.423667 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-sys\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.393384 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.423702 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98b322c1-0c60-4b8d-b79b-32c529a63ccd-cni-binary-copy\") pod \"multus-additional-cni-plugins-rvnph\" (UID: \"98b322c1-0c60-4b8d-b79b-32c529a63ccd\") " pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:04.393384 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.423728 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c327a988-59e8-48d2-a9fd-ff4668b66a22-tmp-dir\") pod \"node-resolver-dml6l\" (UID: \"c327a988-59e8-48d2-a9fd-ff4668b66a22\") " pod="openshift-dns/node-resolver-dml6l" Sep 17 07:28:04.393384 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.423753 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c327a988-59e8-48d2-a9fd-ff4668b66a22-hosts-file\") pod \"node-resolver-dml6l\" (UID: \"c327a988-59e8-48d2-a9fd-ff4668b66a22\") " pod="openshift-dns/node-resolver-dml6l" Sep 17 07:28:04.393384 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.423771 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-etc-sysctl-conf\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.393384 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:02.423795 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-etc-systemd\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.393384 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393193 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c538774b-3450-4093-84a6-7cd9f9a0167e-etc-tuned\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.393384 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393195 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-lib-modules\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.393384 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393221 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret\") pod \"global-pull-secret-syncer-mmcvp\" (UID: \"d4580c7f-1c6c-420d-962c-0ba1a6a222fa\") " pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:04.393384 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393244 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-etc-sysconfig\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.393384 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393266 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxnfp\" (UniqueName: \"kubernetes.io/projected/c538774b-3450-4093-84a6-7cd9f9a0167e-kube-api-access-vxnfp\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.393384 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393298 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-etc-sysctl-d\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.393384 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393321 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-etc-sysctl-conf\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.393384 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393325 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8kqz\" (UniqueName: \"kubernetes.io/projected/c327a988-59e8-48d2-a9fd-ff4668b66a22-kube-api-access-h8kqz\") pod \"node-resolver-dml6l\" (UID: \"c327a988-59e8-48d2-a9fd-ff4668b66a22\") " pod="openshift-dns/node-resolver-dml6l" Sep 17 07:28:04.393384 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393363 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-dbus\") pod \"global-pull-secret-syncer-mmcvp\" (UID: \"d4580c7f-1c6c-420d-962c-0ba1a6a222fa\") " pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:04.393384 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393387 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-etc-kubernetes\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.394664 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393399 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/073304c8-0913-4ef9-8388-920b0f7dd353-konnectivity-ca\") pod \"konnectivity-agent-v46tc\" (UID: \"073304c8-0913-4ef9-8388-920b0f7dd353\") " pod="kube-system/konnectivity-agent-v46tc" Sep 17 07:28:04.394664 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393408 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-host\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.394664 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393432 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98b322c1-0c60-4b8d-b79b-32c529a63ccd-cnibin\") pod \"multus-additional-cni-plugins-rvnph\" (UID: \"98b322c1-0c60-4b8d-b79b-32c529a63ccd\") " pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:04.394664 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393449 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c327a988-59e8-48d2-a9fd-ff4668b66a22-tmp-dir\") pod \"node-resolver-dml6l\" (UID: \"c327a988-59e8-48d2-a9fd-ff4668b66a22\") " pod="openshift-dns/node-resolver-dml6l" Sep 17 07:28:04.394664 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393459 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-kubelet-config\") pod \"global-pull-secret-syncer-mmcvp\" (UID: \"d4580c7f-1c6c-420d-962c-0ba1a6a222fa\") " pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:04.394664 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393485 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c538774b-3450-4093-84a6-7cd9f9a0167e-tmp\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.394664 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:04.393509 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:04.394664 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393518 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d59a6e2b-8dd8-45ff-be5f-01ad01f02124-serviceca\") pod \"node-ca-gmkxm\" (UID: \"d59a6e2b-8dd8-45ff-be5f-01ad01f02124\") " pod="openshift-image-registry/node-ca-gmkxm" Sep 17 07:28:04.394664 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393554 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7w5b\" (UniqueName: \"kubernetes.io/projected/d59a6e2b-8dd8-45ff-be5f-01ad01f02124-kube-api-access-b7w5b\") pod \"node-ca-gmkxm\" (UID: \"d59a6e2b-8dd8-45ff-be5f-01ad01f02124\") " pod="openshift-image-registry/node-ca-gmkxm" Sep 17 07:28:04.394664 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393563 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-host\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.394664 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393583 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-etc-sysconfig\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.394664 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:04.393588 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret podName:d4580c7f-1c6c-420d-962c-0ba1a6a222fa nodeName:}" failed. No retries permitted until 2026-09-17 07:28:04.89357026 +0000 UTC m=+3.083546184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret") pod "global-pull-secret-syncer-mmcvp" (UID: "d4580c7f-1c6c-420d-962c-0ba1a6a222fa") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:04.394664 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393606 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98b322c1-0c60-4b8d-b79b-32c529a63ccd-cnibin\") pod \"multus-additional-cni-plugins-rvnph\" (UID: \"98b322c1-0c60-4b8d-b79b-32c529a63ccd\") " pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:04.394664 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393613 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c327a988-59e8-48d2-a9fd-ff4668b66a22-hosts-file\") pod \"node-resolver-dml6l\" (UID: \"c327a988-59e8-48d2-a9fd-ff4668b66a22\") " pod="openshift-dns/node-resolver-dml6l" Sep 17 07:28:04.394664 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393617 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/98b322c1-0c60-4b8d-b79b-32c529a63ccd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rvnph\" (UID: \"98b322c1-0c60-4b8d-b79b-32c529a63ccd\") " pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:04.394664 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393638 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98b322c1-0c60-4b8d-b79b-32c529a63ccd-cni-binary-copy\") pod \"multus-additional-cni-plugins-rvnph\" (UID: \"98b322c1-0c60-4b8d-b79b-32c529a63ccd\") " pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:04.394664 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393652 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-run\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.395405 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393667 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-dbus\") pod \"global-pull-secret-syncer-mmcvp\" (UID: \"d4580c7f-1c6c-420d-962c-0ba1a6a222fa\") " pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:04.395405 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393671 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-kubelet-config\") pod \"global-pull-secret-syncer-mmcvp\" (UID: \"d4580c7f-1c6c-420d-962c-0ba1a6a222fa\") " pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:04.395405 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393678 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-var-lib-kubelet\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.395405 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393725 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-etc-systemd\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.395405 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393761 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-var-lib-kubelet\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.395405 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393824 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98b322c1-0c60-4b8d-b79b-32c529a63ccd-system-cni-dir\") pod \"multus-additional-cni-plugins-rvnph\" (UID: \"98b322c1-0c60-4b8d-b79b-32c529a63ccd\") " pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:04.395405 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393867 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-etc-sysctl-d\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.395405 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393908 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-run\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.395405 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393946 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c538774b-3450-4093-84a6-7cd9f9a0167e-etc-kubernetes\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.395405 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.393945 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/98b322c1-0c60-4b8d-b79b-32c529a63ccd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rvnph\" (UID: \"98b322c1-0c60-4b8d-b79b-32c529a63ccd\") " pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:04.395405 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.394007 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98b322c1-0c60-4b8d-b79b-32c529a63ccd-system-cni-dir\") pod \"multus-additional-cni-plugins-rvnph\" (UID: \"98b322c1-0c60-4b8d-b79b-32c529a63ccd\") " pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:04.395405 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.394069 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtz29\" (UniqueName: \"kubernetes.io/projected/98b322c1-0c60-4b8d-b79b-32c529a63ccd-kube-api-access-rtz29\") pod \"multus-additional-cni-plugins-rvnph\" (UID: \"98b322c1-0c60-4b8d-b79b-32c529a63ccd\") " pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:04.395405 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.394107 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/98b322c1-0c60-4b8d-b79b-32c529a63ccd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rvnph\" (UID: \"98b322c1-0c60-4b8d-b79b-32c529a63ccd\") " pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:04.395405 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.394360 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d59a6e2b-8dd8-45ff-be5f-01ad01f02124-serviceca\") pod \"node-ca-gmkxm\" (UID: \"d59a6e2b-8dd8-45ff-be5f-01ad01f02124\") " pod="openshift-image-registry/node-ca-gmkxm" Sep 17 07:28:04.395405 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.394455 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/98b322c1-0c60-4b8d-b79b-32c529a63ccd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rvnph\" (UID: \"98b322c1-0c60-4b8d-b79b-32c529a63ccd\") " pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:04.396415 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.396393 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/073304c8-0913-4ef9-8388-920b0f7dd353-agent-certs\") pod \"konnectivity-agent-v46tc\" (UID: \"073304c8-0913-4ef9-8388-920b0f7dd353\") " pod="kube-system/konnectivity-agent-v46tc" Sep 17 07:28:04.396622 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.396600 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c538774b-3450-4093-84a6-7cd9f9a0167e-tmp\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.396695 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.396622 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c538774b-3450-4093-84a6-7cd9f9a0167e-etc-tuned\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.402675 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.402622 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8kqz\" (UniqueName: \"kubernetes.io/projected/c327a988-59e8-48d2-a9fd-ff4668b66a22-kube-api-access-h8kqz\") pod \"node-resolver-dml6l\" (UID: \"c327a988-59e8-48d2-a9fd-ff4668b66a22\") " pod="openshift-dns/node-resolver-dml6l" Sep 17 07:28:04.403208 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.403185 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7w5b\" (UniqueName: \"kubernetes.io/projected/d59a6e2b-8dd8-45ff-be5f-01ad01f02124-kube-api-access-b7w5b\") pod \"node-ca-gmkxm\" (UID: \"d59a6e2b-8dd8-45ff-be5f-01ad01f02124\") " pod="openshift-image-registry/node-ca-gmkxm" Sep 17 07:28:04.403364 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.403342 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtz29\" (UniqueName: \"kubernetes.io/projected/98b322c1-0c60-4b8d-b79b-32c529a63ccd-kube-api-access-rtz29\") pod \"multus-additional-cni-plugins-rvnph\" (UID: \"98b322c1-0c60-4b8d-b79b-32c529a63ccd\") " pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:04.403529 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.403512 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxnfp\" (UniqueName: \"kubernetes.io/projected/c538774b-3450-4093-84a6-7cd9f9a0167e-kube-api-access-vxnfp\") pod \"tuned-xfn69\" (UID: \"c538774b-3450-4093-84a6-7cd9f9a0167e\") " pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.478135 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.478096 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" Sep 17 07:28:04.490743 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.490711 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pm5vm" Sep 17 07:28:04.500755 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.500714 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x7r9x" Sep 17 07:28:04.504786 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.504760 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:04.511602 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.511575 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dml6l" Sep 17 07:28:04.519410 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.519383 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v46tc" Sep 17 07:28:04.526072 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.526053 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gmkxm" Sep 17 07:28:04.532284 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.532261 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xfn69" Sep 17 07:28:04.536487 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.536469 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rvnph" Sep 17 07:28:04.633266 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.633227 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:28:04.796608 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.796511 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs\") pod \"network-metrics-daemon-zjz9v\" (UID: \"5eb544bb-f51e-4755-891e-ed3e8f031f0d\") " pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:04.796764 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:04.796668 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:04.796764 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:04.796739 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs podName:5eb544bb-f51e-4755-891e-ed3e8f031f0d nodeName:}" failed. No retries permitted until 2026-09-17 07:28:05.796724555 +0000 UTC m=+3.986700474 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs") pod "network-metrics-daemon-zjz9v" (UID: "5eb544bb-f51e-4755-891e-ed3e8f031f0d") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:04.897464 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.897426 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-422lx\" (UniqueName: \"kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx\") pod \"network-check-target-njpsg\" (UID: \"e7339f6e-91b9-4b00-b7aa-915f9ea79232\") " pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:04.897637 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:04.897476 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret\") pod \"global-pull-secret-syncer-mmcvp\" (UID: \"d4580c7f-1c6c-420d-962c-0ba1a6a222fa\") " pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:04.897637 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:04.897599 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:04.897730 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:04.897634 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:28:04.897730 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:04.897658 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret podName:d4580c7f-1c6c-420d-962c-0ba1a6a222fa nodeName:}" failed. No retries permitted until 2026-09-17 07:28:05.897641601 +0000 UTC m=+4.087617543 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret") pod "global-pull-secret-syncer-mmcvp" (UID: "d4580c7f-1c6c-420d-962c-0ba1a6a222fa") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:04.897730 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:04.897658 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:28:04.897730 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:04.897676 2579 projected.go:196] Error preparing data for projected volume kube-api-access-422lx for pod openshift-network-diagnostics/network-check-target-njpsg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:04.897893 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:04.897731 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx podName:e7339f6e-91b9-4b00-b7aa-915f9ea79232 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:05.89771584 +0000 UTC m=+4.087691760 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-422lx" (UniqueName: "kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx") pod "network-check-target-njpsg" (UID: "e7339f6e-91b9-4b00-b7aa-915f9ea79232") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:05.220303 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:05.220069 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fb6478a_4047_42e0_a66f_336860a6867c.slice/crio-01bdb7203b6a959a03f0aa1b0a4962321aa1b91dd4fcdb77c6874279c7799875 WatchSource:0}: Error finding container 01bdb7203b6a959a03f0aa1b0a4962321aa1b91dd4fcdb77c6874279c7799875: Status 404 returned error can't find the container with id 01bdb7203b6a959a03f0aa1b0a4962321aa1b91dd4fcdb77c6874279c7799875 Sep 17 07:28:05.224336 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:05.224310 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25122293_6ff9_4681_ac0f_3283e6de9a5d.slice/crio-205b8febd8fc1a2dc405caf581efd03d10280313d37e93a3184c70426b0503a3 WatchSource:0}: Error finding container 205b8febd8fc1a2dc405caf581efd03d10280313d37e93a3184c70426b0503a3: Status 404 returned error can't find the container with id 205b8febd8fc1a2dc405caf581efd03d10280313d37e93a3184c70426b0503a3 Sep 17 07:28:05.226541 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:05.226518 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42e9a882_f084_4752_af08_6911b3237ae2.slice/crio-5b8a5ca59181f60997911223f9753cbb9bb7807b0b68acba8061b71d177f2fc5 WatchSource:0}: Error finding container 5b8a5ca59181f60997911223f9753cbb9bb7807b0b68acba8061b71d177f2fc5: Status 404 returned error can't find the container with id 5b8a5ca59181f60997911223f9753cbb9bb7807b0b68acba8061b71d177f2fc5 Sep 17 07:28:05.242646 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:05.242610 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 07:23:04 +0000 UTC" deadline="2028-04-13 13:49:33.347046294 +0000 UTC" Sep 17 07:28:05.242646 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:05.242640 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13782h21m28.104410567s" Sep 17 07:28:05.248449 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:05.248424 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod073304c8_0913_4ef9_8388_920b0f7dd353.slice/crio-09b2bc590e3b027fd7664b4f51a2675d8b9c7e429de0fe0899f4edf484a67b26 WatchSource:0}: Error finding container 09b2bc590e3b027fd7664b4f51a2675d8b9c7e429de0fe0899f4edf484a67b26: Status 404 returned error can't find the container with id 09b2bc590e3b027fd7664b4f51a2675d8b9c7e429de0fe0899f4edf484a67b26 Sep 17 07:28:05.249128 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:05.249080 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c554023_f031_4773_b6ef_ef864215ba80.slice/crio-7c98fedbb05524b038eb739c84c2cf25ba326c9b2ab763493493564804afab57 WatchSource:0}: Error finding container 7c98fedbb05524b038eb739c84c2cf25ba326c9b2ab763493493564804afab57: Status 404 returned error can't find the container with id 7c98fedbb05524b038eb739c84c2cf25ba326c9b2ab763493493564804afab57 Sep 17 07:28:05.250554 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:05.250475 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd59a6e2b_8dd8_45ff_be5f_01ad01f02124.slice/crio-fd580385d4db430e1cfdf31a274eda818ed82c69565eb1d1f546d66796b16970 WatchSource:0}: Error finding container fd580385d4db430e1cfdf31a274eda818ed82c69565eb1d1f546d66796b16970: Status 404 returned error can't find the container with id fd580385d4db430e1cfdf31a274eda818ed82c69565eb1d1f546d66796b16970 Sep 17 07:28:05.305365 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:05.305329 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:05.305550 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:05.305468 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njpsg" podUID="e7339f6e-91b9-4b00-b7aa-915f9ea79232" Sep 17 07:28:05.315926 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:05.315890 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gmkxm" event={"ID":"d59a6e2b-8dd8-45ff-be5f-01ad01f02124","Type":"ContainerStarted","Data":"fd580385d4db430e1cfdf31a274eda818ed82c69565eb1d1f546d66796b16970"} Sep 17 07:28:05.317243 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:05.317210 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v46tc" event={"ID":"073304c8-0913-4ef9-8388-920b0f7dd353","Type":"ContainerStarted","Data":"09b2bc590e3b027fd7664b4f51a2675d8b9c7e429de0fe0899f4edf484a67b26"} Sep 17 07:28:05.318504 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:05.318456 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" event={"ID":"25122293-6ff9-4681-ac0f-3283e6de9a5d","Type":"ContainerStarted","Data":"205b8febd8fc1a2dc405caf581efd03d10280313d37e93a3184c70426b0503a3"} Sep 17 07:28:05.320185 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:05.320157 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x7r9x" event={"ID":"6fb6478a-4047-42e0-a66f-336860a6867c","Type":"ContainerStarted","Data":"01bdb7203b6a959a03f0aa1b0a4962321aa1b91dd4fcdb77c6874279c7799875"} Sep 17 07:28:05.321508 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:05.321475 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xfn69" event={"ID":"c538774b-3450-4093-84a6-7cd9f9a0167e","Type":"ContainerStarted","Data":"932ab7da5a35eaaa06e199bfd845c2687969e388ac5366aeb1205d7c3d9b641e"} Sep 17 07:28:05.322772 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:05.322747 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rvnph" event={"ID":"98b322c1-0c60-4b8d-b79b-32c529a63ccd","Type":"ContainerStarted","Data":"6b98e0297c4d1dbd30296b0a877ebd442797528dbaebc2b69c7f63fa83dcc4a0"} Sep 17 07:28:05.323865 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:05.323833 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dml6l" event={"ID":"c327a988-59e8-48d2-a9fd-ff4668b66a22","Type":"ContainerStarted","Data":"1aab084745a9a24293661f84abce8224a6fcd664d7bb84090b2bde1bde5bb9d8"} Sep 17 07:28:05.324909 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:05.324886 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pm5vm" event={"ID":"1c554023-f031-4773-b6ef-ef864215ba80","Type":"ContainerStarted","Data":"7c98fedbb05524b038eb739c84c2cf25ba326c9b2ab763493493564804afab57"} Sep 17 07:28:05.326177 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:05.326157 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" event={"ID":"42e9a882-f084-4752-af08-6911b3237ae2","Type":"ContainerStarted","Data":"5b8a5ca59181f60997911223f9753cbb9bb7807b0b68acba8061b71d177f2fc5"} Sep 17 07:28:05.804890 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:05.804557 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs\") pod \"network-metrics-daemon-zjz9v\" (UID: \"5eb544bb-f51e-4755-891e-ed3e8f031f0d\") " pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:05.804890 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:05.804831 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:05.805118 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:05.804902 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs podName:5eb544bb-f51e-4755-891e-ed3e8f031f0d nodeName:}" failed. No retries permitted until 2026-09-17 07:28:07.804881494 +0000 UTC m=+5.994857416 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs") pod "network-metrics-daemon-zjz9v" (UID: "5eb544bb-f51e-4755-891e-ed3e8f031f0d") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:05.905629 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:05.905589 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-422lx\" (UniqueName: \"kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx\") pod \"network-check-target-njpsg\" (UID: \"e7339f6e-91b9-4b00-b7aa-915f9ea79232\") " pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:05.905806 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:05.905653 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret\") pod \"global-pull-secret-syncer-mmcvp\" (UID: \"d4580c7f-1c6c-420d-962c-0ba1a6a222fa\") " pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:05.905806 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:05.905788 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:05.905927 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:05.905851 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret podName:d4580c7f-1c6c-420d-962c-0ba1a6a222fa nodeName:}" failed. No retries permitted until 2026-09-17 07:28:07.90583242 +0000 UTC m=+6.095808344 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret") pod "global-pull-secret-syncer-mmcvp" (UID: "d4580c7f-1c6c-420d-962c-0ba1a6a222fa") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:05.906423 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:05.906291 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:28:05.906423 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:05.906317 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:28:05.906423 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:05.906330 2579 projected.go:196] Error preparing data for projected volume kube-api-access-422lx for pod openshift-network-diagnostics/network-check-target-njpsg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:05.906423 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:05.906378 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx podName:e7339f6e-91b9-4b00-b7aa-915f9ea79232 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:07.906363848 +0000 UTC m=+6.096339773 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-422lx" (UniqueName: "kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx") pod "network-check-target-njpsg" (UID: "e7339f6e-91b9-4b00-b7aa-915f9ea79232") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:06.243516 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:06.243350 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 07:23:04 +0000 UTC" deadline="2028-02-16 09:44:07.805609757 +0000 UTC" Sep 17 07:28:06.243516 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:06.243415 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12410h16m1.562200704s" Sep 17 07:28:06.312594 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:06.312030 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:06.312594 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:06.312212 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:06.312594 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:06.312259 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mmcvp" podUID="d4580c7f-1c6c-420d-962c-0ba1a6a222fa" Sep 17 07:28:06.312594 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:06.312328 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjz9v" podUID="5eb544bb-f51e-4755-891e-ed3e8f031f0d" Sep 17 07:28:06.340597 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:06.340534 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal" event={"ID":"24d443373f3eb9148c2f63dcb77fe917","Type":"ContainerStarted","Data":"8809a8048442897e46e8ff76b37c4a606b54f7d8e359829b70a4c42caec5caf9"} Sep 17 07:28:06.359383 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:06.359344 2579 generic.go:356] "Generic (PLEG): container finished" podID="2b580332feb733df4d6e54bb54860651" containerID="dcc319502cac890728f7cd01eefdf79cd8c98878d0bd0f4f98ddaaa49bddb321" exitCode=0 Sep 17 07:28:06.359568 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:06.359406 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" event={"ID":"2b580332feb733df4d6e54bb54860651","Type":"ContainerDied","Data":"dcc319502cac890728f7cd01eefdf79cd8c98878d0bd0f4f98ddaaa49bddb321"} Sep 17 07:28:06.377906 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:06.377846 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal" podStartSLOduration=3.37782524 podStartE2EDuration="3.37782524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:28:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:06.357718033 +0000 UTC m=+4.547693977" watchObservedRunningTime="2026-09-17 07:28:06.37782524 +0000 UTC m=+4.567801183" Sep 17 07:28:07.306172 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:07.306131 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:07.306789 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:07.306266 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njpsg" podUID="e7339f6e-91b9-4b00-b7aa-915f9ea79232" Sep 17 07:28:07.374475 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:07.374438 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" event={"ID":"2b580332feb733df4d6e54bb54860651","Type":"ContainerStarted","Data":"e75b598c0616076be45fceda91ac3c7abb481e015e0738671279bdf76574b9c3"} Sep 17 07:28:07.398461 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:07.398388 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" podStartSLOduration=4.398363314 podStartE2EDuration="4.398363314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:28:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:07.397275718 +0000 UTC m=+5.587251661" watchObservedRunningTime="2026-09-17 07:28:07.398363314 +0000 UTC m=+5.588339257" Sep 17 07:28:07.822679 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:07.822636 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs\") pod \"network-metrics-daemon-zjz9v\" (UID: \"5eb544bb-f51e-4755-891e-ed3e8f031f0d\") " pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:07.822888 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:07.822784 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:07.822888 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:07.822852 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs podName:5eb544bb-f51e-4755-891e-ed3e8f031f0d nodeName:}" failed. No retries permitted until 2026-09-17 07:28:11.822832575 +0000 UTC m=+10.012808511 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs") pod "network-metrics-daemon-zjz9v" (UID: "5eb544bb-f51e-4755-891e-ed3e8f031f0d") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:07.923358 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:07.923319 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-422lx\" (UniqueName: \"kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx\") pod \"network-check-target-njpsg\" (UID: \"e7339f6e-91b9-4b00-b7aa-915f9ea79232\") " pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:07.923531 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:07.923406 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret\") pod \"global-pull-secret-syncer-mmcvp\" (UID: \"d4580c7f-1c6c-420d-962c-0ba1a6a222fa\") " pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:07.923607 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:07.923569 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:07.924285 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:07.923653 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret podName:d4580c7f-1c6c-420d-962c-0ba1a6a222fa nodeName:}" failed. No retries permitted until 2026-09-17 07:28:11.923635792 +0000 UTC m=+10.113611720 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret") pod "global-pull-secret-syncer-mmcvp" (UID: "d4580c7f-1c6c-420d-962c-0ba1a6a222fa") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:07.924285 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:07.924168 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:28:07.924285 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:07.924187 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:28:07.924285 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:07.924200 2579 projected.go:196] Error preparing data for projected volume kube-api-access-422lx for pod openshift-network-diagnostics/network-check-target-njpsg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:07.924285 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:07.924246 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx podName:e7339f6e-91b9-4b00-b7aa-915f9ea79232 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:11.924232037 +0000 UTC m=+10.114207962 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-422lx" (UniqueName: "kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx") pod "network-check-target-njpsg" (UID: "e7339f6e-91b9-4b00-b7aa-915f9ea79232") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:08.306336 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:08.306189 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:08.306336 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:08.306309 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mmcvp" podUID="d4580c7f-1c6c-420d-962c-0ba1a6a222fa" Sep 17 07:28:08.306916 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:08.306659 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:08.306916 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:08.306743 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjz9v" podUID="5eb544bb-f51e-4755-891e-ed3e8f031f0d" Sep 17 07:28:09.305779 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:09.305742 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:09.305966 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:09.305880 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njpsg" podUID="e7339f6e-91b9-4b00-b7aa-915f9ea79232" Sep 17 07:28:10.306199 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:10.305620 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:10.306199 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:10.305765 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjz9v" podUID="5eb544bb-f51e-4755-891e-ed3e8f031f0d" Sep 17 07:28:10.306199 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:10.306067 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:10.306199 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:10.306157 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mmcvp" podUID="d4580c7f-1c6c-420d-962c-0ba1a6a222fa" Sep 17 07:28:11.306043 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:11.305941 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:11.306208 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:11.306099 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njpsg" podUID="e7339f6e-91b9-4b00-b7aa-915f9ea79232" Sep 17 07:28:11.857634 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:11.857570 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs\") pod \"network-metrics-daemon-zjz9v\" (UID: \"5eb544bb-f51e-4755-891e-ed3e8f031f0d\") " pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:11.858478 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:11.857721 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:11.858478 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:11.857793 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs podName:5eb544bb-f51e-4755-891e-ed3e8f031f0d nodeName:}" failed. No retries permitted until 2026-09-17 07:28:19.857774904 +0000 UTC m=+18.047750847 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs") pod "network-metrics-daemon-zjz9v" (UID: "5eb544bb-f51e-4755-891e-ed3e8f031f0d") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:11.959423 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:11.958713 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret\") pod \"global-pull-secret-syncer-mmcvp\" (UID: \"d4580c7f-1c6c-420d-962c-0ba1a6a222fa\") " pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:11.959423 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:11.958808 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-422lx\" (UniqueName: \"kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx\") pod \"network-check-target-njpsg\" (UID: \"e7339f6e-91b9-4b00-b7aa-915f9ea79232\") " pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:11.959423 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:11.958968 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:28:11.959423 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:11.958988 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:28:11.959423 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:11.959018 2579 projected.go:196] Error preparing data for projected volume kube-api-access-422lx for pod openshift-network-diagnostics/network-check-target-njpsg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:11.959423 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:11.959082 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx podName:e7339f6e-91b9-4b00-b7aa-915f9ea79232 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:19.959062325 +0000 UTC m=+18.149038269 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-422lx" (UniqueName: "kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx") pod "network-check-target-njpsg" (UID: "e7339f6e-91b9-4b00-b7aa-915f9ea79232") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:11.959423 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:11.959197 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:11.959423 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:11.959269 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret podName:d4580c7f-1c6c-420d-962c-0ba1a6a222fa nodeName:}" failed. No retries permitted until 2026-09-17 07:28:19.95925108 +0000 UTC m=+18.149227003 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret") pod "global-pull-secret-syncer-mmcvp" (UID: "d4580c7f-1c6c-420d-962c-0ba1a6a222fa") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:12.310469 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:12.310379 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:12.310626 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:12.310503 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mmcvp" podUID="d4580c7f-1c6c-420d-962c-0ba1a6a222fa" Sep 17 07:28:12.310626 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:12.310540 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:12.310626 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:12.310613 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjz9v" podUID="5eb544bb-f51e-4755-891e-ed3e8f031f0d" Sep 17 07:28:13.305350 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:13.305318 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:13.305760 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:13.305446 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njpsg" podUID="e7339f6e-91b9-4b00-b7aa-915f9ea79232" Sep 17 07:28:14.305828 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:14.305789 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:14.306321 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:14.305917 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mmcvp" podUID="d4580c7f-1c6c-420d-962c-0ba1a6a222fa" Sep 17 07:28:14.306321 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:14.305975 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:14.306321 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:14.306139 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjz9v" podUID="5eb544bb-f51e-4755-891e-ed3e8f031f0d" Sep 17 07:28:15.305325 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:15.305287 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:15.305516 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:15.305419 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njpsg" podUID="e7339f6e-91b9-4b00-b7aa-915f9ea79232" Sep 17 07:28:16.308250 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:16.308181 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:16.308673 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:16.308305 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjz9v" podUID="5eb544bb-f51e-4755-891e-ed3e8f031f0d" Sep 17 07:28:16.308673 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:16.308359 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:16.308673 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:16.308471 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mmcvp" podUID="d4580c7f-1c6c-420d-962c-0ba1a6a222fa" Sep 17 07:28:17.305830 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:17.305794 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:17.306021 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:17.305917 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njpsg" podUID="e7339f6e-91b9-4b00-b7aa-915f9ea79232" Sep 17 07:28:18.305802 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:18.305761 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:18.306378 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:18.305809 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:18.306378 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:18.305934 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mmcvp" podUID="d4580c7f-1c6c-420d-962c-0ba1a6a222fa" Sep 17 07:28:18.306378 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:18.306104 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjz9v" podUID="5eb544bb-f51e-4755-891e-ed3e8f031f0d" Sep 17 07:28:19.306165 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:19.306135 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:19.306680 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:19.306251 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njpsg" podUID="e7339f6e-91b9-4b00-b7aa-915f9ea79232" Sep 17 07:28:19.915344 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:19.915307 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs\") pod \"network-metrics-daemon-zjz9v\" (UID: \"5eb544bb-f51e-4755-891e-ed3e8f031f0d\") " pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:19.915568 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:19.915436 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:19.915568 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:19.915498 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs podName:5eb544bb-f51e-4755-891e-ed3e8f031f0d nodeName:}" failed. No retries permitted until 2026-09-17 07:28:35.915482462 +0000 UTC m=+34.105458398 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs") pod "network-metrics-daemon-zjz9v" (UID: "5eb544bb-f51e-4755-891e-ed3e8f031f0d") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:20.016275 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:20.016230 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-422lx\" (UniqueName: \"kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx\") pod \"network-check-target-njpsg\" (UID: \"e7339f6e-91b9-4b00-b7aa-915f9ea79232\") " pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:20.016454 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:20.016288 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret\") pod \"global-pull-secret-syncer-mmcvp\" (UID: \"d4580c7f-1c6c-420d-962c-0ba1a6a222fa\") " pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:20.016454 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:20.016396 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:20.016454 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:20.016400 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:28:20.016454 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:20.016423 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:28:20.016454 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:20.016437 2579 projected.go:196] Error preparing data for projected volume kube-api-access-422lx for pod openshift-network-diagnostics/network-check-target-njpsg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:20.016454 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:20.016458 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret podName:d4580c7f-1c6c-420d-962c-0ba1a6a222fa nodeName:}" failed. No retries permitted until 2026-09-17 07:28:36.01643971 +0000 UTC m=+34.206415635 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret") pod "global-pull-secret-syncer-mmcvp" (UID: "d4580c7f-1c6c-420d-962c-0ba1a6a222fa") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:20.016707 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:20.016477 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx podName:e7339f6e-91b9-4b00-b7aa-915f9ea79232 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:36.016465777 +0000 UTC m=+34.206441718 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-422lx" (UniqueName: "kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx") pod "network-check-target-njpsg" (UID: "e7339f6e-91b9-4b00-b7aa-915f9ea79232") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:20.305794 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:20.305704 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:20.305961 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:20.305834 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjz9v" podUID="5eb544bb-f51e-4755-891e-ed3e8f031f0d" Sep 17 07:28:20.305961 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:20.305704 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:20.305961 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:20.305937 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mmcvp" podUID="d4580c7f-1c6c-420d-962c-0ba1a6a222fa" Sep 17 07:28:21.306132 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:21.306094 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:21.306599 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:21.306223 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njpsg" podUID="e7339f6e-91b9-4b00-b7aa-915f9ea79232" Sep 17 07:28:22.306777 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:22.306732 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:22.307282 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:22.306861 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjz9v" podUID="5eb544bb-f51e-4755-891e-ed3e8f031f0d" Sep 17 07:28:22.307282 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:22.306911 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:22.307282 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:22.307019 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mmcvp" podUID="d4580c7f-1c6c-420d-962c-0ba1a6a222fa" Sep 17 07:28:23.305600 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:23.305569 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:23.305734 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:23.305672 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njpsg" podUID="e7339f6e-91b9-4b00-b7aa-915f9ea79232" Sep 17 07:28:24.305956 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.305525 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:24.306452 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.305548 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:24.306452 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:24.306098 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mmcvp" podUID="d4580c7f-1c6c-420d-962c-0ba1a6a222fa" Sep 17 07:28:24.306452 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:24.306242 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjz9v" podUID="5eb544bb-f51e-4755-891e-ed3e8f031f0d" Sep 17 07:28:24.423351 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.423324 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xfn69" event={"ID":"c538774b-3450-4093-84a6-7cd9f9a0167e","Type":"ContainerStarted","Data":"ac7131638d5343837f2bcb40964abba18d942d026091a212ff74b8840a9cb834"} Sep 17 07:28:24.424657 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.424632 2579 generic.go:356] "Generic (PLEG): container finished" podID="98b322c1-0c60-4b8d-b79b-32c529a63ccd" containerID="3d610953e5ca9b5b852d240b9e61675093c2d1426f51aff9ce966333bafe2bbd" exitCode=0 Sep 17 07:28:24.424771 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.424701 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rvnph" event={"ID":"98b322c1-0c60-4b8d-b79b-32c529a63ccd","Type":"ContainerDied","Data":"3d610953e5ca9b5b852d240b9e61675093c2d1426f51aff9ce966333bafe2bbd"} Sep 17 07:28:24.426102 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.426075 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dml6l" event={"ID":"c327a988-59e8-48d2-a9fd-ff4668b66a22","Type":"ContainerStarted","Data":"bbe602c676e2ed781d22049b0162a93e99138928fae88ee3555ae0deb4569974"} Sep 17 07:28:24.429105 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.429084 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" event={"ID":"42e9a882-f084-4752-af08-6911b3237ae2","Type":"ContainerStarted","Data":"c167866f6cab5a7e2d96661666bd2bbd6ce07fa7d33aef7f6fde2f6a2111c9fb"} Sep 17 07:28:24.429198 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.429141 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" event={"ID":"42e9a882-f084-4752-af08-6911b3237ae2","Type":"ContainerStarted","Data":"0fe477b0786aa8e648b3bcd427f729030a3c9f3ba10eac8e7084eafc59fe2fff"} Sep 17 07:28:24.429198 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.429184 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" event={"ID":"42e9a882-f084-4752-af08-6911b3237ae2","Type":"ContainerStarted","Data":"c7c2e68ba64ba0f7ee346c9310b18aaff5d4cba0d650ffd75f9fd8ced118e744"} Sep 17 07:28:24.429292 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.429218 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" event={"ID":"42e9a882-f084-4752-af08-6911b3237ae2","Type":"ContainerStarted","Data":"7a60961415f896f9d46554b42404238e5eae09e7b7b77d8412d5c1511a382fd2"} Sep 17 07:28:24.429292 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.429248 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" event={"ID":"42e9a882-f084-4752-af08-6911b3237ae2","Type":"ContainerStarted","Data":"fe29a116042a8e97017d45dabd1831cfd997f1490cabe307dc73e4bdacbd2ddd"} Sep 17 07:28:24.429292 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.429279 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" event={"ID":"42e9a882-f084-4752-af08-6911b3237ae2","Type":"ContainerStarted","Data":"61a846a07c39463292fa92cc0a92ef25ef2d1ccf00e429284e7d226e791f33f7"} Sep 17 07:28:24.430437 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.430417 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gmkxm" event={"ID":"d59a6e2b-8dd8-45ff-be5f-01ad01f02124","Type":"ContainerStarted","Data":"14d1354f8b1b869c9469ef5d8e0556eaf3b602ac8468ab27b4d4b7e305ce55c8"} Sep 17 07:28:24.432647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.432164 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v46tc" event={"ID":"073304c8-0913-4ef9-8388-920b0f7dd353","Type":"ContainerStarted","Data":"b579c068aef82cb63bb80ca547d51bdad5be1ad359f45cb4f0eb4b648a8a97c1"} Sep 17 07:28:24.433716 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.433692 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" event={"ID":"25122293-6ff9-4681-ac0f-3283e6de9a5d","Type":"ContainerStarted","Data":"78dfe07a71db79b2a8dfe04005364ec2f7696804fd38492f1135e993dceac63e"} Sep 17 07:28:24.435024 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.434987 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x7r9x" event={"ID":"6fb6478a-4047-42e0-a66f-336860a6867c","Type":"ContainerStarted","Data":"c0869db88d863b30514d0e37c04bd57e6e67dc0d3097131519f2dcecd191eb22"} Sep 17 07:28:24.463627 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.463603 2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 17 07:28:24.472313 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.472258 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xfn69" podStartSLOduration=4.416818396 podStartE2EDuration="22.472238616s" totalImagesPullingTime="18.05542022s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:24.447891654 +0000 UTC m=+22.637867608" watchObservedRunningTime="2026-09-17 07:28:24.472238616 +0000 UTC m=+22.662214557" Sep 17 07:28:24.505639 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.505586 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dml6l" podStartSLOduration=4.449556825 podStartE2EDuration="22.505570883s" totalImagesPullingTime="18.056014058s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:24.472112051 +0000 UTC m=+22.662087994" watchObservedRunningTime="2026-09-17 07:28:24.505570883 +0000 UTC m=+22.695546887" Sep 17 07:28:24.562594 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.562544 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-v46tc" podStartSLOduration=12.024302014 podStartE2EDuration="22.562529546s" totalImagesPullingTime="10.538227532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:24.561686169 +0000 UTC m=+22.751662111" watchObservedRunningTime="2026-09-17 07:28:24.562529546 +0000 UTC m=+22.752505487" Sep 17 07:28:24.562757 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.562657 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-x7r9x" podStartSLOduration=4.45780726 podStartE2EDuration="22.562649226s" totalImagesPullingTime="18.104841966s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:24.538571206 +0000 UTC m=+22.728547148" watchObservedRunningTime="2026-09-17 07:28:24.562649226 +0000 UTC m=+22.752625169" Sep 17 07:28:24.591871 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:24.591831 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gmkxm" podStartSLOduration=4.529305679 podStartE2EDuration="22.591818061s" totalImagesPullingTime="18.062512382s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:24.591766791 +0000 UTC m=+22.781742733" watchObservedRunningTime="2026-09-17 07:28:24.591818061 +0000 UTC m=+22.781794002" Sep 17 07:28:25.250138 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:25.249986 2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T07:28:24.463622983Z","UUID":"e3adcf11-f0ef-4b27-8c09-cd2eb0e06e1e","Handler":null,"Name":"","Endpoint":""} Sep 17 07:28:25.251541 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:25.251509 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 17 07:28:25.251659 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:25.251552 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 17 07:28:25.306219 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:25.306186 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:25.306892 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:25.306289 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njpsg" podUID="e7339f6e-91b9-4b00-b7aa-915f9ea79232" Sep 17 07:28:25.439965 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:25.439934 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" event={"ID":"25122293-6ff9-4681-ac0f-3283e6de9a5d","Type":"ContainerStarted","Data":"f72b38d7050d310cc51c42ec0b1c3cd1d4580b55b4c6dde5a590805930dd2e75"} Sep 17 07:28:25.441693 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:25.441667 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pm5vm" event={"ID":"1c554023-f031-4773-b6ef-ef864215ba80","Type":"ContainerStarted","Data":"7330fdb2909432af10a9a7096d72b62ccaf6aff20c8039850f2c76c99dbad33f"} Sep 17 07:28:26.305875 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:26.305842 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:26.306071 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:26.305842 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:26.306071 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:26.306046 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mmcvp" podUID="d4580c7f-1c6c-420d-962c-0ba1a6a222fa" Sep 17 07:28:26.306198 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:26.306162 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjz9v" podUID="5eb544bb-f51e-4755-891e-ed3e8f031f0d" Sep 17 07:28:26.451109 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:26.451027 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" event={"ID":"42e9a882-f084-4752-af08-6911b3237ae2","Type":"ContainerStarted","Data":"7e6ef17c65be21d3b182ec98d0e806fc54196d5519e9b1e5ae5cd3cce680e291"} Sep 17 07:28:26.453485 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:26.453450 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" event={"ID":"25122293-6ff9-4681-ac0f-3283e6de9a5d","Type":"ContainerStarted","Data":"809cea8050ba492a6911a0a3584a6cea0cf246c465ec046693f262377bb52dc7"} Sep 17 07:28:26.474751 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:26.474700 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pm5vm" podStartSLOduration=6.412586113 podStartE2EDuration="24.474683533s" totalImagesPullingTime="18.06209742s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:25.477244976 +0000 UTC m=+23.667220921" watchObservedRunningTime="2026-09-17 07:28:26.474683533 +0000 UTC m=+24.664659474" Sep 17 07:28:26.475167 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:26.475131 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2bng" podStartSLOduration=4.7400675880000005 podStartE2EDuration="24.475119574s" totalImagesPullingTime="19.735051986s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:28:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:26.474417301 +0000 UTC m=+24.664393244" watchObservedRunningTime="2026-09-17 07:28:26.475119574 +0000 UTC m=+24.665095520" Sep 17 07:28:27.305419 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:27.305382 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:27.305587 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:27.305519 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njpsg" podUID="e7339f6e-91b9-4b00-b7aa-915f9ea79232" Sep 17 07:28:28.305840 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:28.305602 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:28.306339 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:28.305620 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:28.306339 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:28.305962 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjz9v" podUID="5eb544bb-f51e-4755-891e-ed3e8f031f0d" Sep 17 07:28:28.306339 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:28.306056 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mmcvp" podUID="d4580c7f-1c6c-420d-962c-0ba1a6a222fa" Sep 17 07:28:28.834420 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:28.834347 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-v46tc" Sep 17 07:28:28.834959 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:28.834943 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-v46tc" Sep 17 07:28:29.305642 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:29.305579 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:29.305763 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:29.305677 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njpsg" podUID="e7339f6e-91b9-4b00-b7aa-915f9ea79232" Sep 17 07:28:29.461271 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:29.461239 2579 generic.go:356] "Generic (PLEG): container finished" podID="98b322c1-0c60-4b8d-b79b-32c529a63ccd" containerID="64f3fc3458c6b57691de83078d835167003c47f5c6a1c7d220ae0ab633378ae9" exitCode=0 Sep 17 07:28:29.461733 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:29.461322 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rvnph" event={"ID":"98b322c1-0c60-4b8d-b79b-32c529a63ccd","Type":"ContainerDied","Data":"64f3fc3458c6b57691de83078d835167003c47f5c6a1c7d220ae0ab633378ae9"} Sep 17 07:28:29.465292 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:29.465257 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" event={"ID":"42e9a882-f084-4752-af08-6911b3237ae2","Type":"ContainerStarted","Data":"71cd25b846961494235dcdbfb64f8a94f5a2f62af85b1377a8109f67626dbc6b"} Sep 17 07:28:29.465513 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:29.465490 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-v46tc" Sep 17 07:28:29.466023 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:29.465989 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-v46tc" Sep 17 07:28:29.518474 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:29.518434 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" podStartSLOduration=9.137240387 podStartE2EDuration="27.518421785s" totalImagesPullingTime="18.381181398s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:29.517192292 +0000 UTC m=+27.707168234" watchObservedRunningTime="2026-09-17 07:28:29.518421785 +0000 UTC m=+27.708397726" Sep 17 07:28:30.305913 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:30.305760 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:30.306048 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:30.305828 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:30.306048 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:30.306025 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjz9v" podUID="5eb544bb-f51e-4755-891e-ed3e8f031f0d" Sep 17 07:28:30.306126 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:30.306100 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mmcvp" podUID="d4580c7f-1c6c-420d-962c-0ba1a6a222fa" Sep 17 07:28:30.358912 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:30.358886 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mmcvp"] Sep 17 07:28:30.362247 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:30.362227 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zjz9v"] Sep 17 07:28:30.363029 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:30.362982 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-njpsg"] Sep 17 07:28:30.363116 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:30.363101 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:30.363205 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:30.363175 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njpsg" podUID="e7339f6e-91b9-4b00-b7aa-915f9ea79232" Sep 17 07:28:30.469467 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:30.469386 2579 generic.go:356] "Generic (PLEG): container finished" podID="98b322c1-0c60-4b8d-b79b-32c529a63ccd" containerID="81519e2b08da574a093d5d04528807729dd34a715ff3a9f329768291d6effc13" exitCode=0 Sep 17 07:28:30.469467 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:30.469432 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rvnph" event={"ID":"98b322c1-0c60-4b8d-b79b-32c529a63ccd","Type":"ContainerDied","Data":"81519e2b08da574a093d5d04528807729dd34a715ff3a9f329768291d6effc13"} Sep 17 07:28:30.470220 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:30.469474 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:30.470220 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:30.469533 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:30.470220 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:30.469797 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mmcvp" podUID="d4580c7f-1c6c-420d-962c-0ba1a6a222fa" Sep 17 07:28:30.470220 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:30.470053 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjz9v" podUID="5eb544bb-f51e-4755-891e-ed3e8f031f0d" Sep 17 07:28:30.470778 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:30.470453 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:30.470778 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:30.470521 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:30.470778 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:30.470569 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:30.485226 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:30.485205 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:30.485334 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:30.485298 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:28:31.474163 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:31.474088 2579 generic.go:356] "Generic (PLEG): container finished" podID="98b322c1-0c60-4b8d-b79b-32c529a63ccd" containerID="30e0f11957833ba3f55a4346b97acc3ae6d180cd2e6de5f342ab6372ee6bbbd1" exitCode=0 Sep 17 07:28:31.474540 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:31.474179 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rvnph" event={"ID":"98b322c1-0c60-4b8d-b79b-32c529a63ccd","Type":"ContainerDied","Data":"30e0f11957833ba3f55a4346b97acc3ae6d180cd2e6de5f342ab6372ee6bbbd1"} Sep 17 07:28:32.307519 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:32.307480 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:32.307698 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:32.307614 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njpsg" podUID="e7339f6e-91b9-4b00-b7aa-915f9ea79232" Sep 17 07:28:32.307698 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:32.307686 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:32.307818 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:32.307799 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjz9v" podUID="5eb544bb-f51e-4755-891e-ed3e8f031f0d" Sep 17 07:28:32.308242 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:32.308221 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:32.308360 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:32.308343 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mmcvp" podUID="d4580c7f-1c6c-420d-962c-0ba1a6a222fa" Sep 17 07:28:34.305316 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:34.305279 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:34.305859 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:34.305409 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mmcvp" podUID="d4580c7f-1c6c-420d-962c-0ba1a6a222fa" Sep 17 07:28:34.305859 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:34.305450 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:34.305859 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:34.305566 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njpsg" podUID="e7339f6e-91b9-4b00-b7aa-915f9ea79232" Sep 17 07:28:34.305859 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:34.305603 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:34.305859 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:34.305713 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjz9v" podUID="5eb544bb-f51e-4755-891e-ed3e8f031f0d" Sep 17 07:28:35.932541 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:35.932513 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs\") pod \"network-metrics-daemon-zjz9v\" (UID: \"5eb544bb-f51e-4755-891e-ed3e8f031f0d\") " pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:35.932862 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:35.932626 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:35.932862 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:35.932679 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs podName:5eb544bb-f51e-4755-891e-ed3e8f031f0d nodeName:}" failed. No retries permitted until 2026-09-17 07:29:07.93266622 +0000 UTC m=+66.122642139 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs") pod "network-metrics-daemon-zjz9v" (UID: "5eb544bb-f51e-4755-891e-ed3e8f031f0d") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:36.033223 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.033196 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-422lx\" (UniqueName: \"kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx\") pod \"network-check-target-njpsg\" (UID: \"e7339f6e-91b9-4b00-b7aa-915f9ea79232\") " pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:36.033329 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.033237 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret\") pod \"global-pull-secret-syncer-mmcvp\" (UID: \"d4580c7f-1c6c-420d-962c-0ba1a6a222fa\") " pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:36.033396 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:36.033377 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:28:36.033435 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:36.033405 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:28:36.033435 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:36.033415 2579 projected.go:196] Error preparing data for projected volume kube-api-access-422lx for pod openshift-network-diagnostics/network-check-target-njpsg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:36.033505 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:36.033459 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx podName:e7339f6e-91b9-4b00-b7aa-915f9ea79232 nodeName:}" failed. No retries permitted until 2026-09-17 07:29:08.033447109 +0000 UTC m=+66.223423033 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-422lx" (UniqueName: "kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx") pod "network-check-target-njpsg" (UID: "e7339f6e-91b9-4b00-b7aa-915f9ea79232") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:36.033505 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:36.033381 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:36.033505 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:36.033502 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret podName:d4580c7f-1c6c-420d-962c-0ba1a6a222fa nodeName:}" failed. No retries permitted until 2026-09-17 07:29:08.033490372 +0000 UTC m=+66.223466291 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret") pod "global-pull-secret-syncer-mmcvp" (UID: "d4580c7f-1c6c-420d-962c-0ba1a6a222fa") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:36.054293 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.054274 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeReady" Sep 17 07:28:36.054404 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.054394 2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 17 07:28:36.159201 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.159174 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-z6l74"] Sep 17 07:28:36.173197 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.173178 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zcjf5"] Sep 17 07:28:36.173337 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.173315 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z6l74" Sep 17 07:28:36.177541 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.177485 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 17 07:28:36.177894 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.177802 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jn6qz\"" Sep 17 07:28:36.178133 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.178075 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 17 07:28:36.178342 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.178321 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 17 07:28:36.188326 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.188308 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z6l74"] Sep 17 07:28:36.188398 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.188393 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zcjf5" Sep 17 07:28:36.194135 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.194117 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zcjf5"] Sep 17 07:28:36.196308 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.196290 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 17 07:28:36.196394 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.196365 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2jzn2\"" Sep 17 07:28:36.197181 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.197164 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 17 07:28:36.234509 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.234488 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v557\" (UniqueName: \"kubernetes.io/projected/c01ae36f-26da-4056-a557-7d45a7cc54f5-kube-api-access-7v557\") pod \"ingress-canary-z6l74\" (UID: \"c01ae36f-26da-4056-a557-7d45a7cc54f5\") " pod="openshift-ingress-canary/ingress-canary-z6l74" Sep 17 07:28:36.234591 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.234520 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c01ae36f-26da-4056-a557-7d45a7cc54f5-cert\") pod \"ingress-canary-z6l74\" (UID: \"c01ae36f-26da-4056-a557-7d45a7cc54f5\") " pod="openshift-ingress-canary/ingress-canary-z6l74" Sep 17 07:28:36.305150 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.305124 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:28:36.305263 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.305130 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:28:36.305368 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.305130 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:28:36.316759 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.316743 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 07:28:36.316850 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.316829 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 17 07:28:36.317109 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.317070 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nzmmj\"" Sep 17 07:28:36.317109 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.317071 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-c888k\"" Sep 17 07:28:36.317792 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.317776 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 17 07:28:36.320777 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.320763 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 17 07:28:36.335483 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.335466 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1799de17-1341-4f16-9277-79056534a444-metrics-tls\") pod \"dns-default-zcjf5\" (UID: \"1799de17-1341-4f16-9277-79056534a444\") " pod="openshift-dns/dns-default-zcjf5" Sep 17 07:28:36.335560 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.335507 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v557\" (UniqueName: \"kubernetes.io/projected/c01ae36f-26da-4056-a557-7d45a7cc54f5-kube-api-access-7v557\") pod \"ingress-canary-z6l74\" (UID: \"c01ae36f-26da-4056-a557-7d45a7cc54f5\") " pod="openshift-ingress-canary/ingress-canary-z6l74" Sep 17 07:28:36.335560 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.335530 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c01ae36f-26da-4056-a557-7d45a7cc54f5-cert\") pod \"ingress-canary-z6l74\" (UID: \"c01ae36f-26da-4056-a557-7d45a7cc54f5\") " pod="openshift-ingress-canary/ingress-canary-z6l74" Sep 17 07:28:36.335560 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.335553 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1799de17-1341-4f16-9277-79056534a444-config-volume\") pod \"dns-default-zcjf5\" (UID: \"1799de17-1341-4f16-9277-79056534a444\") " pod="openshift-dns/dns-default-zcjf5" Sep 17 07:28:36.335660 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.335570 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll9kg\" (UniqueName: \"kubernetes.io/projected/1799de17-1341-4f16-9277-79056534a444-kube-api-access-ll9kg\") pod \"dns-default-zcjf5\" (UID: \"1799de17-1341-4f16-9277-79056534a444\") " pod="openshift-dns/dns-default-zcjf5" Sep 17 07:28:36.335660 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.335604 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1799de17-1341-4f16-9277-79056534a444-tmp-dir\") pod \"dns-default-zcjf5\" (UID: \"1799de17-1341-4f16-9277-79056534a444\") " pod="openshift-dns/dns-default-zcjf5" Sep 17 07:28:36.339397 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.339382 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c01ae36f-26da-4056-a557-7d45a7cc54f5-cert\") pod \"ingress-canary-z6l74\" (UID: \"c01ae36f-26da-4056-a557-7d45a7cc54f5\") " pod="openshift-ingress-canary/ingress-canary-z6l74" Sep 17 07:28:36.352363 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.352341 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v557\" (UniqueName: \"kubernetes.io/projected/c01ae36f-26da-4056-a557-7d45a7cc54f5-kube-api-access-7v557\") pod \"ingress-canary-z6l74\" (UID: \"c01ae36f-26da-4056-a557-7d45a7cc54f5\") " pod="openshift-ingress-canary/ingress-canary-z6l74" Sep 17 07:28:36.436586 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.436539 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1799de17-1341-4f16-9277-79056534a444-config-volume\") pod \"dns-default-zcjf5\" (UID: \"1799de17-1341-4f16-9277-79056534a444\") " pod="openshift-dns/dns-default-zcjf5" Sep 17 07:28:36.436586 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.436565 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll9kg\" (UniqueName: \"kubernetes.io/projected/1799de17-1341-4f16-9277-79056534a444-kube-api-access-ll9kg\") pod \"dns-default-zcjf5\" (UID: \"1799de17-1341-4f16-9277-79056534a444\") " pod="openshift-dns/dns-default-zcjf5" Sep 17 07:28:36.436710 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.436598 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1799de17-1341-4f16-9277-79056534a444-tmp-dir\") pod \"dns-default-zcjf5\" (UID: \"1799de17-1341-4f16-9277-79056534a444\") " pod="openshift-dns/dns-default-zcjf5" Sep 17 07:28:36.436710 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.436615 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1799de17-1341-4f16-9277-79056534a444-metrics-tls\") pod \"dns-default-zcjf5\" (UID: \"1799de17-1341-4f16-9277-79056534a444\") " pod="openshift-dns/dns-default-zcjf5" Sep 17 07:28:36.436780 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:36.436716 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:28:36.436812 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:36.436784 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1799de17-1341-4f16-9277-79056534a444-metrics-tls podName:1799de17-1341-4f16-9277-79056534a444 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:36.936765433 +0000 UTC m=+35.126741357 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1799de17-1341-4f16-9277-79056534a444-metrics-tls") pod "dns-default-zcjf5" (UID: "1799de17-1341-4f16-9277-79056534a444") : secret "dns-default-metrics-tls" not found Sep 17 07:28:36.436885 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.436868 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1799de17-1341-4f16-9277-79056534a444-tmp-dir\") pod \"dns-default-zcjf5\" (UID: \"1799de17-1341-4f16-9277-79056534a444\") " pod="openshift-dns/dns-default-zcjf5" Sep 17 07:28:36.437117 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.437098 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1799de17-1341-4f16-9277-79056534a444-config-volume\") pod \"dns-default-zcjf5\" (UID: \"1799de17-1341-4f16-9277-79056534a444\") " pod="openshift-dns/dns-default-zcjf5" Sep 17 07:28:36.454508 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.454484 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll9kg\" (UniqueName: \"kubernetes.io/projected/1799de17-1341-4f16-9277-79056534a444-kube-api-access-ll9kg\") pod \"dns-default-zcjf5\" (UID: \"1799de17-1341-4f16-9277-79056534a444\") " pod="openshift-dns/dns-default-zcjf5" Sep 17 07:28:36.489029 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.489003 2579 generic.go:356] "Generic (PLEG): container finished" podID="98b322c1-0c60-4b8d-b79b-32c529a63ccd" containerID="0055a448b518a9f358df66a480626238c6a07de2beb743788a8ed0263be36a72" exitCode=0 Sep 17 07:28:36.489121 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.489043 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rvnph" event={"ID":"98b322c1-0c60-4b8d-b79b-32c529a63ccd","Type":"ContainerDied","Data":"0055a448b518a9f358df66a480626238c6a07de2beb743788a8ed0263be36a72"} Sep 17 07:28:36.535424 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.535405 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z6l74" Sep 17 07:28:36.856559 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.856321 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z6l74"] Sep 17 07:28:36.861239 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:36.861216 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc01ae36f_26da_4056_a557_7d45a7cc54f5.slice/crio-bd40106d4f68ad5dbbb6d8ecaeb052b10747d9e6b0f0ed19826fe6273402d6f2 WatchSource:0}: Error finding container bd40106d4f68ad5dbbb6d8ecaeb052b10747d9e6b0f0ed19826fe6273402d6f2: Status 404 returned error can't find the container with id bd40106d4f68ad5dbbb6d8ecaeb052b10747d9e6b0f0ed19826fe6273402d6f2 Sep 17 07:28:36.941015 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:36.940963 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1799de17-1341-4f16-9277-79056534a444-metrics-tls\") pod \"dns-default-zcjf5\" (UID: \"1799de17-1341-4f16-9277-79056534a444\") " pod="openshift-dns/dns-default-zcjf5" Sep 17 07:28:36.941386 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:36.941100 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:28:36.941386 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:36.941153 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1799de17-1341-4f16-9277-79056534a444-metrics-tls podName:1799de17-1341-4f16-9277-79056534a444 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:37.941138763 +0000 UTC m=+36.131114688 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1799de17-1341-4f16-9277-79056534a444-metrics-tls") pod "dns-default-zcjf5" (UID: "1799de17-1341-4f16-9277-79056534a444") : secret "dns-default-metrics-tls" not found Sep 17 07:28:37.493935 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:37.493905 2579 generic.go:356] "Generic (PLEG): container finished" podID="98b322c1-0c60-4b8d-b79b-32c529a63ccd" containerID="cd928fd3863d27b8cac977d8283e94f22ec97664c88c41e101309841cc0d8f07" exitCode=0 Sep 17 07:28:37.494108 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:37.493984 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rvnph" event={"ID":"98b322c1-0c60-4b8d-b79b-32c529a63ccd","Type":"ContainerDied","Data":"cd928fd3863d27b8cac977d8283e94f22ec97664c88c41e101309841cc0d8f07"} Sep 17 07:28:37.495175 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:37.495150 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z6l74" event={"ID":"c01ae36f-26da-4056-a557-7d45a7cc54f5","Type":"ContainerStarted","Data":"bd40106d4f68ad5dbbb6d8ecaeb052b10747d9e6b0f0ed19826fe6273402d6f2"} Sep 17 07:28:37.948768 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:37.948688 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1799de17-1341-4f16-9277-79056534a444-metrics-tls\") pod \"dns-default-zcjf5\" (UID: \"1799de17-1341-4f16-9277-79056534a444\") " pod="openshift-dns/dns-default-zcjf5" Sep 17 07:28:37.949314 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:37.948834 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:28:37.949314 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:37.948896 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1799de17-1341-4f16-9277-79056534a444-metrics-tls podName:1799de17-1341-4f16-9277-79056534a444 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:39.948877284 +0000 UTC m=+38.138853208 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1799de17-1341-4f16-9277-79056534a444-metrics-tls") pod "dns-default-zcjf5" (UID: "1799de17-1341-4f16-9277-79056534a444") : secret "dns-default-metrics-tls" not found Sep 17 07:28:38.500330 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:38.500292 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rvnph" event={"ID":"98b322c1-0c60-4b8d-b79b-32c529a63ccd","Type":"ContainerStarted","Data":"603f0286c694581cfc48e20e3e412a05a08e6c7c545aee3e1cc1763ecb21965d"} Sep 17 07:28:38.533980 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:38.533932 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rvnph" podStartSLOduration=8.552747335 podStartE2EDuration="36.533916615s" totalImagesPullingTime="27.98116928s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:28:02 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:38.532975742 +0000 UTC m=+36.722951685" watchObservedRunningTime="2026-09-17 07:28:38.533916615 +0000 UTC m=+36.723892556" Sep 17 07:28:39.964743 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:39.964699 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1799de17-1341-4f16-9277-79056534a444-metrics-tls\") pod \"dns-default-zcjf5\" (UID: \"1799de17-1341-4f16-9277-79056534a444\") " pod="openshift-dns/dns-default-zcjf5" Sep 17 07:28:39.965408 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:39.964831 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:28:39.965408 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:39.964898 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1799de17-1341-4f16-9277-79056534a444-metrics-tls podName:1799de17-1341-4f16-9277-79056534a444 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:43.964876834 +0000 UTC m=+42.154852754 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1799de17-1341-4f16-9277-79056534a444-metrics-tls") pod "dns-default-zcjf5" (UID: "1799de17-1341-4f16-9277-79056534a444") : secret "dns-default-metrics-tls" not found Sep 17 07:28:40.506703 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:40.506662 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z6l74" event={"ID":"c01ae36f-26da-4056-a557-7d45a7cc54f5","Type":"ContainerStarted","Data":"ab92c2216d1a0af1c44fad21bd0cc6583d390a03f4da5a067478a2049780cc14"} Sep 17 07:28:40.526491 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:40.526446 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-z6l74" podStartSLOduration=1.860916593 podStartE2EDuration="4.526432951s" totalImagesPullingTime="2.665516358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:28:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:40.525457016 +0000 UTC m=+38.715432958" watchObservedRunningTime="2026-09-17 07:28:40.526432951 +0000 UTC m=+38.716408884" Sep 17 07:28:42.820498 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:42.819305 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qmmms"] Sep 17 07:28:42.854173 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:42.854145 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qmmms"] Sep 17 07:28:42.854309 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:42.854234 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:42.856652 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:42.856623 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 17 07:28:42.856782 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:42.856654 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 17 07:28:42.856782 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:42.856763 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 17 07:28:42.856903 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:42.856787 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4qrls\"" Sep 17 07:28:42.856903 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:42.856848 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 17 07:28:42.985213 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:42.985176 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1187c234-335b-4635-8bee-126377c0b290-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:42.985377 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:42.985224 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1187c234-335b-4635-8bee-126377c0b290-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:42.985377 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:42.985281 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfzz7\" (UniqueName: \"kubernetes.io/projected/1187c234-335b-4635-8bee-126377c0b290-kube-api-access-zfzz7\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:42.985377 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:42.985340 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1187c234-335b-4635-8bee-126377c0b290-crio-socket\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:42.985377 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:42.985362 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1187c234-335b-4635-8bee-126377c0b290-data-volume\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:43.086299 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:43.086244 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1187c234-335b-4635-8bee-126377c0b290-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:43.086299 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:43.086282 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1187c234-335b-4635-8bee-126377c0b290-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:43.086445 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:43.086305 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfzz7\" (UniqueName: \"kubernetes.io/projected/1187c234-335b-4635-8bee-126377c0b290-kube-api-access-zfzz7\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:43.086445 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:43.086356 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1187c234-335b-4635-8bee-126377c0b290-crio-socket\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:43.086445 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:43.086371 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1187c234-335b-4635-8bee-126377c0b290-data-volume\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:43.086445 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:43.086382 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 07:28:43.086445 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:43.086399 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:28:43.086624 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:43.086453 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1187c234-335b-4635-8bee-126377c0b290-insights-runtime-extractor-tls podName:1187c234-335b-4635-8bee-126377c0b290 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:43.58643628 +0000 UTC m=+41.776412219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1187c234-335b-4635-8bee-126377c0b290-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qmmms" (UID: "1187c234-335b-4635-8bee-126377c0b290") : secret "insights-runtime-extractor-tls" not found Sep 17 07:28:43.086624 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:43.086475 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1187c234-335b-4635-8bee-126377c0b290-kube-rbac-proxy-cm podName:1187c234-335b-4635-8bee-126377c0b290 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:43.586468177 +0000 UTC m=+41.776444096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1187c234-335b-4635-8bee-126377c0b290-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qmmms" (UID: "1187c234-335b-4635-8bee-126377c0b290") : configmap "kube-rbac-proxy" not found Sep 17 07:28:43.086624 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:43.086595 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1187c234-335b-4635-8bee-126377c0b290-crio-socket\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:43.086765 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:43.086703 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1187c234-335b-4635-8bee-126377c0b290-data-volume\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:43.098329 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:43.098306 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfzz7\" (UniqueName: \"kubernetes.io/projected/1187c234-335b-4635-8bee-126377c0b290-kube-api-access-zfzz7\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:43.590110 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:43.590080 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1187c234-335b-4635-8bee-126377c0b290-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:43.590240 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:43.590116 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1187c234-335b-4635-8bee-126377c0b290-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:43.590240 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:43.590187 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 07:28:43.590240 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:43.590235 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1187c234-335b-4635-8bee-126377c0b290-kube-rbac-proxy-cm podName:1187c234-335b-4635-8bee-126377c0b290 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:44.590219182 +0000 UTC m=+42.780195102 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1187c234-335b-4635-8bee-126377c0b290-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qmmms" (UID: "1187c234-335b-4635-8bee-126377c0b290") : configmap "kube-rbac-proxy" not found Sep 17 07:28:43.590355 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:43.590280 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:28:43.590355 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:43.590328 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1187c234-335b-4635-8bee-126377c0b290-insights-runtime-extractor-tls podName:1187c234-335b-4635-8bee-126377c0b290 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:44.590315522 +0000 UTC m=+42.780291458 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1187c234-335b-4635-8bee-126377c0b290-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qmmms" (UID: "1187c234-335b-4635-8bee-126377c0b290") : secret "insights-runtime-extractor-tls" not found Sep 17 07:28:43.978690 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:43.978616 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-gf8zr"] Sep 17 07:28:43.992089 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:43.992060 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1799de17-1341-4f16-9277-79056534a444-metrics-tls\") pod \"dns-default-zcjf5\" (UID: \"1799de17-1341-4f16-9277-79056534a444\") " pod="openshift-dns/dns-default-zcjf5" Sep 17 07:28:43.992219 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:43.992200 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:28:43.992271 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:43.992262 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1799de17-1341-4f16-9277-79056534a444-metrics-tls podName:1799de17-1341-4f16-9277-79056534a444 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:51.992248088 +0000 UTC m=+50.182224008 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1799de17-1341-4f16-9277-79056534a444-metrics-tls") pod "dns-default-zcjf5" (UID: "1799de17-1341-4f16-9277-79056534a444") : secret "dns-default-metrics-tls" not found Sep 17 07:28:44.000922 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:44.000895 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-gf8zr"] Sep 17 07:28:44.001048 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:44.001015 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-gf8zr" Sep 17 07:28:44.005061 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:44.005040 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 17 07:28:44.005982 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:44.005959 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 17 07:28:44.006082 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:44.005962 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-msqwg\"" Sep 17 07:28:44.092917 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:44.092883 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5pgs\" (UniqueName: \"kubernetes.io/projected/ca74784d-18e9-487d-bc82-fa648083f4b3-kube-api-access-r5pgs\") pod \"migrator-8b4d697c5-gf8zr\" (UID: \"ca74784d-18e9-487d-bc82-fa648083f4b3\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-gf8zr" Sep 17 07:28:44.193382 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:44.193353 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5pgs\" (UniqueName: \"kubernetes.io/projected/ca74784d-18e9-487d-bc82-fa648083f4b3-kube-api-access-r5pgs\") pod \"migrator-8b4d697c5-gf8zr\" (UID: \"ca74784d-18e9-487d-bc82-fa648083f4b3\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-gf8zr" Sep 17 07:28:44.201562 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:44.201526 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5pgs\" (UniqueName: \"kubernetes.io/projected/ca74784d-18e9-487d-bc82-fa648083f4b3-kube-api-access-r5pgs\") pod \"migrator-8b4d697c5-gf8zr\" (UID: \"ca74784d-18e9-487d-bc82-fa648083f4b3\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-gf8zr" Sep 17 07:28:44.309086 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:44.309063 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-gf8zr" Sep 17 07:28:44.596775 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:44.596688 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1187c234-335b-4635-8bee-126377c0b290-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:44.596775 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:44.596738 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1187c234-335b-4635-8bee-126377c0b290-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:44.596954 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:44.596827 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 07:28:44.596954 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:44.596855 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:28:44.596954 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:44.596889 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1187c234-335b-4635-8bee-126377c0b290-kube-rbac-proxy-cm podName:1187c234-335b-4635-8bee-126377c0b290 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:46.596871488 +0000 UTC m=+44.786847413 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1187c234-335b-4635-8bee-126377c0b290-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qmmms" (UID: "1187c234-335b-4635-8bee-126377c0b290") : configmap "kube-rbac-proxy" not found Sep 17 07:28:44.596954 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:44.596903 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1187c234-335b-4635-8bee-126377c0b290-insights-runtime-extractor-tls podName:1187c234-335b-4635-8bee-126377c0b290 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:46.596897411 +0000 UTC m=+44.786873332 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1187c234-335b-4635-8bee-126377c0b290-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qmmms" (UID: "1187c234-335b-4635-8bee-126377c0b290") : secret "insights-runtime-extractor-tls" not found Sep 17 07:28:44.597404 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:44.597384 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-gf8zr"] Sep 17 07:28:44.600542 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:44.600523 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca74784d_18e9_487d_bc82_fa648083f4b3.slice/crio-9c4278015722ae27b819f018fd51abae2fdc9480331d9dbe6b2b7c896738d6cf WatchSource:0}: Error finding container 9c4278015722ae27b819f018fd51abae2fdc9480331d9dbe6b2b7c896738d6cf: Status 404 returned error can't find the container with id 9c4278015722ae27b819f018fd51abae2fdc9480331d9dbe6b2b7c896738d6cf Sep 17 07:28:44.758749 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:44.758726 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dml6l_c327a988-59e8-48d2-a9fd-ff4668b66a22/dns-node-resolver/0.log" Sep 17 07:28:45.519832 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:45.519797 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-gf8zr" event={"ID":"ca74784d-18e9-487d-bc82-fa648083f4b3","Type":"ContainerStarted","Data":"9c4278015722ae27b819f018fd51abae2fdc9480331d9dbe6b2b7c896738d6cf"} Sep 17 07:28:45.759591 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:45.759562 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gmkxm_d59a6e2b-8dd8-45ff-be5f-01ad01f02124/node-ca/0.log" Sep 17 07:28:46.524268 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:46.524234 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-gf8zr" event={"ID":"ca74784d-18e9-487d-bc82-fa648083f4b3","Type":"ContainerStarted","Data":"a3a5c95430fc86d8df234d834a0ff0993711bd5a6fb439bc90ab22555eb66d9d"} Sep 17 07:28:46.524614 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:46.524296 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-gf8zr" event={"ID":"ca74784d-18e9-487d-bc82-fa648083f4b3","Type":"ContainerStarted","Data":"f673a19e4c69f372ea53e64a65ef8698a7d2a97d0cd408044b59f08889125592"} Sep 17 07:28:46.541615 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:46.541574 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-gf8zr" podStartSLOduration=2.030300823 podStartE2EDuration="3.541561548s" totalImagesPullingTime="1.511260725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:28:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:46.541163163 +0000 UTC m=+44.731139118" watchObservedRunningTime="2026-09-17 07:28:46.541561548 +0000 UTC m=+44.731537500" Sep 17 07:28:46.609402 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:46.609376 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1187c234-335b-4635-8bee-126377c0b290-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:46.609502 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:46.609413 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1187c234-335b-4635-8bee-126377c0b290-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:46.609551 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:46.609496 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 07:28:46.609551 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:46.609519 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:28:46.609616 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:46.609558 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1187c234-335b-4635-8bee-126377c0b290-kube-rbac-proxy-cm podName:1187c234-335b-4635-8bee-126377c0b290 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:50.609545225 +0000 UTC m=+48.799521145 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1187c234-335b-4635-8bee-126377c0b290-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qmmms" (UID: "1187c234-335b-4635-8bee-126377c0b290") : configmap "kube-rbac-proxy" not found Sep 17 07:28:46.609616 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:46.609581 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1187c234-335b-4635-8bee-126377c0b290-insights-runtime-extractor-tls podName:1187c234-335b-4635-8bee-126377c0b290 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:50.609564563 +0000 UTC m=+48.799540497 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1187c234-335b-4635-8bee-126377c0b290-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qmmms" (UID: "1187c234-335b-4635-8bee-126377c0b290") : secret "insights-runtime-extractor-tls" not found Sep 17 07:28:46.759615 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:46.759591 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z6l74_c01ae36f-26da-4056-a557-7d45a7cc54f5/serve-healthcheck-canary/0.log" Sep 17 07:28:50.638288 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:50.638255 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1187c234-335b-4635-8bee-126377c0b290-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:50.638699 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:50.638302 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1187c234-335b-4635-8bee-126377c0b290-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:50.638699 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:50.638393 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:28:50.638699 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:50.638452 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1187c234-335b-4635-8bee-126377c0b290-insights-runtime-extractor-tls podName:1187c234-335b-4635-8bee-126377c0b290 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:58.638436008 +0000 UTC m=+56.828411927 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1187c234-335b-4635-8bee-126377c0b290-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qmmms" (UID: "1187c234-335b-4635-8bee-126377c0b290") : secret "insights-runtime-extractor-tls" not found Sep 17 07:28:50.638873 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:50.638852 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1187c234-335b-4635-8bee-126377c0b290-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:52.048945 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:52.048906 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1799de17-1341-4f16-9277-79056534a444-metrics-tls\") pod \"dns-default-zcjf5\" (UID: \"1799de17-1341-4f16-9277-79056534a444\") " pod="openshift-dns/dns-default-zcjf5" Sep 17 07:28:52.049338 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:52.049047 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:28:52.049338 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:28:52.049105 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1799de17-1341-4f16-9277-79056534a444-metrics-tls podName:1799de17-1341-4f16-9277-79056534a444 nodeName:}" failed. No retries permitted until 2026-09-17 07:29:08.049088424 +0000 UTC m=+66.239064349 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1799de17-1341-4f16-9277-79056534a444-metrics-tls") pod "dns-default-zcjf5" (UID: "1799de17-1341-4f16-9277-79056534a444") : secret "dns-default-metrics-tls" not found Sep 17 07:28:58.701282 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:58.701246 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1187c234-335b-4635-8bee-126377c0b290-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:58.703444 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:58.703418 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1187c234-335b-4635-8bee-126377c0b290-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmmms\" (UID: \"1187c234-335b-4635-8bee-126377c0b290\") " pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:58.762673 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:58.762650 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qmmms" Sep 17 07:28:59.063100 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:59.063027 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qmmms"] Sep 17 07:28:59.065155 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:28:59.065127 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1187c234_335b_4635_8bee_126377c0b290.slice/crio-063148a2bda58fb998e7ed099bb1f27f334b4555618d4b69381db644a6e503fe WatchSource:0}: Error finding container 063148a2bda58fb998e7ed099bb1f27f334b4555618d4b69381db644a6e503fe: Status 404 returned error can't find the container with id 063148a2bda58fb998e7ed099bb1f27f334b4555618d4b69381db644a6e503fe Sep 17 07:28:59.556948 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:59.556912 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qmmms" event={"ID":"1187c234-335b-4635-8bee-126377c0b290","Type":"ContainerStarted","Data":"af713001df63079d65fffa2b6d27c08fd49ba5cf7393d25fc5e2cf1a0776808a"} Sep 17 07:28:59.557118 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:28:59.556970 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qmmms" event={"ID":"1187c234-335b-4635-8bee-126377c0b290","Type":"ContainerStarted","Data":"063148a2bda58fb998e7ed099bb1f27f334b4555618d4b69381db644a6e503fe"} Sep 17 07:29:00.561053 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:00.561017 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qmmms" event={"ID":"1187c234-335b-4635-8bee-126377c0b290","Type":"ContainerStarted","Data":"c3506308d7198790b1b3ac0de544370c659cf9d617e60194b91a74da2cc67fb4"} Sep 17 07:29:02.487419 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:02.487389 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nsjvb" Sep 17 07:29:02.568487 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:02.568452 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qmmms" event={"ID":"1187c234-335b-4635-8bee-126377c0b290","Type":"ContainerStarted","Data":"195048d137f2bfec7b7ed88461bb0e91e35c0ec9ed39cce9d91d45c04bab4fc7"} Sep 17 07:29:06.594797 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.594739 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qmmms" podStartSLOduration=22.104226342 podStartE2EDuration="24.594723232s" totalImagesPullingTime="2.49049689s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:28:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:02.590564321 +0000 UTC m=+60.780540262" watchObservedRunningTime="2026-09-17 07:29:06.594723232 +0000 UTC m=+64.784699174" Sep 17 07:29:06.595365 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.595336 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-k74qf"] Sep 17 07:29:06.603203 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.603180 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c45796f64-z4sln"] Sep 17 07:29:06.603338 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.603301 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-k74qf" Sep 17 07:29:06.608867 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.608844 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c45796f64-z4sln" Sep 17 07:29:06.628924 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.628879 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-k74qf"] Sep 17 07:29:06.629632 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.629608 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c45796f64-z4sln"] Sep 17 07:29:06.636238 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.636209 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 17 07:29:06.636828 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.636812 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 17 07:29:06.637162 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.637141 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-4qjkt\"" Sep 17 07:29:06.637278 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.637173 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-mzls8\"" Sep 17 07:29:06.637278 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.637225 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 17 07:29:06.637278 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.637268 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 17 07:29:06.637460 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.637291 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 17 07:29:06.644727 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.644708 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 17 07:29:06.645527 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.645508 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5"] Sep 17 07:29:06.666153 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.666122 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" Sep 17 07:29:06.670245 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.670225 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Sep 17 07:29:06.670371 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.670225 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Sep 17 07:29:06.671381 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.671360 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Sep 17 07:29:06.671507 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.671435 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Sep 17 07:29:06.696977 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.696949 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5"] Sep 17 07:29:06.761493 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.761459 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fd1ed8f0-f91c-4789-915c-4f5dc7af25f6-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-k74qf\" (UID: \"fd1ed8f0-f91c-4789-915c-4f5dc7af25f6\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-k74qf" Sep 17 07:29:06.761687 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.761509 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fd1ed8f0-f91c-4789-915c-4f5dc7af25f6-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-k74qf\" (UID: \"fd1ed8f0-f91c-4789-915c-4f5dc7af25f6\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-k74qf" Sep 17 07:29:06.761687 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.761569 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwcfp\" (UniqueName: \"kubernetes.io/projected/865a7bee-7634-4062-9f9a-65cde81f4cf3-kube-api-access-rwcfp\") pod \"managed-serviceaccount-addon-agent-5c45796f64-z4sln\" (UID: \"865a7bee-7634-4062-9f9a-65cde81f4cf3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c45796f64-z4sln" Sep 17 07:29:06.761687 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.761602 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/865a7bee-7634-4062-9f9a-65cde81f4cf3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c45796f64-z4sln\" (UID: \"865a7bee-7634-4062-9f9a-65cde81f4cf3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c45796f64-z4sln" Sep 17 07:29:06.862391 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.862302 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/faee3b32-feb1-46f7-89a2-e618fe59082c-hub\") pod \"cluster-proxy-proxy-agent-845f75f54f-t77z5\" (UID: \"faee3b32-feb1-46f7-89a2-e618fe59082c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" Sep 17 07:29:06.862391 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.862340 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/faee3b32-feb1-46f7-89a2-e618fe59082c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-845f75f54f-t77z5\" (UID: \"faee3b32-feb1-46f7-89a2-e618fe59082c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" Sep 17 07:29:06.862391 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.862359 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fd1ed8f0-f91c-4789-915c-4f5dc7af25f6-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-k74qf\" (UID: \"fd1ed8f0-f91c-4789-915c-4f5dc7af25f6\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-k74qf" Sep 17 07:29:06.862629 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.862394 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fd1ed8f0-f91c-4789-915c-4f5dc7af25f6-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-k74qf\" (UID: \"fd1ed8f0-f91c-4789-915c-4f5dc7af25f6\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-k74qf" Sep 17 07:29:06.862629 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.862437 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/faee3b32-feb1-46f7-89a2-e618fe59082c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-845f75f54f-t77z5\" (UID: \"faee3b32-feb1-46f7-89a2-e618fe59082c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" Sep 17 07:29:06.862629 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.862546 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6749\" (UniqueName: \"kubernetes.io/projected/faee3b32-feb1-46f7-89a2-e618fe59082c-kube-api-access-d6749\") pod \"cluster-proxy-proxy-agent-845f75f54f-t77z5\" (UID: \"faee3b32-feb1-46f7-89a2-e618fe59082c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" Sep 17 07:29:06.862629 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.862595 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/faee3b32-feb1-46f7-89a2-e618fe59082c-ca\") pod \"cluster-proxy-proxy-agent-845f75f54f-t77z5\" (UID: \"faee3b32-feb1-46f7-89a2-e618fe59082c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" Sep 17 07:29:06.862629 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.862619 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwcfp\" (UniqueName: \"kubernetes.io/projected/865a7bee-7634-4062-9f9a-65cde81f4cf3-kube-api-access-rwcfp\") pod \"managed-serviceaccount-addon-agent-5c45796f64-z4sln\" (UID: \"865a7bee-7634-4062-9f9a-65cde81f4cf3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c45796f64-z4sln" Sep 17 07:29:06.862805 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.862660 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/865a7bee-7634-4062-9f9a-65cde81f4cf3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c45796f64-z4sln\" (UID: \"865a7bee-7634-4062-9f9a-65cde81f4cf3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c45796f64-z4sln" Sep 17 07:29:06.862805 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.862686 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/faee3b32-feb1-46f7-89a2-e618fe59082c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-845f75f54f-t77z5\" (UID: \"faee3b32-feb1-46f7-89a2-e618fe59082c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" Sep 17 07:29:06.863140 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.863119 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fd1ed8f0-f91c-4789-915c-4f5dc7af25f6-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-k74qf\" (UID: \"fd1ed8f0-f91c-4789-915c-4f5dc7af25f6\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-k74qf" Sep 17 07:29:06.864865 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.864844 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fd1ed8f0-f91c-4789-915c-4f5dc7af25f6-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-k74qf\" (UID: \"fd1ed8f0-f91c-4789-915c-4f5dc7af25f6\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-k74qf" Sep 17 07:29:06.865080 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.865063 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/865a7bee-7634-4062-9f9a-65cde81f4cf3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c45796f64-z4sln\" (UID: \"865a7bee-7634-4062-9f9a-65cde81f4cf3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c45796f64-z4sln" Sep 17 07:29:06.881951 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.881928 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwcfp\" (UniqueName: \"kubernetes.io/projected/865a7bee-7634-4062-9f9a-65cde81f4cf3-kube-api-access-rwcfp\") pod \"managed-serviceaccount-addon-agent-5c45796f64-z4sln\" (UID: \"865a7bee-7634-4062-9f9a-65cde81f4cf3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c45796f64-z4sln" Sep 17 07:29:06.911959 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.911913 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-k74qf" Sep 17 07:29:06.933497 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.933470 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c45796f64-z4sln" Sep 17 07:29:06.964923 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.964887 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/faee3b32-feb1-46f7-89a2-e618fe59082c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-845f75f54f-t77z5\" (UID: \"faee3b32-feb1-46f7-89a2-e618fe59082c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" Sep 17 07:29:06.965078 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.964982 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/faee3b32-feb1-46f7-89a2-e618fe59082c-hub\") pod \"cluster-proxy-proxy-agent-845f75f54f-t77z5\" (UID: \"faee3b32-feb1-46f7-89a2-e618fe59082c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" Sep 17 07:29:06.965078 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.965028 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/faee3b32-feb1-46f7-89a2-e618fe59082c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-845f75f54f-t77z5\" (UID: \"faee3b32-feb1-46f7-89a2-e618fe59082c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" Sep 17 07:29:06.965222 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.965085 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/faee3b32-feb1-46f7-89a2-e618fe59082c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-845f75f54f-t77z5\" (UID: \"faee3b32-feb1-46f7-89a2-e618fe59082c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" Sep 17 07:29:06.965222 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.965118 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6749\" (UniqueName: \"kubernetes.io/projected/faee3b32-feb1-46f7-89a2-e618fe59082c-kube-api-access-d6749\") pod \"cluster-proxy-proxy-agent-845f75f54f-t77z5\" (UID: \"faee3b32-feb1-46f7-89a2-e618fe59082c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" Sep 17 07:29:06.965222 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.965162 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/faee3b32-feb1-46f7-89a2-e618fe59082c-ca\") pod \"cluster-proxy-proxy-agent-845f75f54f-t77z5\" (UID: \"faee3b32-feb1-46f7-89a2-e618fe59082c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" Sep 17 07:29:06.969311 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.969131 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/faee3b32-feb1-46f7-89a2-e618fe59082c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-845f75f54f-t77z5\" (UID: \"faee3b32-feb1-46f7-89a2-e618fe59082c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" Sep 17 07:29:06.970210 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.970162 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/faee3b32-feb1-46f7-89a2-e618fe59082c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-845f75f54f-t77z5\" (UID: \"faee3b32-feb1-46f7-89a2-e618fe59082c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" Sep 17 07:29:06.970794 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.970773 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/faee3b32-feb1-46f7-89a2-e618fe59082c-ca\") pod \"cluster-proxy-proxy-agent-845f75f54f-t77z5\" (UID: \"faee3b32-feb1-46f7-89a2-e618fe59082c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" Sep 17 07:29:06.971484 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.971459 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/faee3b32-feb1-46f7-89a2-e618fe59082c-hub\") pod \"cluster-proxy-proxy-agent-845f75f54f-t77z5\" (UID: \"faee3b32-feb1-46f7-89a2-e618fe59082c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" Sep 17 07:29:06.971754 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.971731 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/faee3b32-feb1-46f7-89a2-e618fe59082c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-845f75f54f-t77z5\" (UID: \"faee3b32-feb1-46f7-89a2-e618fe59082c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" Sep 17 07:29:06.990832 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:06.990782 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6749\" (UniqueName: \"kubernetes.io/projected/faee3b32-feb1-46f7-89a2-e618fe59082c-kube-api-access-d6749\") pod \"cluster-proxy-proxy-agent-845f75f54f-t77z5\" (UID: \"faee3b32-feb1-46f7-89a2-e618fe59082c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" Sep 17 07:29:07.216045 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:07.215939 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-k74qf"] Sep 17 07:29:07.219566 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:29:07.219539 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd1ed8f0_f91c_4789_915c_4f5dc7af25f6.slice/crio-c513474bbba1f16cb37aa0c94f7ee8d83ea8a4f7b83ba4dbb33ede9ca9f4d1c0 WatchSource:0}: Error finding container c513474bbba1f16cb37aa0c94f7ee8d83ea8a4f7b83ba4dbb33ede9ca9f4d1c0: Status 404 returned error can't find the container with id c513474bbba1f16cb37aa0c94f7ee8d83ea8a4f7b83ba4dbb33ede9ca9f4d1c0 Sep 17 07:29:07.237845 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:07.237819 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c45796f64-z4sln"] Sep 17 07:29:07.241125 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:29:07.241103 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod865a7bee_7634_4062_9f9a_65cde81f4cf3.slice/crio-e0948ce734c580fddb03a56d4626434f143b7cc7022ece969dffafbfa1b2e907 WatchSource:0}: Error finding container e0948ce734c580fddb03a56d4626434f143b7cc7022ece969dffafbfa1b2e907: Status 404 returned error can't find the container with id e0948ce734c580fddb03a56d4626434f143b7cc7022ece969dffafbfa1b2e907 Sep 17 07:29:07.274271 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:07.274232 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" Sep 17 07:29:07.569258 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:07.569224 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5"] Sep 17 07:29:07.572447 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:29:07.572425 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfaee3b32_feb1_46f7_89a2_e618fe59082c.slice/crio-aeeee3aa3f62069c9b40d221cabe68acaf522afdb10356951feb6e1922247b95 WatchSource:0}: Error finding container aeeee3aa3f62069c9b40d221cabe68acaf522afdb10356951feb6e1922247b95: Status 404 returned error can't find the container with id aeeee3aa3f62069c9b40d221cabe68acaf522afdb10356951feb6e1922247b95 Sep 17 07:29:07.581579 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:07.581544 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-k74qf" event={"ID":"fd1ed8f0-f91c-4789-915c-4f5dc7af25f6","Type":"ContainerStarted","Data":"c513474bbba1f16cb37aa0c94f7ee8d83ea8a4f7b83ba4dbb33ede9ca9f4d1c0"} Sep 17 07:29:07.582563 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:07.582545 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" event={"ID":"faee3b32-feb1-46f7-89a2-e618fe59082c","Type":"ContainerStarted","Data":"aeeee3aa3f62069c9b40d221cabe68acaf522afdb10356951feb6e1922247b95"} Sep 17 07:29:07.583558 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:07.583538 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c45796f64-z4sln" event={"ID":"865a7bee-7634-4062-9f9a-65cde81f4cf3","Type":"ContainerStarted","Data":"e0948ce734c580fddb03a56d4626434f143b7cc7022ece969dffafbfa1b2e907"} Sep 17 07:29:07.974084 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:07.973977 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs\") pod \"network-metrics-daemon-zjz9v\" (UID: \"5eb544bb-f51e-4755-891e-ed3e8f031f0d\") " pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:29:07.977427 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:07.977397 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 17 07:29:07.987294 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:07.987264 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eb544bb-f51e-4755-891e-ed3e8f031f0d-metrics-certs\") pod \"network-metrics-daemon-zjz9v\" (UID: \"5eb544bb-f51e-4755-891e-ed3e8f031f0d\") " pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:29:08.074742 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:08.074705 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1799de17-1341-4f16-9277-79056534a444-metrics-tls\") pod \"dns-default-zcjf5\" (UID: \"1799de17-1341-4f16-9277-79056534a444\") " pod="openshift-dns/dns-default-zcjf5" Sep 17 07:29:08.074926 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:08.074776 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-422lx\" (UniqueName: \"kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx\") pod \"network-check-target-njpsg\" (UID: \"e7339f6e-91b9-4b00-b7aa-915f9ea79232\") " pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:29:08.074926 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:08.074811 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret\") pod \"global-pull-secret-syncer-mmcvp\" (UID: \"d4580c7f-1c6c-420d-962c-0ba1a6a222fa\") " pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:29:08.077325 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:08.077290 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1799de17-1341-4f16-9277-79056534a444-metrics-tls\") pod \"dns-default-zcjf5\" (UID: \"1799de17-1341-4f16-9277-79056534a444\") " pod="openshift-dns/dns-default-zcjf5" Sep 17 07:29:08.077661 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:08.077640 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 07:29:08.077778 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:08.077748 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 17 07:29:08.087933 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:08.087907 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 17 07:29:08.088094 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:08.087946 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4580c7f-1c6c-420d-962c-0ba1a6a222fa-original-pull-secret\") pod \"global-pull-secret-syncer-mmcvp\" (UID: \"d4580c7f-1c6c-420d-962c-0ba1a6a222fa\") " pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:29:08.098968 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:08.098941 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-422lx\" (UniqueName: \"kubernetes.io/projected/e7339f6e-91b9-4b00-b7aa-915f9ea79232-kube-api-access-422lx\") pod \"network-check-target-njpsg\" (UID: \"e7339f6e-91b9-4b00-b7aa-915f9ea79232\") " pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:29:08.116195 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:08.116165 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nzmmj\"" Sep 17 07:29:08.123137 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:08.123112 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-c888k\"" Sep 17 07:29:08.124162 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:08.124140 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:29:08.124294 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:08.124141 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mmcvp" Sep 17 07:29:08.130081 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:08.130060 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjz9v" Sep 17 07:29:08.333300 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:08.333265 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2jzn2\"" Sep 17 07:29:08.338537 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:08.338507 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zcjf5" Sep 17 07:29:08.430261 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:08.430225 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-njpsg"] Sep 17 07:29:08.431954 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:08.431873 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mmcvp"] Sep 17 07:29:08.459174 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:08.459142 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zjz9v"] Sep 17 07:29:08.648647 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:29:08.648570 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eb544bb_f51e_4755_891e_ed3e8f031f0d.slice/crio-a1001df38a18425f28f127f6c24691ff1d03cf10d4cea6b912d7a3ff13121290 WatchSource:0}: Error finding container a1001df38a18425f28f127f6c24691ff1d03cf10d4cea6b912d7a3ff13121290: Status 404 returned error can't find the container with id a1001df38a18425f28f127f6c24691ff1d03cf10d4cea6b912d7a3ff13121290 Sep 17 07:29:08.649504 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:29:08.649346 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7339f6e_91b9_4b00_b7aa_915f9ea79232.slice/crio-0b8d532ddb25ff5fb60b302a784764e0c9f79b7977044cdcd6dbe85653ed02d3 WatchSource:0}: Error finding container 0b8d532ddb25ff5fb60b302a784764e0c9f79b7977044cdcd6dbe85653ed02d3: Status 404 returned error can't find the container with id 0b8d532ddb25ff5fb60b302a784764e0c9f79b7977044cdcd6dbe85653ed02d3 Sep 17 07:29:08.650665 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:29:08.650317 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4580c7f_1c6c_420d_962c_0ba1a6a222fa.slice/crio-4fd515db16d5496c3cbbd44079c11b2f1d78eb35c8e2ed8ae5254ecb7ee07b6a WatchSource:0}: Error finding container 4fd515db16d5496c3cbbd44079c11b2f1d78eb35c8e2ed8ae5254ecb7ee07b6a: Status 404 returned error can't find the container with id 4fd515db16d5496c3cbbd44079c11b2f1d78eb35c8e2ed8ae5254ecb7ee07b6a Sep 17 07:29:08.938933 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:08.938833 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zcjf5"] Sep 17 07:29:08.943064 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:29:08.943034 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1799de17_1341_4f16_9277_79056534a444.slice/crio-720720285cfe57d9a262b471c1f767bcb3716733da1f592a1dba8a4588b23f21 WatchSource:0}: Error finding container 720720285cfe57d9a262b471c1f767bcb3716733da1f592a1dba8a4588b23f21: Status 404 returned error can't find the container with id 720720285cfe57d9a262b471c1f767bcb3716733da1f592a1dba8a4588b23f21 Sep 17 07:29:09.597003 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:09.596948 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-k74qf" event={"ID":"fd1ed8f0-f91c-4789-915c-4f5dc7af25f6","Type":"ContainerStarted","Data":"6319da85e7b899800fd9d3007a1ec904896c4d7d578228a720061765634b7d1d"} Sep 17 07:29:09.599245 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:09.599216 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zcjf5" event={"ID":"1799de17-1341-4f16-9277-79056534a444","Type":"ContainerStarted","Data":"720720285cfe57d9a262b471c1f767bcb3716733da1f592a1dba8a4588b23f21"} Sep 17 07:29:09.601216 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:09.601149 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mmcvp" event={"ID":"d4580c7f-1c6c-420d-962c-0ba1a6a222fa","Type":"ContainerStarted","Data":"4fd515db16d5496c3cbbd44079c11b2f1d78eb35c8e2ed8ae5254ecb7ee07b6a"} Sep 17 07:29:09.603127 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:09.603071 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-njpsg" event={"ID":"e7339f6e-91b9-4b00-b7aa-915f9ea79232","Type":"ContainerStarted","Data":"0b8d532ddb25ff5fb60b302a784764e0c9f79b7977044cdcd6dbe85653ed02d3"} Sep 17 07:29:09.604812 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:09.604756 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zjz9v" event={"ID":"5eb544bb-f51e-4755-891e-ed3e8f031f0d","Type":"ContainerStarted","Data":"a1001df38a18425f28f127f6c24691ff1d03cf10d4cea6b912d7a3ff13121290"} Sep 17 07:29:09.618304 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:09.617641 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-k74qf" podStartSLOduration=2.153193846 podStartE2EDuration="3.617621817s" totalImagesPullingTime="1.464427971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:29:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:09.617276131 +0000 UTC m=+67.807252075" watchObservedRunningTime="2026-09-17 07:29:09.617621817 +0000 UTC m=+67.807597763" Sep 17 07:29:11.620058 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:11.620004 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zjz9v" event={"ID":"5eb544bb-f51e-4755-891e-ed3e8f031f0d","Type":"ContainerStarted","Data":"e8f25da56e12933e87b0b0ec0715e152cec629004cee563e3070fc0bb1c27b5e"} Sep 17 07:29:11.622973 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:11.622943 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c45796f64-z4sln" event={"ID":"865a7bee-7634-4062-9f9a-65cde81f4cf3","Type":"ContainerStarted","Data":"0a243c5f22c33dd5195b3ee66b01c5ad3f5cdb6140d01de178e4c2454981743b"} Sep 17 07:29:16.643416 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:16.643379 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zjz9v" event={"ID":"5eb544bb-f51e-4755-891e-ed3e8f031f0d","Type":"ContainerStarted","Data":"75c7b6665cafaf60df52ab4639e6c3b4fc9a51af655fa1483bb992efb8a58db8"} Sep 17 07:29:16.645187 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:16.645161 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zcjf5" event={"ID":"1799de17-1341-4f16-9277-79056534a444","Type":"ContainerStarted","Data":"b35aa8eca4766a36bdedc5b00e4b0b48459086f3d21711c979e396a6cfdcd2bb"} Sep 17 07:29:16.645305 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:16.645201 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zcjf5" event={"ID":"1799de17-1341-4f16-9277-79056534a444","Type":"ContainerStarted","Data":"9469046b4a2eff2a0395ff23c990ed4db6fdcdaf30e00d9c91b3f3629d522022"} Sep 17 07:29:16.645305 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:16.645239 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zcjf5" Sep 17 07:29:16.646551 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:16.646529 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mmcvp" event={"ID":"d4580c7f-1c6c-420d-962c-0ba1a6a222fa","Type":"ContainerStarted","Data":"d650e7915a2688de36423f646513187c3c0226a0c9acca8c1dcea7815db731fc"} Sep 17 07:29:16.648348 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:16.648327 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" event={"ID":"faee3b32-feb1-46f7-89a2-e618fe59082c","Type":"ContainerStarted","Data":"57f0231613d3eecbb4b37cfe91e44f83c3b69312a720e7847239e2f395967450"} Sep 17 07:29:16.648437 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:16.648356 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" event={"ID":"faee3b32-feb1-46f7-89a2-e618fe59082c","Type":"ContainerStarted","Data":"66aa28f708bf3299da8206b6ec2d0d0b7c87f7f92c2b80d95d5dd8c24295a79d"} Sep 17 07:29:16.648437 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:16.648371 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" event={"ID":"faee3b32-feb1-46f7-89a2-e618fe59082c","Type":"ContainerStarted","Data":"958a5744672de1ef95cf8699df2ef1691a74f10c9e6a7f2c91be377d861c4e39"} Sep 17 07:29:16.649680 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:16.649662 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-njpsg" event={"ID":"e7339f6e-91b9-4b00-b7aa-915f9ea79232","Type":"ContainerStarted","Data":"4241b403515d3515510e141aaace619aae812706beaf33513daede857136668a"} Sep 17 07:29:16.649765 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:16.649739 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:29:16.661968 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:16.661931 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c45796f64-z4sln" podStartSLOduration=6.699031407 podStartE2EDuration="10.661918112s" totalImagesPullingTime="3.962886705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:29:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:11.640764511 +0000 UTC m=+69.830740455" watchObservedRunningTime="2026-09-17 07:29:16.661918112 +0000 UTC m=+74.851894094" Sep 17 07:29:16.662089 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:16.662071 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zjz9v" podStartSLOduration=72.119201438 podStartE2EDuration="1m14.662066363s" totalImagesPullingTime="2.542864925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:16.661337094 +0000 UTC m=+74.851313048" watchObservedRunningTime="2026-09-17 07:29:16.662066363 +0000 UTC m=+74.852042304" Sep 17 07:29:16.688171 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:16.688124 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zcjf5" podStartSLOduration=34.056322737 podStartE2EDuration="40.688111818s" totalImagesPullingTime="6.631789081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:28:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:16.688060516 +0000 UTC m=+74.878036458" watchObservedRunningTime="2026-09-17 07:29:16.688111818 +0000 UTC m=+74.878087759" Sep 17 07:29:16.705779 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:16.705743 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-njpsg" podStartSLOduration=67.773719059 podStartE2EDuration="1m14.705731968s" totalImagesPullingTime="6.932012909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:16.705037801 +0000 UTC m=+74.895013742" watchObservedRunningTime="2026-09-17 07:29:16.705731968 +0000 UTC m=+74.895707910" Sep 17 07:29:16.724074 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:16.724021 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mmcvp" podStartSLOduration=67.791992803 podStartE2EDuration="1m14.724009262s" totalImagesPullingTime="6.932016459s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:16.723661356 +0000 UTC m=+74.913637294" watchObservedRunningTime="2026-09-17 07:29:16.724009262 +0000 UTC m=+74.913985198" Sep 17 07:29:16.745522 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:16.745491 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-845f75f54f-t77z5" podStartSLOduration=2.709886815 podStartE2EDuration="10.745480288s" totalImagesPullingTime="8.035593473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:29:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:16.744965793 +0000 UTC m=+74.934941736" watchObservedRunningTime="2026-09-17 07:29:16.745480288 +0000 UTC m=+74.935456229" Sep 17 07:29:26.656079 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:26.655950 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zcjf5" Sep 17 07:29:31.776324 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.776289 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5wb9t"] Sep 17 07:29:31.782472 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.782441 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.786373 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.786347 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 17 07:29:31.786472 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.786412 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 17 07:29:31.788068 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.788049 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 17 07:29:31.788272 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.788257 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 17 07:29:31.788334 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.788321 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gnwph\"" Sep 17 07:29:31.788382 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.788344 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 17 07:29:31.788382 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.788349 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 17 07:29:31.836948 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.836923 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/848dcf22-a209-4c6b-ad51-801dffe84fb0-node-exporter-wtmp\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.837063 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.836966 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/848dcf22-a209-4c6b-ad51-801dffe84fb0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.837063 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.836983 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/848dcf22-a209-4c6b-ad51-801dffe84fb0-node-exporter-textfile\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.837063 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.837023 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/848dcf22-a209-4c6b-ad51-801dffe84fb0-node-exporter-accelerators-collector-config\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.837208 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.837091 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/848dcf22-a209-4c6b-ad51-801dffe84fb0-sys\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.837208 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.837145 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/848dcf22-a209-4c6b-ad51-801dffe84fb0-root\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.837208 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.837168 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/848dcf22-a209-4c6b-ad51-801dffe84fb0-node-exporter-tls\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.837208 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.837190 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtxx6\" (UniqueName: \"kubernetes.io/projected/848dcf22-a209-4c6b-ad51-801dffe84fb0-kube-api-access-jtxx6\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.837364 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.837211 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/848dcf22-a209-4c6b-ad51-801dffe84fb0-metrics-client-ca\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.937587 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.937550 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/848dcf22-a209-4c6b-ad51-801dffe84fb0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.937587 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.937589 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/848dcf22-a209-4c6b-ad51-801dffe84fb0-node-exporter-textfile\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.937784 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.937611 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/848dcf22-a209-4c6b-ad51-801dffe84fb0-node-exporter-accelerators-collector-config\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.937784 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.937724 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/848dcf22-a209-4c6b-ad51-801dffe84fb0-sys\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.937784 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.937759 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/848dcf22-a209-4c6b-ad51-801dffe84fb0-root\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.937932 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.937786 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/848dcf22-a209-4c6b-ad51-801dffe84fb0-node-exporter-tls\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.937932 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.937815 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtxx6\" (UniqueName: \"kubernetes.io/projected/848dcf22-a209-4c6b-ad51-801dffe84fb0-kube-api-access-jtxx6\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.937932 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.937845 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/848dcf22-a209-4c6b-ad51-801dffe84fb0-sys\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.937932 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.937856 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/848dcf22-a209-4c6b-ad51-801dffe84fb0-root\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.937932 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.937851 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/848dcf22-a209-4c6b-ad51-801dffe84fb0-metrics-client-ca\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.937932 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:29:31.937913 2579 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 17 07:29:31.937932 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.937926 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/848dcf22-a209-4c6b-ad51-801dffe84fb0-node-exporter-wtmp\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.938224 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.937931 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/848dcf22-a209-4c6b-ad51-801dffe84fb0-node-exporter-textfile\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.938224 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:29:31.937973 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/848dcf22-a209-4c6b-ad51-801dffe84fb0-node-exporter-tls podName:848dcf22-a209-4c6b-ad51-801dffe84fb0 nodeName:}" failed. No retries permitted until 2026-09-17 07:29:32.437953997 +0000 UTC m=+90.627929937 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/848dcf22-a209-4c6b-ad51-801dffe84fb0-node-exporter-tls") pod "node-exporter-5wb9t" (UID: "848dcf22-a209-4c6b-ad51-801dffe84fb0") : secret "node-exporter-tls" not found Sep 17 07:29:31.938224 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.938072 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/848dcf22-a209-4c6b-ad51-801dffe84fb0-node-exporter-wtmp\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.938336 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.938235 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/848dcf22-a209-4c6b-ad51-801dffe84fb0-node-exporter-accelerators-collector-config\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.938452 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.938436 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/848dcf22-a209-4c6b-ad51-801dffe84fb0-metrics-client-ca\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.939946 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.939924 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/848dcf22-a209-4c6b-ad51-801dffe84fb0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:31.965174 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:31.965145 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtxx6\" (UniqueName: \"kubernetes.io/projected/848dcf22-a209-4c6b-ad51-801dffe84fb0-kube-api-access-jtxx6\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:32.441822 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:32.441790 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/848dcf22-a209-4c6b-ad51-801dffe84fb0-node-exporter-tls\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:32.444074 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:32.443983 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/848dcf22-a209-4c6b-ad51-801dffe84fb0-node-exporter-tls\") pod \"node-exporter-5wb9t\" (UID: \"848dcf22-a209-4c6b-ad51-801dffe84fb0\") " pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:32.691675 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:32.691629 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5wb9t" Sep 17 07:29:32.704953 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:29:32.704922 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod848dcf22_a209_4c6b_ad51_801dffe84fb0.slice/crio-2ef47da6ddb9444414eead2740743c8f5a29fa8b843324815901cb5cb3038fdd WatchSource:0}: Error finding container 2ef47da6ddb9444414eead2740743c8f5a29fa8b843324815901cb5cb3038fdd: Status 404 returned error can't find the container with id 2ef47da6ddb9444414eead2740743c8f5a29fa8b843324815901cb5cb3038fdd Sep 17 07:29:33.704585 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:33.704547 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5wb9t" event={"ID":"848dcf22-a209-4c6b-ad51-801dffe84fb0","Type":"ContainerStarted","Data":"2ef47da6ddb9444414eead2740743c8f5a29fa8b843324815901cb5cb3038fdd"} Sep 17 07:29:34.709164 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:34.709133 2579 generic.go:356] "Generic (PLEG): container finished" podID="848dcf22-a209-4c6b-ad51-801dffe84fb0" containerID="e3bcfb6a4e69ae2664de95bac8041895a710b72246030a95fc7ddb3d6534ba7b" exitCode=0 Sep 17 07:29:34.709507 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:34.709175 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5wb9t" event={"ID":"848dcf22-a209-4c6b-ad51-801dffe84fb0","Type":"ContainerDied","Data":"e3bcfb6a4e69ae2664de95bac8041895a710b72246030a95fc7ddb3d6534ba7b"} Sep 17 07:29:35.717041 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:35.717005 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5wb9t" event={"ID":"848dcf22-a209-4c6b-ad51-801dffe84fb0","Type":"ContainerStarted","Data":"fb5e7838b4c43b6ddc697d0fdce6ffe33aaba761d724487bb68c3060887f8a0e"} Sep 17 07:29:35.717416 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:35.717076 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5wb9t" event={"ID":"848dcf22-a209-4c6b-ad51-801dffe84fb0","Type":"ContainerStarted","Data":"3d6fdb0e68e8da0156e62aad5373934bee08e2923d233ab59d4afe37b025faeb"} Sep 17 07:29:35.745894 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:35.745845 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5wb9t" podStartSLOduration=3.675588919 podStartE2EDuration="4.7458325s" totalImagesPullingTime="1.009868408s" totalInitContainerRuntime="60.375173ms" isStatefulPod=true podCreationTimestamp="2026-09-17 07:29:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:35.744108761 +0000 UTC m=+93.934084703" watchObservedRunningTime="2026-09-17 07:29:35.7458325 +0000 UTC m=+93.935808441" Sep 17 07:29:47.655925 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:29:47.655890 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-njpsg" Sep 17 07:33:02.188927 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:33:02.188897 2579 kubelet.go:1648] "Image garbage collection succeeded" Sep 17 07:40:16.946733 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:16.946656 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-test-apply-4p977"] Sep 17 07:40:16.948796 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:16.948778 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-4p977" Sep 17 07:40:16.951331 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:16.951307 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 17 07:40:16.951449 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:16.951308 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 17 07:40:16.952171 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:16.952147 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 17 07:40:16.952267 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:16.952213 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-cronjob-dockercfg-zrtjg\"" Sep 17 07:40:16.957730 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:16.957709 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-4p977"] Sep 17 07:40:17.004410 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:17.004383 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw6ww\" (UniqueName: \"kubernetes.io/projected/12221f7a-ee96-4ae9-a787-cec60118176b-kube-api-access-jw6ww\") pod \"feast-test-apply-4p977\" (UID: \"12221f7a-ee96-4ae9-a787-cec60118176b\") " pod="test-ns-feast/feast-test-apply-4p977" Sep 17 07:40:17.104925 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:17.104883 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw6ww\" (UniqueName: \"kubernetes.io/projected/12221f7a-ee96-4ae9-a787-cec60118176b-kube-api-access-jw6ww\") pod \"feast-test-apply-4p977\" (UID: \"12221f7a-ee96-4ae9-a787-cec60118176b\") " pod="test-ns-feast/feast-test-apply-4p977" Sep 17 07:40:17.114612 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:17.114588 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw6ww\" (UniqueName: \"kubernetes.io/projected/12221f7a-ee96-4ae9-a787-cec60118176b-kube-api-access-jw6ww\") pod \"feast-test-apply-4p977\" (UID: \"12221f7a-ee96-4ae9-a787-cec60118176b\") " pod="test-ns-feast/feast-test-apply-4p977" Sep 17 07:40:17.257965 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:17.257862 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-4p977" Sep 17 07:40:17.526688 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:17.526659 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-4p977"] Sep 17 07:40:17.530550 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:40:17.530523 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12221f7a_ee96_4ae9_a787_cec60118176b.slice/crio-35251da2b2b23c8335623353dd0d8057938ac74eab7e65629307b8f8196ea185 WatchSource:0}: Error finding container 35251da2b2b23c8335623353dd0d8057938ac74eab7e65629307b8f8196ea185: Status 404 returned error can't find the container with id 35251da2b2b23c8335623353dd0d8057938ac74eab7e65629307b8f8196ea185 Sep 17 07:40:17.532414 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:17.532398 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 07:40:17.830951 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:17.830865 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-4p977" event={"ID":"12221f7a-ee96-4ae9-a787-cec60118176b","Type":"ContainerStarted","Data":"35251da2b2b23c8335623353dd0d8057938ac74eab7e65629307b8f8196ea185"} Sep 17 07:40:22.852892 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:22.852857 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-4p977" event={"ID":"12221f7a-ee96-4ae9-a787-cec60118176b","Type":"ContainerStarted","Data":"3945dded4b46f49762053bfef9c1de48b8fbedc55689ad3fe5ea6cc06960c852"} Sep 17 07:40:29.878788 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:29.878758 2579 generic.go:356] "Generic (PLEG): container finished" podID="12221f7a-ee96-4ae9-a787-cec60118176b" containerID="3945dded4b46f49762053bfef9c1de48b8fbedc55689ad3fe5ea6cc06960c852" exitCode=0 Sep 17 07:40:29.879171 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:29.878804 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-4p977" event={"ID":"12221f7a-ee96-4ae9-a787-cec60118176b","Type":"ContainerDied","Data":"3945dded4b46f49762053bfef9c1de48b8fbedc55689ad3fe5ea6cc06960c852"} Sep 17 07:40:30.884529 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:30.884493 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-4p977" event={"ID":"12221f7a-ee96-4ae9-a787-cec60118176b","Type":"ContainerStarted","Data":"49d0e55e6ef973b7a381be3a594f3ea9f0599e8aa0fdbab0efd09bb892f89130"} Sep 17 07:40:30.903489 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:30.903441 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-test-apply-4p977" podStartSLOduration=3.334303332 podStartE2EDuration="14.903427279s" totalImagesPullingTime="5.121603107s" totalInitContainerRuntime="6.44752084s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:40:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:40:30.903308466 +0000 UTC m=+749.093284408" watchObservedRunningTime="2026-09-17 07:40:30.903427279 +0000 UTC m=+749.093403220" Sep 17 07:40:39.915884 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:39.915853 2579 generic.go:356] "Generic (PLEG): container finished" podID="12221f7a-ee96-4ae9-a787-cec60118176b" containerID="49d0e55e6ef973b7a381be3a594f3ea9f0599e8aa0fdbab0efd09bb892f89130" exitCode=0 Sep 17 07:40:39.916212 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:39.915930 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-4p977" event={"ID":"12221f7a-ee96-4ae9-a787-cec60118176b","Type":"ContainerDied","Data":"49d0e55e6ef973b7a381be3a594f3ea9f0599e8aa0fdbab0efd09bb892f89130"} Sep 17 07:40:41.038100 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:41.038077 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-4p977" Sep 17 07:40:41.174733 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:41.174642 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jw6ww\" (UniqueName: \"kubernetes.io/projected/12221f7a-ee96-4ae9-a787-cec60118176b-kube-api-access-jw6ww\") pod \"12221f7a-ee96-4ae9-a787-cec60118176b\" (UID: \"12221f7a-ee96-4ae9-a787-cec60118176b\") " Sep 17 07:40:41.176791 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:41.176764 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12221f7a-ee96-4ae9-a787-cec60118176b-kube-api-access-jw6ww" (OuterVolumeSpecName: "kube-api-access-jw6ww") pod "12221f7a-ee96-4ae9-a787-cec60118176b" (UID: "12221f7a-ee96-4ae9-a787-cec60118176b"). InnerVolumeSpecName "kube-api-access-jw6ww". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:40:41.275845 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:41.275808 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jw6ww\" (UniqueName: \"kubernetes.io/projected/12221f7a-ee96-4ae9-a787-cec60118176b-kube-api-access-jw6ww\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:40:41.924191 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:41.924157 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-4p977" event={"ID":"12221f7a-ee96-4ae9-a787-cec60118176b","Type":"ContainerDied","Data":"35251da2b2b23c8335623353dd0d8057938ac74eab7e65629307b8f8196ea185"} Sep 17 07:40:41.924191 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:41.924195 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35251da2b2b23c8335623353dd0d8057938ac74eab7e65629307b8f8196ea185" Sep 17 07:40:41.924391 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:41.924196 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-4p977" Sep 17 07:40:42.115264 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:42.115229 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-4p977_12221f7a-ee96-4ae9-a787-cec60118176b/feast-0/0.log" Sep 17 07:40:42.127270 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:40:42.127239 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-4p977_12221f7a-ee96-4ae9-a787-cec60118176b/feast-1/0.log" Sep 17 07:41:19.344521 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:41:19.344485 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-test-apply-4p977"] Sep 17 07:41:19.351396 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:41:19.351363 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-test-apply-4p977"] Sep 17 07:41:20.310296 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:41:20.310261 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12221f7a-ee96-4ae9-a787-cec60118176b" path="/var/lib/kubelet/pods/12221f7a-ee96-4ae9-a787-cec60118176b/volumes" Sep 17 07:47:02.267460 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:02.267429 2579 scope.go:117] "RemoveContainer" containerID="49d0e55e6ef973b7a381be3a594f3ea9f0599e8aa0fdbab0efd09bb892f89130" Sep 17 07:47:02.275777 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:02.275755 2579 scope.go:117] "RemoveContainer" containerID="3945dded4b46f49762053bfef9c1de48b8fbedc55689ad3fe5ea6cc06960c852" Sep 17 07:47:08.815670 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:08.815632 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-pr9xn"] Sep 17 07:47:08.816080 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:08.815864 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12221f7a-ee96-4ae9-a787-cec60118176b" containerName="feast-0" Sep 17 07:47:08.816080 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:08.815876 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="12221f7a-ee96-4ae9-a787-cec60118176b" containerName="feast-0" Sep 17 07:47:08.816080 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:08.815884 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12221f7a-ee96-4ae9-a787-cec60118176b" containerName="feast-1" Sep 17 07:47:08.816080 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:08.815890 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="12221f7a-ee96-4ae9-a787-cec60118176b" containerName="feast-1" Sep 17 07:47:08.816080 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:08.815930 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="12221f7a-ee96-4ae9-a787-cec60118176b" containerName="feast-1" Sep 17 07:47:08.819973 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:08.819953 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pr9xn" Sep 17 07:47:08.823007 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:08.822965 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-zjrrp\"" Sep 17 07:47:08.823497 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:08.823483 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 17 07:47:08.823566 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:08.823520 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 17 07:47:08.836481 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:08.836456 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-pr9xn"] Sep 17 07:47:08.880617 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:08.880577 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt2hb\" (UniqueName: \"kubernetes.io/projected/306697f1-0bc7-43db-bc6b-97230e0e7917-kube-api-access-zt2hb\") pod \"feast-operator-controller-manager-689b877b4b-pr9xn\" (UID: \"306697f1-0bc7-43db-bc6b-97230e0e7917\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pr9xn" Sep 17 07:47:08.981577 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:08.981525 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt2hb\" (UniqueName: \"kubernetes.io/projected/306697f1-0bc7-43db-bc6b-97230e0e7917-kube-api-access-zt2hb\") pod \"feast-operator-controller-manager-689b877b4b-pr9xn\" (UID: \"306697f1-0bc7-43db-bc6b-97230e0e7917\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pr9xn" Sep 17 07:47:08.991708 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:08.991675 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt2hb\" (UniqueName: \"kubernetes.io/projected/306697f1-0bc7-43db-bc6b-97230e0e7917-kube-api-access-zt2hb\") pod \"feast-operator-controller-manager-689b877b4b-pr9xn\" (UID: \"306697f1-0bc7-43db-bc6b-97230e0e7917\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pr9xn" Sep 17 07:47:09.129611 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:09.129515 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pr9xn" Sep 17 07:47:09.432671 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:09.432588 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-pr9xn"] Sep 17 07:47:09.436080 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:47:09.436041 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod306697f1_0bc7_43db_bc6b_97230e0e7917.slice/crio-6c5ae22fa8485e5a822079ae7ed733b4b646d6acf39119a4601a9232d44e6a4e WatchSource:0}: Error finding container 6c5ae22fa8485e5a822079ae7ed733b4b646d6acf39119a4601a9232d44e6a4e: Status 404 returned error can't find the container with id 6c5ae22fa8485e5a822079ae7ed733b4b646d6acf39119a4601a9232d44e6a4e Sep 17 07:47:09.437950 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:09.437932 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 07:47:10.189280 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:10.189235 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pr9xn" event={"ID":"306697f1-0bc7-43db-bc6b-97230e0e7917","Type":"ContainerStarted","Data":"6c5ae22fa8485e5a822079ae7ed733b4b646d6acf39119a4601a9232d44e6a4e"} Sep 17 07:47:12.196838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:12.196804 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pr9xn" event={"ID":"306697f1-0bc7-43db-bc6b-97230e0e7917","Type":"ContainerStarted","Data":"c51de627948cfbb0f7999eaea978bd3e659c2f3bd80475b9ac9aee5d50043ff3"} Sep 17 07:47:12.197331 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:12.196878 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pr9xn" Sep 17 07:47:12.219724 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:12.219677 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pr9xn" podStartSLOduration=2.495796503 podStartE2EDuration="4.219663109s" totalImagesPullingTime="1.723866606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:47:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:47:12.218093664 +0000 UTC m=+1150.408069607" watchObservedRunningTime="2026-09-17 07:47:12.219663109 +0000 UTC m=+1150.409639092" Sep 17 07:47:23.203042 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:23.203013 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pr9xn" Sep 17 07:47:25.824750 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.822328 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn"] Sep 17 07:47:25.827388 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.827367 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:25.830100 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.830079 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 17 07:47:25.830255 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.830233 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 17 07:47:25.830862 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.830841 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 17 07:47:25.830976 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.830870 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-cfsjr\"" Sep 17 07:47:25.830976 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.830946 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 17 07:47:25.830976 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.830903 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 17 07:47:25.831172 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.830874 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 17 07:47:25.842615 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.842589 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn"] Sep 17 07:47:25.893359 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.893325 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/80313f99-034a-4604-a6ef-bd9cc8efea5d-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-6h6qn\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:25.893524 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.893366 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xk6z\" (UniqueName: \"kubernetes.io/projected/80313f99-034a-4604-a6ef-bd9cc8efea5d-kube-api-access-6xk6z\") pod \"feast-simple-feast-setup-57576b8fcf-6h6qn\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:25.893524 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.893386 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6h6qn\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:25.893524 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.893462 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6h6qn\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:25.893649 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.893531 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6h6qn\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:25.893649 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.893573 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6h6qn\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:25.994605 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.994562 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/80313f99-034a-4604-a6ef-bd9cc8efea5d-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-6h6qn\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:25.994605 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.994611 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xk6z\" (UniqueName: \"kubernetes.io/projected/80313f99-034a-4604-a6ef-bd9cc8efea5d-kube-api-access-6xk6z\") pod \"feast-simple-feast-setup-57576b8fcf-6h6qn\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:25.994875 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.994630 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6h6qn\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:25.994875 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.994652 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6h6qn\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:25.994875 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.994681 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6h6qn\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:25.994875 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.994704 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6h6qn\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:25.995121 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.995074 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/80313f99-034a-4604-a6ef-bd9cc8efea5d-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-6h6qn\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:25.997669 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.997640 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6h6qn\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:25.997800 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.997681 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6h6qn\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:25.997800 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.997674 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6h6qn\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:25.997911 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:25.997797 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6h6qn\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:26.002497 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:26.002477 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xk6z\" (UniqueName: \"kubernetes.io/projected/80313f99-034a-4604-a6ef-bd9cc8efea5d-kube-api-access-6xk6z\") pod \"feast-simple-feast-setup-57576b8fcf-6h6qn\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:26.136089 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:26.135975 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:26.415167 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:26.415127 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn"] Sep 17 07:47:26.417804 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:47:26.417777 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80313f99_034a_4604_a6ef_bd9cc8efea5d.slice/crio-3b49b9029fccc3ec8485e8fd1ab57d767e54997ff65ba2f13bc8f3a0f702ca75 WatchSource:0}: Error finding container 3b49b9029fccc3ec8485e8fd1ab57d767e54997ff65ba2f13bc8f3a0f702ca75: Status 404 returned error can't find the container with id 3b49b9029fccc3ec8485e8fd1ab57d767e54997ff65ba2f13bc8f3a0f702ca75 Sep 17 07:47:27.251906 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:27.251809 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" event={"ID":"80313f99-034a-4604-a6ef-bd9cc8efea5d","Type":"ContainerStarted","Data":"3b49b9029fccc3ec8485e8fd1ab57d767e54997ff65ba2f13bc8f3a0f702ca75"} Sep 17 07:47:40.301956 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:40.301913 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" event={"ID":"80313f99-034a-4604-a6ef-bd9cc8efea5d","Type":"ContainerStarted","Data":"ea436bbf3a0227ac61934c7b54d88045738377f5d6f7bef0d10b60f96261547a"} Sep 17 07:47:45.323110 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:45.323070 2579 generic.go:356] "Generic (PLEG): container finished" podID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerID="ea436bbf3a0227ac61934c7b54d88045738377f5d6f7bef0d10b60f96261547a" exitCode=0 Sep 17 07:47:45.323530 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:45.323144 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" event={"ID":"80313f99-034a-4604-a6ef-bd9cc8efea5d","Type":"ContainerDied","Data":"ea436bbf3a0227ac61934c7b54d88045738377f5d6f7bef0d10b60f96261547a"} Sep 17 07:47:46.328365 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:46.328335 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" event={"ID":"80313f99-034a-4604-a6ef-bd9cc8efea5d","Type":"ContainerStarted","Data":"3ce6a8fdb43e4602b59d3b61cced8eaab912872f716460e953271c92276ed34b"} Sep 17 07:47:51.346210 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:51.346131 2579 generic.go:356] "Generic (PLEG): container finished" podID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerID="3ce6a8fdb43e4602b59d3b61cced8eaab912872f716460e953271c92276ed34b" exitCode=0 Sep 17 07:47:51.346210 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:51.346194 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" event={"ID":"80313f99-034a-4604-a6ef-bd9cc8efea5d","Type":"ContainerDied","Data":"3ce6a8fdb43e4602b59d3b61cced8eaab912872f716460e953271c92276ed34b"} Sep 17 07:47:52.355280 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:52.355243 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" event={"ID":"80313f99-034a-4604-a6ef-bd9cc8efea5d","Type":"ContainerStarted","Data":"20ab419993260047051a2551c3fe5a0bbd72b0c63512582c6960dfaac439ad9b"} Sep 17 07:47:52.355876 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:52.355853 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" event={"ID":"80313f99-034a-4604-a6ef-bd9cc8efea5d","Type":"ContainerStarted","Data":"6b2ad0593484d8189d9ae4a9019626608fe0cf4524bba8a9372ce00b42a539d9"} Sep 17 07:47:52.356055 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:52.356037 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" event={"ID":"80313f99-034a-4604-a6ef-bd9cc8efea5d","Type":"ContainerStarted","Data":"e3621204dca7d62be797204e282ba62b0ac51ba5e3fbb510db9d2b4321331cb1"} Sep 17 07:47:52.356169 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:52.356155 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" event={"ID":"80313f99-034a-4604-a6ef-bd9cc8efea5d","Type":"ContainerStarted","Data":"81d4c14d9ec9baa883a64a1129aa1ddb63b33aeb1dacea038d3c4c76e95b54b3"} Sep 17 07:47:52.385506 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:52.385448 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" podStartSLOduration=14.150966991 podStartE2EDuration="27.385428735s" totalImagesPullingTime="13.234461744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:47:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:47:52.383721278 +0000 UTC m=+1190.573697222" watchObservedRunningTime="2026-09-17 07:47:52.385428735 +0000 UTC m=+1190.575404678" Sep 17 07:47:53.137020 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:53.136968 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:53.137283 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:53.137264 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:53.137406 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:53.137394 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:53.137496 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:53.137486 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:53.139155 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:53.139108 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.15:8816: connect: connection refused" Sep 17 07:47:53.139275 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:53.139149 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.15:8443: connect: connection refused" Sep 17 07:47:53.139275 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:53.139162 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.15:6567/health\": dial tcp 10.134.0.15:6567: connect: connection refused" Sep 17 07:47:53.139275 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:53.139247 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.15:6571: connect: connection refused" Sep 17 07:47:56.136736 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:56.136691 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:56.137281 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:56.137234 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.15:6567/health\": dial tcp 10.134.0.15:6567: connect: connection refused" Sep 17 07:47:56.137281 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:56.137276 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:56.137461 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:56.137017 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.15:8443: connect: connection refused" Sep 17 07:47:56.137461 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:56.137308 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:56.137461 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:56.137328 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:56.137461 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:56.136953 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.15:6571: connect: connection refused" Sep 17 07:47:56.137461 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:56.137048 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.15:8816: connect: connection refused" Sep 17 07:47:59.136844 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:59.136789 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.15:8443: connect: connection refused" Sep 17 07:47:59.137738 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:59.137716 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:59.137832 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:59.137828 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:59.138723 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:59.138703 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:59.139175 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:59.139154 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:59.182556 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:59.182524 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:47:59.392767 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:47:59.392676 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:48:02.136931 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:48:02.136904 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:48:02.399768 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:48:02.399690 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:49:04.144136 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.143939 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn"] Sep 17 07:49:04.144690 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.144246 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="registry" containerID="cri-o://81d4c14d9ec9baa883a64a1129aa1ddb63b33aeb1dacea038d3c4c76e95b54b3" gracePeriod=30 Sep 17 07:49:04.144690 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.144338 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="online" containerID="cri-o://e3621204dca7d62be797204e282ba62b0ac51ba5e3fbb510db9d2b4321331cb1" gracePeriod=30 Sep 17 07:49:04.144690 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.144357 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="ui" containerID="cri-o://20ab419993260047051a2551c3fe5a0bbd72b0c63512582c6960dfaac439ad9b" gracePeriod=30 Sep 17 07:49:04.144690 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.144356 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="offline" containerID="cri-o://6b2ad0593484d8189d9ae4a9019626608fe0cf4524bba8a9372ce00b42a539d9" gracePeriod=30 Sep 17 07:49:04.574145 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.574104 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs"] Sep 17 07:49:04.577693 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.577675 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:04.580283 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.580261 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-5rv2j\"" Sep 17 07:49:04.591912 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.591883 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs"] Sep 17 07:49:04.717321 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.717287 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9ckrs\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:04.717321 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.717325 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9ckrs\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:04.717560 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.717343 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9lmh\" (UniqueName: \"kubernetes.io/projected/7868106b-568f-45a7-83cf-222105046dc8-kube-api-access-k9lmh\") pod \"feast-simple-feast-setup-57576b8fcf-9ckrs\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:04.717560 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.717373 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7868106b-568f-45a7-83cf-222105046dc8-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-9ckrs\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:04.717560 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.717453 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9ckrs\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:04.717560 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.717488 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9ckrs\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:04.818688 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.818654 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9ckrs\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:04.818688 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.818691 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9ckrs\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:04.818932 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.818710 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9lmh\" (UniqueName: \"kubernetes.io/projected/7868106b-568f-45a7-83cf-222105046dc8-kube-api-access-k9lmh\") pod \"feast-simple-feast-setup-57576b8fcf-9ckrs\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:04.818932 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.818739 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7868106b-568f-45a7-83cf-222105046dc8-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-9ckrs\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:04.818932 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.818764 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9ckrs\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:04.818932 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.818784 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9ckrs\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:04.818932 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:49:04.818876 2579 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 17 07:49:04.819276 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:49:04.818979 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-online-tls podName:7868106b-568f-45a7-83cf-222105046dc8 nodeName:}" failed. No retries permitted until 2026-09-17 07:49:05.318957916 +0000 UTC m=+1263.508933853 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-online-tls") pod "feast-simple-feast-setup-57576b8fcf-9ckrs" (UID: "7868106b-568f-45a7-83cf-222105046dc8") : secret "feast-simple-feast-setup-online-tls" not found Sep 17 07:49:04.819276 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.819265 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7868106b-568f-45a7-83cf-222105046dc8-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-9ckrs\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:04.821518 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.821490 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9ckrs\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:04.821518 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.821502 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9ckrs\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:04.822273 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.822253 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9ckrs\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:04.827758 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:04.827614 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9lmh\" (UniqueName: \"kubernetes.io/projected/7868106b-568f-45a7-83cf-222105046dc8-kube-api-access-k9lmh\") pod \"feast-simple-feast-setup-57576b8fcf-9ckrs\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:05.322647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:05.322608 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9ckrs\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:05.324975 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:05.324953 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9ckrs\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:05.486598 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:05.486559 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:05.613828 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:05.613748 2579 generic.go:356] "Generic (PLEG): container finished" podID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerID="20ab419993260047051a2551c3fe5a0bbd72b0c63512582c6960dfaac439ad9b" exitCode=0 Sep 17 07:49:05.613828 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:05.613770 2579 generic.go:356] "Generic (PLEG): container finished" podID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerID="e3621204dca7d62be797204e282ba62b0ac51ba5e3fbb510db9d2b4321331cb1" exitCode=0 Sep 17 07:49:05.613828 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:05.613815 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" event={"ID":"80313f99-034a-4604-a6ef-bd9cc8efea5d","Type":"ContainerDied","Data":"20ab419993260047051a2551c3fe5a0bbd72b0c63512582c6960dfaac439ad9b"} Sep 17 07:49:05.614055 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:05.613855 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" event={"ID":"80313f99-034a-4604-a6ef-bd9cc8efea5d","Type":"ContainerDied","Data":"e3621204dca7d62be797204e282ba62b0ac51ba5e3fbb510db9d2b4321331cb1"} Sep 17 07:49:05.788472 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:05.788444 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs"] Sep 17 07:49:05.790216 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:49:05.790193 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7868106b_568f_45a7_83cf_222105046dc8.slice/crio-0af56bf53dc600a5701db5f8d90fb88cc13f584f8d6a01ec52d1f68c8bba4541 WatchSource:0}: Error finding container 0af56bf53dc600a5701db5f8d90fb88cc13f584f8d6a01ec52d1f68c8bba4541: Status 404 returned error can't find the container with id 0af56bf53dc600a5701db5f8d90fb88cc13f584f8d6a01ec52d1f68c8bba4541 Sep 17 07:49:06.619350 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:06.619308 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" event={"ID":"7868106b-568f-45a7-83cf-222105046dc8","Type":"ContainerStarted","Data":"abe5004ff4a296b7e619c1995a76d38b2f8a3ba24728df690f78bf223d699c7b"} Sep 17 07:49:06.619730 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:06.619369 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" event={"ID":"7868106b-568f-45a7-83cf-222105046dc8","Type":"ContainerStarted","Data":"0af56bf53dc600a5701db5f8d90fb88cc13f584f8d6a01ec52d1f68c8bba4541"} Sep 17 07:49:09.389510 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:09.389468 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.15:6567/health\": dial tcp 10.134.0.15:6567: connect: connection refused" Sep 17 07:49:11.640459 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:11.640418 2579 generic.go:356] "Generic (PLEG): container finished" podID="7868106b-568f-45a7-83cf-222105046dc8" containerID="abe5004ff4a296b7e619c1995a76d38b2f8a3ba24728df690f78bf223d699c7b" exitCode=0 Sep 17 07:49:11.640459 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:11.640466 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" event={"ID":"7868106b-568f-45a7-83cf-222105046dc8","Type":"ContainerDied","Data":"abe5004ff4a296b7e619c1995a76d38b2f8a3ba24728df690f78bf223d699c7b"} Sep 17 07:49:12.399484 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:12.399435 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.15:8443: connect: connection refused" Sep 17 07:49:12.645647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:12.645613 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" event={"ID":"7868106b-568f-45a7-83cf-222105046dc8","Type":"ContainerStarted","Data":"49919e4178909120d54733febfb64509e521b85718180d9caf9b557a20da4c96"} Sep 17 07:49:18.670034 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:18.669980 2579 generic.go:356] "Generic (PLEG): container finished" podID="7868106b-568f-45a7-83cf-222105046dc8" containerID="49919e4178909120d54733febfb64509e521b85718180d9caf9b557a20da4c96" exitCode=0 Sep 17 07:49:18.670522 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:18.670035 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" event={"ID":"7868106b-568f-45a7-83cf-222105046dc8","Type":"ContainerDied","Data":"49919e4178909120d54733febfb64509e521b85718180d9caf9b557a20da4c96"} Sep 17 07:49:19.389928 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:19.389874 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.15:6567/health\": dial tcp 10.134.0.15:6567: connect: connection refused" Sep 17 07:49:19.680954 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:19.680856 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" event={"ID":"7868106b-568f-45a7-83cf-222105046dc8","Type":"ContainerStarted","Data":"98548dabb2ecea26f4e389447c9a9c1d7bcb10d779ca8bfecab09e461853553b"} Sep 17 07:49:19.680954 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:19.680922 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" event={"ID":"7868106b-568f-45a7-83cf-222105046dc8","Type":"ContainerStarted","Data":"263f6017eca94f40bdcc54bfcfb2673282084d3878f7d33ada22aac382b8b122"} Sep 17 07:49:19.680954 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:19.680944 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" event={"ID":"7868106b-568f-45a7-83cf-222105046dc8","Type":"ContainerStarted","Data":"d1d745eb1d05cccb90bf67133c8fe8f6c60f7239ef3fd5660ca4972615aed7ff"} Sep 17 07:49:19.681596 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:19.680962 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" event={"ID":"7868106b-568f-45a7-83cf-222105046dc8","Type":"ContainerStarted","Data":"6504b75a780d4daf67cb70694bbc7a7637307849972e067fa27dfc4574db37c8"} Sep 17 07:49:20.487674 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:20.487582 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:20.487674 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:20.487649 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:20.487674 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:20.487670 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:20.488018 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:20.487688 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:20.490286 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:20.490164 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.16:8816: connect: connection refused" Sep 17 07:49:20.490286 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:20.490183 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.16:6571: connect: connection refused" Sep 17 07:49:20.490286 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:20.490220 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.16:8443: connect: connection refused" Sep 17 07:49:20.490286 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:20.490181 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.16:6567/health\": dial tcp 10.134.0.16:6567: connect: connection refused" Sep 17 07:49:22.399648 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:22.399592 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.15:8443: connect: connection refused" Sep 17 07:49:23.487121 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:23.487073 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.16:8816: connect: connection refused" Sep 17 07:49:23.487716 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:23.487073 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.16:6571: connect: connection refused" Sep 17 07:49:23.487716 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:23.487112 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.16:6567/health\": dial tcp 10.134.0.16:6567: connect: connection refused" Sep 17 07:49:23.487903 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:23.487136 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.16:8443: connect: connection refused" Sep 17 07:49:25.486859 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:25.486814 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:25.487386 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:25.486884 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:25.487386 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:25.486906 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:25.487386 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:25.486924 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:26.487179 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:26.487132 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.16:6567/health\": dial tcp 10.134.0.16:6567: connect: connection refused" Sep 17 07:49:26.487685 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:26.487197 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.16:8443: connect: connection refused" Sep 17 07:49:26.487685 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:26.487127 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.16:6571: connect: connection refused" Sep 17 07:49:26.487685 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:26.487345 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.16:8816: connect: connection refused" Sep 17 07:49:29.390338 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:29.390288 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.15:6567/health\": dial tcp 10.134.0.15:6567: connect: connection refused" Sep 17 07:49:29.390828 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:29.390468 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:49:29.436862 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:29.436802 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podStartSLOduration=25.436787005 podStartE2EDuration="25.436787005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:49:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:49:19.720473674 +0000 UTC m=+1277.910449619" watchObservedRunningTime="2026-09-17 07:49:29.436787005 +0000 UTC m=+1287.626762946" Sep 17 07:49:29.487974 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:29.487934 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:29.488196 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:29.488176 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:29.488530 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:29.488501 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:29.489067 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:29.489046 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:29.489172 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:29.489125 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:29.489215 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:29.489173 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:29.494043 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:29.493982 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:29.730117 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:29.730022 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:49:32.399287 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:32.399242 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.15:8443: connect: connection refused" Sep 17 07:49:32.399702 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:32.399377 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:49:34.747240 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.747208 2579 generic.go:356] "Generic (PLEG): container finished" podID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerID="6b2ad0593484d8189d9ae4a9019626608fe0cf4524bba8a9372ce00b42a539d9" exitCode=137 Sep 17 07:49:34.747240 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.747233 2579 generic.go:356] "Generic (PLEG): container finished" podID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerID="81d4c14d9ec9baa883a64a1129aa1ddb63b33aeb1dacea038d3c4c76e95b54b3" exitCode=137 Sep 17 07:49:34.747763 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.747272 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" event={"ID":"80313f99-034a-4604-a6ef-bd9cc8efea5d","Type":"ContainerDied","Data":"6b2ad0593484d8189d9ae4a9019626608fe0cf4524bba8a9372ce00b42a539d9"} Sep 17 07:49:34.747763 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.747313 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" event={"ID":"80313f99-034a-4604-a6ef-bd9cc8efea5d","Type":"ContainerDied","Data":"81d4c14d9ec9baa883a64a1129aa1ddb63b33aeb1dacea038d3c4c76e95b54b3"} Sep 17 07:49:34.790332 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.790298 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:49:34.859791 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.859755 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-ui-tls\") pod \"80313f99-034a-4604-a6ef-bd9cc8efea5d\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " Sep 17 07:49:34.859791 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.859797 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/80313f99-034a-4604-a6ef-bd9cc8efea5d-feast-data\") pod \"80313f99-034a-4604-a6ef-bd9cc8efea5d\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " Sep 17 07:49:34.860097 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.859820 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-registry-tls\") pod \"80313f99-034a-4604-a6ef-bd9cc8efea5d\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " Sep 17 07:49:34.860097 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.859847 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-online-tls\") pod \"80313f99-034a-4604-a6ef-bd9cc8efea5d\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " Sep 17 07:49:34.860097 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.860070 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xk6z\" (UniqueName: \"kubernetes.io/projected/80313f99-034a-4604-a6ef-bd9cc8efea5d-kube-api-access-6xk6z\") pod \"80313f99-034a-4604-a6ef-bd9cc8efea5d\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " Sep 17 07:49:34.860267 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.860136 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-offline-tls\") pod \"80313f99-034a-4604-a6ef-bd9cc8efea5d\" (UID: \"80313f99-034a-4604-a6ef-bd9cc8efea5d\") " Sep 17 07:49:34.860455 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.860417 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80313f99-034a-4604-a6ef-bd9cc8efea5d-feast-data" (OuterVolumeSpecName: "feast-data") pod "80313f99-034a-4604-a6ef-bd9cc8efea5d" (UID: "80313f99-034a-4604-a6ef-bd9cc8efea5d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:49:34.862272 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.862247 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "80313f99-034a-4604-a6ef-bd9cc8efea5d" (UID: "80313f99-034a-4604-a6ef-bd9cc8efea5d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:49:34.862378 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.862301 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "80313f99-034a-4604-a6ef-bd9cc8efea5d" (UID: "80313f99-034a-4604-a6ef-bd9cc8efea5d"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:49:34.862378 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.862310 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-online-tls" (OuterVolumeSpecName: "online-tls") pod "80313f99-034a-4604-a6ef-bd9cc8efea5d" (UID: "80313f99-034a-4604-a6ef-bd9cc8efea5d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:49:34.862553 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.862531 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80313f99-034a-4604-a6ef-bd9cc8efea5d-kube-api-access-6xk6z" (OuterVolumeSpecName: "kube-api-access-6xk6z") pod "80313f99-034a-4604-a6ef-bd9cc8efea5d" (UID: "80313f99-034a-4604-a6ef-bd9cc8efea5d"). InnerVolumeSpecName "kube-api-access-6xk6z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:49:34.862613 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.862597 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "80313f99-034a-4604-a6ef-bd9cc8efea5d" (UID: "80313f99-034a-4604-a6ef-bd9cc8efea5d"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:49:34.961201 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.961111 2579 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-online-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:49:34.961201 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.961142 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xk6z\" (UniqueName: \"kubernetes.io/projected/80313f99-034a-4604-a6ef-bd9cc8efea5d-kube-api-access-6xk6z\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:49:34.961201 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.961153 2579 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-offline-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:49:34.961201 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.961161 2579 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-ui-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:49:34.961201 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.961169 2579 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/80313f99-034a-4604-a6ef-bd9cc8efea5d-feast-data\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:49:34.961201 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:34.961177 2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/80313f99-034a-4604-a6ef-bd9cc8efea5d-registry-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:49:35.754774 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:35.754742 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" event={"ID":"80313f99-034a-4604-a6ef-bd9cc8efea5d","Type":"ContainerDied","Data":"3b49b9029fccc3ec8485e8fd1ab57d767e54997ff65ba2f13bc8f3a0f702ca75"} Sep 17 07:49:35.755261 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:35.754788 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn" Sep 17 07:49:35.755261 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:35.754803 2579 scope.go:117] "RemoveContainer" containerID="20ab419993260047051a2551c3fe5a0bbd72b0c63512582c6960dfaac439ad9b" Sep 17 07:49:35.763695 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:35.763673 2579 scope.go:117] "RemoveContainer" containerID="6b2ad0593484d8189d9ae4a9019626608fe0cf4524bba8a9372ce00b42a539d9" Sep 17 07:49:35.770863 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:35.770844 2579 scope.go:117] "RemoveContainer" containerID="e3621204dca7d62be797204e282ba62b0ac51ba5e3fbb510db9d2b4321331cb1" Sep 17 07:49:35.778104 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:35.778077 2579 scope.go:117] "RemoveContainer" containerID="81d4c14d9ec9baa883a64a1129aa1ddb63b33aeb1dacea038d3c4c76e95b54b3" Sep 17 07:49:35.785652 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:35.785634 2579 scope.go:117] "RemoveContainer" containerID="3ce6a8fdb43e4602b59d3b61cced8eaab912872f716460e953271c92276ed34b" Sep 17 07:49:35.792584 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:35.792569 2579 scope.go:117] "RemoveContainer" containerID="ea436bbf3a0227ac61934c7b54d88045738377f5d6f7bef0d10b60f96261547a" Sep 17 07:49:35.800934 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:35.800908 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn"] Sep 17 07:49:35.803522 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:35.803502 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-6h6qn"] Sep 17 07:49:36.310284 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:49:36.310251 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" path="/var/lib/kubelet/pods/80313f99-034a-4604-a6ef-bd9cc8efea5d/volumes" Sep 17 07:50:31.765261 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.765174 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl"] Sep 17 07:50:31.765702 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.765423 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="ui" Sep 17 07:50:31.765702 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.765434 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="ui" Sep 17 07:50:31.765702 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.765443 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="feast-init" Sep 17 07:50:31.765702 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.765448 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="feast-init" Sep 17 07:50:31.765702 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.765454 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="online" Sep 17 07:50:31.765702 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.765460 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="online" Sep 17 07:50:31.765702 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.765477 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="feast-apply" Sep 17 07:50:31.765702 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.765482 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="feast-apply" Sep 17 07:50:31.765702 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.765491 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="registry" Sep 17 07:50:31.765702 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.765496 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="registry" Sep 17 07:50:31.765702 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.765502 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="offline" Sep 17 07:50:31.765702 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.765507 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="offline" Sep 17 07:50:31.765702 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.765563 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="registry" Sep 17 07:50:31.765702 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.765571 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="online" Sep 17 07:50:31.765702 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.765577 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="offline" Sep 17 07:50:31.765702 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.765583 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="80313f99-034a-4604-a6ef-bd9cc8efea5d" containerName="ui" Sep 17 07:50:31.768656 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.768637 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:31.774342 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.774315 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 17 07:50:31.774503 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.774368 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 17 07:50:31.774503 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.774395 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 17 07:50:31.774503 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.774408 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 17 07:50:31.774503 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.774478 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 17 07:50:31.775071 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.774727 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 17 07:50:31.775071 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.774773 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-5hprp\"" Sep 17 07:50:31.789424 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.789392 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl"] Sep 17 07:50:31.860510 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.860467 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xcltl\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:31.860700 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.860517 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xcltl\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:31.860700 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.860617 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/09e00257-b28b-4c17-8ac5-fa78bb5087a2-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xcltl\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:31.860700 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.860652 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xcltl\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:31.860700 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.860679 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/09e00257-b28b-4c17-8ac5-fa78bb5087a2-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xcltl\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:31.860844 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.860702 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67kv7\" (UniqueName: \"kubernetes.io/projected/09e00257-b28b-4c17-8ac5-fa78bb5087a2-kube-api-access-67kv7\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xcltl\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:31.961672 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.961633 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xcltl\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:31.961672 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.961678 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xcltl\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:31.962024 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.961716 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/09e00257-b28b-4c17-8ac5-fa78bb5087a2-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xcltl\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:31.962024 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.961742 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xcltl\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:31.962024 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.961780 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/09e00257-b28b-4c17-8ac5-fa78bb5087a2-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xcltl\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:31.962024 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:50:31.961800 2579 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Sep 17 07:50:31.962024 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.961803 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67kv7\" (UniqueName: \"kubernetes.io/projected/09e00257-b28b-4c17-8ac5-fa78bb5087a2-kube-api-access-67kv7\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xcltl\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:31.962024 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:50:31.961869 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-online-tls podName:09e00257-b28b-4c17-8ac5-fa78bb5087a2 nodeName:}" failed. No retries permitted until 2026-09-17 07:50:32.461848133 +0000 UTC m=+1350.651824070 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-online-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-xcltl" (UID: "09e00257-b28b-4c17-8ac5-fa78bb5087a2") : secret "feast-simple-feast-remote-setup-online-tls" not found Sep 17 07:50:31.962024 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:50:31.961889 2579 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 17 07:50:31.962024 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:50:31.961945 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-ui-tls podName:09e00257-b28b-4c17-8ac5-fa78bb5087a2 nodeName:}" failed. No retries permitted until 2026-09-17 07:50:32.461930595 +0000 UTC m=+1350.651906519 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-ui-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-xcltl" (UID: "09e00257-b28b-4c17-8ac5-fa78bb5087a2") : secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 17 07:50:31.962384 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.962230 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/09e00257-b28b-4c17-8ac5-fa78bb5087a2-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xcltl\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:31.962539 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.962521 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/09e00257-b28b-4c17-8ac5-fa78bb5087a2-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xcltl\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:31.964217 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.964198 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xcltl\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:31.974537 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:31.974509 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67kv7\" (UniqueName: \"kubernetes.io/projected/09e00257-b28b-4c17-8ac5-fa78bb5087a2-kube-api-access-67kv7\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xcltl\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:32.466036 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:32.465973 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xcltl\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:32.466210 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:32.466068 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xcltl\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:32.468437 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:32.468404 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xcltl\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:32.468437 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:32.468436 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xcltl\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:32.677163 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:32.677127 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:50:32.967577 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:32.967540 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl"] Sep 17 07:50:32.971951 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:50:32.971920 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09e00257_b28b_4c17_8ac5_fa78bb5087a2.slice/crio-80f467e183e0da51bfdd9dd447edc03c90d884374855fda82e0fb4c05078cd2e WatchSource:0}: Error finding container 80f467e183e0da51bfdd9dd447edc03c90d884374855fda82e0fb4c05078cd2e: Status 404 returned error can't find the container with id 80f467e183e0da51bfdd9dd447edc03c90d884374855fda82e0fb4c05078cd2e Sep 17 07:50:33.953312 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:33.953274 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" event={"ID":"09e00257-b28b-4c17-8ac5-fa78bb5087a2","Type":"ContainerStarted","Data":"5b9a9590dad418df15461a2ada33c2bf1e4e46b2873e5b26fa8c010c83cfecae"} Sep 17 07:50:33.953563 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:33.953328 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" event={"ID":"09e00257-b28b-4c17-8ac5-fa78bb5087a2","Type":"ContainerStarted","Data":"80f467e183e0da51bfdd9dd447edc03c90d884374855fda82e0fb4c05078cd2e"} Sep 17 07:50:38.978151 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:38.978116 2579 generic.go:356] "Generic (PLEG): container finished" podID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerID="5b9a9590dad418df15461a2ada33c2bf1e4e46b2873e5b26fa8c010c83cfecae" exitCode=0 Sep 17 07:50:38.978549 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:38.978168 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" event={"ID":"09e00257-b28b-4c17-8ac5-fa78bb5087a2","Type":"ContainerDied","Data":"5b9a9590dad418df15461a2ada33c2bf1e4e46b2873e5b26fa8c010c83cfecae"} Sep 17 07:50:39.983738 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:50:39.983705 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" event={"ID":"09e00257-b28b-4c17-8ac5-fa78bb5087a2","Type":"ContainerStarted","Data":"c12200630c133c2e1925d006c23af92347320697c90c415ac5b47814692e8c1f"} Sep 17 07:51:08.086709 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:08.086669 2579 generic.go:356] "Generic (PLEG): container finished" podID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerID="c12200630c133c2e1925d006c23af92347320697c90c415ac5b47814692e8c1f" exitCode=0 Sep 17 07:51:08.087146 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:08.086745 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" event={"ID":"09e00257-b28b-4c17-8ac5-fa78bb5087a2","Type":"ContainerDied","Data":"c12200630c133c2e1925d006c23af92347320697c90c415ac5b47814692e8c1f"} Sep 17 07:51:09.101919 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:09.101881 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" event={"ID":"09e00257-b28b-4c17-8ac5-fa78bb5087a2","Type":"ContainerStarted","Data":"532e0422457b9de01d3443764768aa87166e7d7e1213b3ceb0f48e672c20d4f2"} Sep 17 07:51:09.102441 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:09.101932 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" event={"ID":"09e00257-b28b-4c17-8ac5-fa78bb5087a2","Type":"ContainerStarted","Data":"c29a07601499f8611bafeee478a1ff3e0f94242fc4aa6004ab9e8877916072ec"} Sep 17 07:51:09.102441 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:09.101945 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" event={"ID":"09e00257-b28b-4c17-8ac5-fa78bb5087a2","Type":"ContainerStarted","Data":"10e244576afe512002d834d8b789e28702ee84cb9a64860ad8e35f0a93b1a3a1"} Sep 17 07:51:09.136140 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:09.136068 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" podStartSLOduration=38.136045061 podStartE2EDuration="38.136045061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:50:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:51:09.135174988 +0000 UTC m=+1387.325150930" watchObservedRunningTime="2026-09-17 07:51:09.136045061 +0000 UTC m=+1387.326021005" Sep 17 07:51:11.678278 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:11.678230 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:51:11.678766 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:11.678293 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:51:11.678766 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:11.678314 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:51:11.679721 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:11.679685 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.17:8816: connect: connection refused" Sep 17 07:51:11.679852 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:11.679714 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.17:8443: connect: connection refused" Sep 17 07:51:11.679852 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:11.679727 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.17:6567/health\": dial tcp 10.134.0.17:6567: connect: connection refused" Sep 17 07:51:12.677892 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:12.677851 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:51:12.678108 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:12.677950 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:51:12.678108 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:12.678012 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:51:14.677980 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:14.677934 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.17:8816: connect: connection refused" Sep 17 07:51:14.678543 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:14.677934 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.17:8443: connect: connection refused" Sep 17 07:51:14.678543 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:14.677946 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.17:6567/health\": dial tcp 10.134.0.17:6567: connect: connection refused" Sep 17 07:51:17.679146 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:17.679107 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:51:17.679636 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:17.679241 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:51:17.680028 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:17.679983 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:51:17.680140 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:17.680118 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:51:18.678275 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:18.678226 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.17:6567/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Sep 17 07:51:21.682140 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:21.682097 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.17:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 17 07:51:24.584695 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:24.584657 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:51:25.428029 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:51:25.427977 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:52:28.120963 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:28.120927 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl"] Sep 17 07:52:28.121564 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:28.121225 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="online" containerID="cri-o://10e244576afe512002d834d8b789e28702ee84cb9a64860ad8e35f0a93b1a3a1" gracePeriod=30 Sep 17 07:52:28.121564 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:28.121263 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="ui" containerID="cri-o://532e0422457b9de01d3443764768aa87166e7d7e1213b3ceb0f48e672c20d4f2" gracePeriod=30 Sep 17 07:52:28.121564 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:28.121284 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="offline" containerID="cri-o://c29a07601499f8611bafeee478a1ff3e0f94242fc4aa6004ab9e8877916072ec" gracePeriod=30 Sep 17 07:52:28.311862 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:28.311829 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs"] Sep 17 07:52:28.312512 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:28.312138 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="registry" containerID="cri-o://6504b75a780d4daf67cb70694bbc7a7637307849972e067fa27dfc4574db37c8" gracePeriod=30 Sep 17 07:52:28.312512 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:28.312176 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="offline" containerID="cri-o://263f6017eca94f40bdcc54bfcfb2673282084d3878f7d33ada22aac382b8b122" gracePeriod=30 Sep 17 07:52:28.312512 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:28.312212 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="ui" containerID="cri-o://98548dabb2ecea26f4e389447c9a9c1d7bcb10d779ca8bfecab09e461853553b" gracePeriod=30 Sep 17 07:52:28.312512 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:28.312297 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="online" containerID="cri-o://d1d745eb1d05cccb90bf67133c8fe8f6c60f7239ef3fd5660ca4972615aed7ff" gracePeriod=30 Sep 17 07:52:29.393783 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:29.393700 2579 generic.go:356] "Generic (PLEG): container finished" podID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerID="532e0422457b9de01d3443764768aa87166e7d7e1213b3ceb0f48e672c20d4f2" exitCode=0 Sep 17 07:52:29.393783 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:29.393725 2579 generic.go:356] "Generic (PLEG): container finished" podID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerID="10e244576afe512002d834d8b789e28702ee84cb9a64860ad8e35f0a93b1a3a1" exitCode=0 Sep 17 07:52:29.394255 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:29.393775 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" event={"ID":"09e00257-b28b-4c17-8ac5-fa78bb5087a2","Type":"ContainerDied","Data":"532e0422457b9de01d3443764768aa87166e7d7e1213b3ceb0f48e672c20d4f2"} Sep 17 07:52:29.394255 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:29.393817 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" event={"ID":"09e00257-b28b-4c17-8ac5-fa78bb5087a2","Type":"ContainerDied","Data":"10e244576afe512002d834d8b789e28702ee84cb9a64860ad8e35f0a93b1a3a1"} Sep 17 07:52:29.396582 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:29.396559 2579 generic.go:356] "Generic (PLEG): container finished" podID="7868106b-568f-45a7-83cf-222105046dc8" containerID="98548dabb2ecea26f4e389447c9a9c1d7bcb10d779ca8bfecab09e461853553b" exitCode=0 Sep 17 07:52:29.396582 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:29.396578 2579 generic.go:356] "Generic (PLEG): container finished" podID="7868106b-568f-45a7-83cf-222105046dc8" containerID="d1d745eb1d05cccb90bf67133c8fe8f6c60f7239ef3fd5660ca4972615aed7ff" exitCode=0 Sep 17 07:52:29.396755 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:29.396607 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" event={"ID":"7868106b-568f-45a7-83cf-222105046dc8","Type":"ContainerDied","Data":"98548dabb2ecea26f4e389447c9a9c1d7bcb10d779ca8bfecab09e461853553b"} Sep 17 07:52:29.396755 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:29.396639 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" event={"ID":"7868106b-568f-45a7-83cf-222105046dc8","Type":"ContainerDied","Data":"d1d745eb1d05cccb90bf67133c8fe8f6c60f7239ef3fd5660ca4972615aed7ff"} Sep 17 07:52:29.489447 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:29.489405 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.16:8443: connect: connection refused" Sep 17 07:52:29.727712 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:29.727601 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.16:6567/health\": dial tcp 10.134.0.16:6567: connect: connection refused" Sep 17 07:52:34.585382 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:34.585331 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.17:6567/health\": dial tcp 10.134.0.17:6567: connect: connection refused" Sep 17 07:52:37.680098 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:37.680046 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.17:8443: connect: connection refused" Sep 17 07:52:39.489280 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:39.489225 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.16:8443: connect: connection refused" Sep 17 07:52:39.727666 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:39.727614 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.16:6567/health\": dial tcp 10.134.0.16:6567: connect: connection refused" Sep 17 07:52:44.585366 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:44.585319 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.17:6567/health\": dial tcp 10.134.0.17:6567: connect: connection refused" Sep 17 07:52:47.680628 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:47.680589 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.17:8443: connect: connection refused" Sep 17 07:52:49.489270 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:49.489213 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.16:8443: connect: connection refused" Sep 17 07:52:49.489702 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:49.489366 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:52:49.727223 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:49.727184 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.16:6567/health\": dial tcp 10.134.0.16:6567: connect: connection refused" Sep 17 07:52:49.727407 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:49.727344 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:52:54.585477 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:54.585437 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.17:6567/health\": dial tcp 10.134.0.17:6567: connect: connection refused" Sep 17 07:52:54.585852 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:54.585556 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:52:57.680211 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:57.680164 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.17:8443: connect: connection refused" Sep 17 07:52:57.680603 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:57.680275 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:52:58.500712 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:58.500630 2579 generic.go:356] "Generic (PLEG): container finished" podID="7868106b-568f-45a7-83cf-222105046dc8" containerID="263f6017eca94f40bdcc54bfcfb2673282084d3878f7d33ada22aac382b8b122" exitCode=137 Sep 17 07:52:58.500712 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:58.500655 2579 generic.go:356] "Generic (PLEG): container finished" podID="7868106b-568f-45a7-83cf-222105046dc8" containerID="6504b75a780d4daf67cb70694bbc7a7637307849972e067fa27dfc4574db37c8" exitCode=137 Sep 17 07:52:58.500880 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:58.500705 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" event={"ID":"7868106b-568f-45a7-83cf-222105046dc8","Type":"ContainerDied","Data":"263f6017eca94f40bdcc54bfcfb2673282084d3878f7d33ada22aac382b8b122"} Sep 17 07:52:58.500880 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:58.500761 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" event={"ID":"7868106b-568f-45a7-83cf-222105046dc8","Type":"ContainerDied","Data":"6504b75a780d4daf67cb70694bbc7a7637307849972e067fa27dfc4574db37c8"} Sep 17 07:52:58.503334 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:58.503312 2579 generic.go:356] "Generic (PLEG): container finished" podID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerID="c29a07601499f8611bafeee478a1ff3e0f94242fc4aa6004ab9e8877916072ec" exitCode=137 Sep 17 07:52:58.503439 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:58.503353 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" event={"ID":"09e00257-b28b-4c17-8ac5-fa78bb5087a2","Type":"ContainerDied","Data":"c29a07601499f8611bafeee478a1ff3e0f94242fc4aa6004ab9e8877916072ec"} Sep 17 07:52:58.760540 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:58.760515 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:52:58.920467 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:58.920439 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-offline-tls\") pod \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " Sep 17 07:52:58.920629 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:58.920487 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-online-tls\") pod \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " Sep 17 07:52:58.920629 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:58.920503 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-ui-tls\") pod \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " Sep 17 07:52:58.920629 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:58.920522 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67kv7\" (UniqueName: \"kubernetes.io/projected/09e00257-b28b-4c17-8ac5-fa78bb5087a2-kube-api-access-67kv7\") pod \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " Sep 17 07:52:58.920629 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:58.920555 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/09e00257-b28b-4c17-8ac5-fa78bb5087a2-feast-data\") pod \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " Sep 17 07:52:58.920629 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:58.920579 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/09e00257-b28b-4c17-8ac5-fa78bb5087a2-registry-tls\") pod \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\" (UID: \"09e00257-b28b-4c17-8ac5-fa78bb5087a2\") " Sep 17 07:52:58.921188 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:58.921157 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09e00257-b28b-4c17-8ac5-fa78bb5087a2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "09e00257-b28b-4c17-8ac5-fa78bb5087a2" (UID: "09e00257-b28b-4c17-8ac5-fa78bb5087a2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:52:58.921443 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:58.921404 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09e00257-b28b-4c17-8ac5-fa78bb5087a2-feast-data" (OuterVolumeSpecName: "feast-data") pod "09e00257-b28b-4c17-8ac5-fa78bb5087a2" (UID: "09e00257-b28b-4c17-8ac5-fa78bb5087a2"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:52:58.922926 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:58.922890 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09e00257-b28b-4c17-8ac5-fa78bb5087a2-kube-api-access-67kv7" (OuterVolumeSpecName: "kube-api-access-67kv7") pod "09e00257-b28b-4c17-8ac5-fa78bb5087a2" (UID: "09e00257-b28b-4c17-8ac5-fa78bb5087a2"). InnerVolumeSpecName "kube-api-access-67kv7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:52:58.923080 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:58.922954 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-online-tls" (OuterVolumeSpecName: "online-tls") pod "09e00257-b28b-4c17-8ac5-fa78bb5087a2" (UID: "09e00257-b28b-4c17-8ac5-fa78bb5087a2"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:52:58.923139 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:58.923112 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "09e00257-b28b-4c17-8ac5-fa78bb5087a2" (UID: "09e00257-b28b-4c17-8ac5-fa78bb5087a2"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:52:58.923173 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:58.923142 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "09e00257-b28b-4c17-8ac5-fa78bb5087a2" (UID: "09e00257-b28b-4c17-8ac5-fa78bb5087a2"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:52:58.959175 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:58.959156 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:52:59.021882 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.021802 2579 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-offline-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:52:59.021882 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.021834 2579 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-online-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:52:59.021882 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.021847 2579 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/09e00257-b28b-4c17-8ac5-fa78bb5087a2-ui-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:52:59.021882 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.021859 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67kv7\" (UniqueName: \"kubernetes.io/projected/09e00257-b28b-4c17-8ac5-fa78bb5087a2-kube-api-access-67kv7\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:52:59.021882 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.021868 2579 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/09e00257-b28b-4c17-8ac5-fa78bb5087a2-feast-data\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:52:59.021882 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.021876 2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/09e00257-b28b-4c17-8ac5-fa78bb5087a2-registry-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:52:59.122986 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.122952 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-registry-tls\") pod \"7868106b-568f-45a7-83cf-222105046dc8\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " Sep 17 07:52:59.122986 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.122990 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-ui-tls\") pod \"7868106b-568f-45a7-83cf-222105046dc8\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " Sep 17 07:52:59.123241 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.123046 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9lmh\" (UniqueName: \"kubernetes.io/projected/7868106b-568f-45a7-83cf-222105046dc8-kube-api-access-k9lmh\") pod \"7868106b-568f-45a7-83cf-222105046dc8\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " Sep 17 07:52:59.123241 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.123072 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-online-tls\") pod \"7868106b-568f-45a7-83cf-222105046dc8\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " Sep 17 07:52:59.123241 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.123094 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7868106b-568f-45a7-83cf-222105046dc8-feast-data\") pod \"7868106b-568f-45a7-83cf-222105046dc8\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " Sep 17 07:52:59.123241 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.123117 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-offline-tls\") pod \"7868106b-568f-45a7-83cf-222105046dc8\" (UID: \"7868106b-568f-45a7-83cf-222105046dc8\") " Sep 17 07:52:59.123663 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.123635 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7868106b-568f-45a7-83cf-222105046dc8-feast-data" (OuterVolumeSpecName: "feast-data") pod "7868106b-568f-45a7-83cf-222105046dc8" (UID: "7868106b-568f-45a7-83cf-222105046dc8"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:52:59.125101 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.125080 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7868106b-568f-45a7-83cf-222105046dc8-kube-api-access-k9lmh" (OuterVolumeSpecName: "kube-api-access-k9lmh") pod "7868106b-568f-45a7-83cf-222105046dc8" (UID: "7868106b-568f-45a7-83cf-222105046dc8"). InnerVolumeSpecName "kube-api-access-k9lmh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:52:59.125700 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.125671 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-online-tls" (OuterVolumeSpecName: "online-tls") pod "7868106b-568f-45a7-83cf-222105046dc8" (UID: "7868106b-568f-45a7-83cf-222105046dc8"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:52:59.125776 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.125686 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "7868106b-568f-45a7-83cf-222105046dc8" (UID: "7868106b-568f-45a7-83cf-222105046dc8"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:52:59.125776 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.125705 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "7868106b-568f-45a7-83cf-222105046dc8" (UID: "7868106b-568f-45a7-83cf-222105046dc8"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:52:59.125776 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.125720 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7868106b-568f-45a7-83cf-222105046dc8" (UID: "7868106b-568f-45a7-83cf-222105046dc8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:52:59.224100 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.224049 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9lmh\" (UniqueName: \"kubernetes.io/projected/7868106b-568f-45a7-83cf-222105046dc8-kube-api-access-k9lmh\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:52:59.224100 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.224093 2579 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-online-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:52:59.224100 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.224102 2579 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7868106b-568f-45a7-83cf-222105046dc8-feast-data\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:52:59.224100 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.224110 2579 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-offline-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:52:59.224100 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.224120 2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-registry-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:52:59.224405 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.224128 2579 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7868106b-568f-45a7-83cf-222105046dc8-ui-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:52:59.509607 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.509574 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" Sep 17 07:52:59.509607 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.509580 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl" event={"ID":"09e00257-b28b-4c17-8ac5-fa78bb5087a2","Type":"ContainerDied","Data":"80f467e183e0da51bfdd9dd447edc03c90d884374855fda82e0fb4c05078cd2e"} Sep 17 07:52:59.509838 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.509653 2579 scope.go:117] "RemoveContainer" containerID="532e0422457b9de01d3443764768aa87166e7d7e1213b3ceb0f48e672c20d4f2" Sep 17 07:52:59.512691 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.512663 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" event={"ID":"7868106b-568f-45a7-83cf-222105046dc8","Type":"ContainerDied","Data":"0af56bf53dc600a5701db5f8d90fb88cc13f584f8d6a01ec52d1f68c8bba4541"} Sep 17 07:52:59.512821 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.512692 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs" Sep 17 07:52:59.519195 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.519158 2579 scope.go:117] "RemoveContainer" containerID="c29a07601499f8611bafeee478a1ff3e0f94242fc4aa6004ab9e8877916072ec" Sep 17 07:52:59.527385 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.527369 2579 scope.go:117] "RemoveContainer" containerID="10e244576afe512002d834d8b789e28702ee84cb9a64860ad8e35f0a93b1a3a1" Sep 17 07:52:59.534478 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.534459 2579 scope.go:117] "RemoveContainer" containerID="c12200630c133c2e1925d006c23af92347320697c90c415ac5b47814692e8c1f" Sep 17 07:52:59.535728 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.535706 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl"] Sep 17 07:52:59.541372 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.541349 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xcltl"] Sep 17 07:52:59.542172 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.542148 2579 scope.go:117] "RemoveContainer" containerID="5b9a9590dad418df15461a2ada33c2bf1e4e46b2873e5b26fa8c010c83cfecae" Sep 17 07:52:59.549280 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.549258 2579 scope.go:117] "RemoveContainer" containerID="98548dabb2ecea26f4e389447c9a9c1d7bcb10d779ca8bfecab09e461853553b" Sep 17 07:52:59.554349 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.554329 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs"] Sep 17 07:52:59.556766 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.556749 2579 scope.go:117] "RemoveContainer" containerID="263f6017eca94f40bdcc54bfcfb2673282084d3878f7d33ada22aac382b8b122" Sep 17 07:52:59.558695 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.558677 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9ckrs"] Sep 17 07:52:59.564456 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.564440 2579 scope.go:117] "RemoveContainer" containerID="d1d745eb1d05cccb90bf67133c8fe8f6c60f7239ef3fd5660ca4972615aed7ff" Sep 17 07:52:59.571265 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.571247 2579 scope.go:117] "RemoveContainer" containerID="6504b75a780d4daf67cb70694bbc7a7637307849972e067fa27dfc4574db37c8" Sep 17 07:52:59.578100 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.578073 2579 scope.go:117] "RemoveContainer" containerID="49919e4178909120d54733febfb64509e521b85718180d9caf9b557a20da4c96" Sep 17 07:52:59.584627 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:52:59.584611 2579 scope.go:117] "RemoveContainer" containerID="abe5004ff4a296b7e619c1995a76d38b2f8a3ba24728df690f78bf223d699c7b" Sep 17 07:53:00.310365 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:00.310335 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" path="/var/lib/kubelet/pods/09e00257-b28b-4c17-8ac5-fa78bb5087a2/volumes" Sep 17 07:53:00.310856 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:00.310843 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7868106b-568f-45a7-83cf-222105046dc8" path="/var/lib/kubelet/pods/7868106b-568f-45a7-83cf-222105046dc8/volumes" Sep 17 07:53:17.568320 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:17.568284 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-pr9xn"] Sep 17 07:53:17.568818 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:17.568547 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pr9xn" podUID="306697f1-0bc7-43db-bc6b-97230e0e7917" containerName="manager" containerID="cri-o://c51de627948cfbb0f7999eaea978bd3e659c2f3bd80475b9ac9aee5d50043ff3" gracePeriod=10 Sep 17 07:53:17.809469 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:17.809442 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pr9xn" Sep 17 07:53:17.947912 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:17.947812 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zt2hb\" (UniqueName: \"kubernetes.io/projected/306697f1-0bc7-43db-bc6b-97230e0e7917-kube-api-access-zt2hb\") pod \"306697f1-0bc7-43db-bc6b-97230e0e7917\" (UID: \"306697f1-0bc7-43db-bc6b-97230e0e7917\") " Sep 17 07:53:17.950015 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:17.949980 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/306697f1-0bc7-43db-bc6b-97230e0e7917-kube-api-access-zt2hb" (OuterVolumeSpecName: "kube-api-access-zt2hb") pod "306697f1-0bc7-43db-bc6b-97230e0e7917" (UID: "306697f1-0bc7-43db-bc6b-97230e0e7917"). InnerVolumeSpecName "kube-api-access-zt2hb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:53:18.048250 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:18.048220 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zt2hb\" (UniqueName: \"kubernetes.io/projected/306697f1-0bc7-43db-bc6b-97230e0e7917-kube-api-access-zt2hb\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:53:18.579919 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:18.579833 2579 generic.go:356] "Generic (PLEG): container finished" podID="306697f1-0bc7-43db-bc6b-97230e0e7917" containerID="c51de627948cfbb0f7999eaea978bd3e659c2f3bd80475b9ac9aee5d50043ff3" exitCode=0 Sep 17 07:53:18.579919 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:18.579876 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pr9xn" Sep 17 07:53:18.579919 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:18.579896 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pr9xn" event={"ID":"306697f1-0bc7-43db-bc6b-97230e0e7917","Type":"ContainerDied","Data":"c51de627948cfbb0f7999eaea978bd3e659c2f3bd80475b9ac9aee5d50043ff3"} Sep 17 07:53:18.580430 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:18.579948 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-pr9xn" event={"ID":"306697f1-0bc7-43db-bc6b-97230e0e7917","Type":"ContainerDied","Data":"6c5ae22fa8485e5a822079ae7ed733b4b646d6acf39119a4601a9232d44e6a4e"} Sep 17 07:53:18.580430 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:18.579978 2579 scope.go:117] "RemoveContainer" containerID="c51de627948cfbb0f7999eaea978bd3e659c2f3bd80475b9ac9aee5d50043ff3" Sep 17 07:53:18.587387 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:18.587370 2579 scope.go:117] "RemoveContainer" containerID="c51de627948cfbb0f7999eaea978bd3e659c2f3bd80475b9ac9aee5d50043ff3" Sep 17 07:53:18.587652 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:53:18.587625 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c51de627948cfbb0f7999eaea978bd3e659c2f3bd80475b9ac9aee5d50043ff3\": container with ID starting with c51de627948cfbb0f7999eaea978bd3e659c2f3bd80475b9ac9aee5d50043ff3 not found: ID does not exist" containerID="c51de627948cfbb0f7999eaea978bd3e659c2f3bd80475b9ac9aee5d50043ff3" Sep 17 07:53:18.587706 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:18.587659 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c51de627948cfbb0f7999eaea978bd3e659c2f3bd80475b9ac9aee5d50043ff3"} err="failed to get container status \"c51de627948cfbb0f7999eaea978bd3e659c2f3bd80475b9ac9aee5d50043ff3\": rpc error: code = NotFound desc = could not find container \"c51de627948cfbb0f7999eaea978bd3e659c2f3bd80475b9ac9aee5d50043ff3\": container with ID starting with c51de627948cfbb0f7999eaea978bd3e659c2f3bd80475b9ac9aee5d50043ff3 not found: ID does not exist" Sep 17 07:53:18.602126 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:18.602095 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-pr9xn"] Sep 17 07:53:18.609325 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:18.609298 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-pr9xn"] Sep 17 07:53:20.310244 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:20.310207 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="306697f1-0bc7-43db-bc6b-97230e0e7917" path="/var/lib/kubelet/pods/306697f1-0bc7-43db-bc6b-97230e0e7917/volumes" Sep 17 07:53:23.860318 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860288 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-nz4xx"] Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860536 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="online" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860548 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="online" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860558 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="feast-init" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860564 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="feast-init" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860571 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="ui" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860576 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="ui" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860584 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="feast-apply" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860589 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="feast-apply" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860596 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="offline" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860601 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="offline" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860609 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="306697f1-0bc7-43db-bc6b-97230e0e7917" containerName="manager" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860613 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="306697f1-0bc7-43db-bc6b-97230e0e7917" containerName="manager" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860621 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="offline" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860626 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="offline" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860633 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="ui" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860638 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="ui" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860644 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="online" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860649 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="online" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860658 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="registry" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860664 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="registry" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860669 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="feast-init" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860674 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="feast-init" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860679 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="feast-apply" Sep 17 07:53:23.860707 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860684 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="feast-apply" Sep 17 07:53:23.861445 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860727 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="offline" Sep 17 07:53:23.861445 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860735 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="offline" Sep 17 07:53:23.861445 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860741 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="306697f1-0bc7-43db-bc6b-97230e0e7917" containerName="manager" Sep 17 07:53:23.861445 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860747 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="registry" Sep 17 07:53:23.861445 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860752 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="online" Sep 17 07:53:23.861445 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860756 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="online" Sep 17 07:53:23.861445 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860762 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7868106b-568f-45a7-83cf-222105046dc8" containerName="ui" Sep 17 07:53:23.861445 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.860767 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="09e00257-b28b-4c17-8ac5-fa78bb5087a2" containerName="ui" Sep 17 07:53:23.864797 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.864777 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-nz4xx" Sep 17 07:53:23.871817 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.871801 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-zjrrp\"" Sep 17 07:53:23.872721 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.872705 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 17 07:53:23.872769 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.872738 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 17 07:53:23.888787 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.888753 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5x8k\" (UniqueName: \"kubernetes.io/projected/1557e14c-cdac-4031-9740-23c7b186d45a-kube-api-access-n5x8k\") pod \"feast-operator-controller-manager-689b877b4b-nz4xx\" (UID: \"1557e14c-cdac-4031-9740-23c7b186d45a\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-nz4xx" Sep 17 07:53:23.890511 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.890491 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-nz4xx"] Sep 17 07:53:23.990034 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:23.989975 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5x8k\" (UniqueName: \"kubernetes.io/projected/1557e14c-cdac-4031-9740-23c7b186d45a-kube-api-access-n5x8k\") pod \"feast-operator-controller-manager-689b877b4b-nz4xx\" (UID: \"1557e14c-cdac-4031-9740-23c7b186d45a\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-nz4xx" Sep 17 07:53:24.008831 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:24.008799 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5x8k\" (UniqueName: \"kubernetes.io/projected/1557e14c-cdac-4031-9740-23c7b186d45a-kube-api-access-n5x8k\") pod \"feast-operator-controller-manager-689b877b4b-nz4xx\" (UID: \"1557e14c-cdac-4031-9740-23c7b186d45a\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-nz4xx" Sep 17 07:53:24.174129 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:24.174022 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-nz4xx" Sep 17 07:53:24.466079 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:24.466051 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-nz4xx"] Sep 17 07:53:24.467557 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:53:24.467529 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1557e14c_cdac_4031_9740_23c7b186d45a.slice/crio-07f738e45e3103254acdebdd78d619f28df85068afc67e151cd87d80e21a98ed WatchSource:0}: Error finding container 07f738e45e3103254acdebdd78d619f28df85068afc67e151cd87d80e21a98ed: Status 404 returned error can't find the container with id 07f738e45e3103254acdebdd78d619f28df85068afc67e151cd87d80e21a98ed Sep 17 07:53:24.601143 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:24.601108 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-nz4xx" event={"ID":"1557e14c-cdac-4031-9740-23c7b186d45a","Type":"ContainerStarted","Data":"f1ee05dbf1c77254c7ff046bbb42c4ec0710e995aceddf2fc7d9a65fd05b331c"} Sep 17 07:53:24.601300 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:24.601161 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-nz4xx" event={"ID":"1557e14c-cdac-4031-9740-23c7b186d45a","Type":"ContainerStarted","Data":"07f738e45e3103254acdebdd78d619f28df85068afc67e151cd87d80e21a98ed"} Sep 17 07:53:24.601300 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:24.601223 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-nz4xx" Sep 17 07:53:24.635544 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:24.635490 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-nz4xx" podStartSLOduration=1.635473739 podStartE2EDuration="1.635473739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:53:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:53:24.632158629 +0000 UTC m=+1522.822134571" watchObservedRunningTime="2026-09-17 07:53:24.635473739 +0000 UTC m=+1522.825449680" Sep 17 07:53:35.606914 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:35.606879 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-nz4xx" Sep 17 07:53:40.682435 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.682400 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t"] Sep 17 07:53:40.685463 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.685447 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:40.690239 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.690212 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 17 07:53:40.690239 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.690226 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 17 07:53:40.690432 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.690241 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 17 07:53:40.690432 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.690266 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 17 07:53:40.690432 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.690277 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 17 07:53:40.691008 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.690983 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 17 07:53:40.691008 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.691004 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-vzvbq\"" Sep 17 07:53:40.723075 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.723048 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t"] Sep 17 07:53:40.797959 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.797935 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-t542t\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:40.798095 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.797963 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-t542t\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:40.798095 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.797987 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-t542t\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:40.798095 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.798070 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-t542t\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:40.798246 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.798100 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt9tb\" (UniqueName: \"kubernetes.io/projected/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-kube-api-access-wt9tb\") pod \"feast-simple-feast-setup-57576b8fcf-t542t\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:40.798246 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.798132 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-t542t\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:40.899257 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.899215 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-t542t\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:40.899426 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.899264 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt9tb\" (UniqueName: \"kubernetes.io/projected/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-kube-api-access-wt9tb\") pod \"feast-simple-feast-setup-57576b8fcf-t542t\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:40.899426 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.899308 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-t542t\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:40.899426 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.899342 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-t542t\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:40.899426 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.899371 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-t542t\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:40.899426 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.899404 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-t542t\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:40.899675 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:53:40.899478 2579 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 17 07:53:40.899675 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:53:40.899563 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-online-tls podName:d4ca42a2-bd91-4860-9a42-a0e4815fb4da nodeName:}" failed. No retries permitted until 2026-09-17 07:53:41.399543783 +0000 UTC m=+1539.589519704 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-online-tls") pod "feast-simple-feast-setup-57576b8fcf-t542t" (UID: "d4ca42a2-bd91-4860-9a42-a0e4815fb4da") : secret "feast-simple-feast-setup-online-tls" not found Sep 17 07:53:40.899675 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.899618 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-t542t\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:40.901850 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.901827 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-t542t\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:40.901949 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.901878 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-t542t\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:40.902090 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.902071 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-t542t\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:40.912506 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:40.912482 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt9tb\" (UniqueName: \"kubernetes.io/projected/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-kube-api-access-wt9tb\") pod \"feast-simple-feast-setup-57576b8fcf-t542t\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:41.403092 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:41.403057 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-t542t\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:41.405365 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:41.405337 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-t542t\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:41.594307 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:41.594274 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:41.875850 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:41.875820 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t"] Sep 17 07:53:41.879357 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:53:41.879333 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4ca42a2_bd91_4860_9a42_a0e4815fb4da.slice/crio-7ed27c6212880d1df4d62abb86dff71f2058418cffa7e2dc69e2ed65ba4017de WatchSource:0}: Error finding container 7ed27c6212880d1df4d62abb86dff71f2058418cffa7e2dc69e2ed65ba4017de: Status 404 returned error can't find the container with id 7ed27c6212880d1df4d62abb86dff71f2058418cffa7e2dc69e2ed65ba4017de Sep 17 07:53:42.665409 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:42.665375 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" event={"ID":"d4ca42a2-bd91-4860-9a42-a0e4815fb4da","Type":"ContainerStarted","Data":"cdb26e98235ddc0251268d18bca73267d275e70575f58b7250fbe676529e7055"} Sep 17 07:53:42.665591 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:42.665421 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" event={"ID":"d4ca42a2-bd91-4860-9a42-a0e4815fb4da","Type":"ContainerStarted","Data":"7ed27c6212880d1df4d62abb86dff71f2058418cffa7e2dc69e2ed65ba4017de"} Sep 17 07:53:47.684525 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:47.684492 2579 generic.go:356] "Generic (PLEG): container finished" podID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerID="cdb26e98235ddc0251268d18bca73267d275e70575f58b7250fbe676529e7055" exitCode=0 Sep 17 07:53:47.684920 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:47.684570 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" event={"ID":"d4ca42a2-bd91-4860-9a42-a0e4815fb4da","Type":"ContainerDied","Data":"cdb26e98235ddc0251268d18bca73267d275e70575f58b7250fbe676529e7055"} Sep 17 07:53:48.689115 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:48.689084 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" event={"ID":"d4ca42a2-bd91-4860-9a42-a0e4815fb4da","Type":"ContainerStarted","Data":"ce5e6f48958b2ae9426f2c844fa1d29d69258e98ba2f2a60353d50a0173d108b"} Sep 17 07:53:53.708776 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:53.708691 2579 generic.go:356] "Generic (PLEG): container finished" podID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerID="ce5e6f48958b2ae9426f2c844fa1d29d69258e98ba2f2a60353d50a0173d108b" exitCode=0 Sep 17 07:53:53.708776 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:53.708737 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" event={"ID":"d4ca42a2-bd91-4860-9a42-a0e4815fb4da","Type":"ContainerDied","Data":"ce5e6f48958b2ae9426f2c844fa1d29d69258e98ba2f2a60353d50a0173d108b"} Sep 17 07:53:54.717880 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:54.717837 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" event={"ID":"d4ca42a2-bd91-4860-9a42-a0e4815fb4da","Type":"ContainerStarted","Data":"814672108f149c7f64bc5ecc2fccbe1569b73e0ecef4b124cb4e3917db5c8b1d"} Sep 17 07:53:54.718390 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:54.717907 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" event={"ID":"d4ca42a2-bd91-4860-9a42-a0e4815fb4da","Type":"ContainerStarted","Data":"1de7dcebc500db1a55c63690de538f169a0c865704056b1fd4e972b4aba1105a"} Sep 17 07:53:54.718390 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:54.717930 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" event={"ID":"d4ca42a2-bd91-4860-9a42-a0e4815fb4da","Type":"ContainerStarted","Data":"8c6cc5f06a435b7c38d8bb4a704e94fa3e50da6e2c97913e54eb9fee2bb93314"} Sep 17 07:53:54.718390 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:54.717951 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" event={"ID":"d4ca42a2-bd91-4860-9a42-a0e4815fb4da","Type":"ContainerStarted","Data":"06b2743d3574c684def768336334ab2f534334914414f39063e724f4186e4bdd"} Sep 17 07:53:54.745452 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:54.745386 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" podStartSLOduration=14.74536358 podStartE2EDuration="14.74536358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:53:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:53:54.743444956 +0000 UTC m=+1552.933420908" watchObservedRunningTime="2026-09-17 07:53:54.74536358 +0000 UTC m=+1552.935339522" Sep 17 07:53:56.595162 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:56.595121 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:56.595714 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:56.595687 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:56.595714 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:56.595720 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:56.595908 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:56.595736 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:53:56.597130 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:56.597090 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.19:8443: connect: connection refused" Sep 17 07:53:56.597254 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:56.597120 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.19:8816: connect: connection refused" Sep 17 07:53:56.597324 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:56.597245 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": dial tcp 10.134.0.19:6567: connect: connection refused" Sep 17 07:53:56.597324 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:56.597304 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.19:6571: connect: connection refused" Sep 17 07:53:59.594598 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:59.594551 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.19:8816: connect: connection refused" Sep 17 07:53:59.595231 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:59.594551 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.19:6571: connect: connection refused" Sep 17 07:53:59.595359 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:59.594554 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": dial tcp 10.134.0.19:6567: connect: connection refused" Sep 17 07:53:59.595359 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:53:59.594559 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.19:8443: connect: connection refused" Sep 17 07:54:01.595125 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:54:01.595074 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:54:01.595507 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:54:01.595145 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:54:01.595507 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:54:01.595159 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:54:01.595507 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:54:01.595169 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:54:02.595785 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:54:02.595699 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:54:02.596270 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:54:02.595867 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:54:02.596270 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:54:02.596071 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:54:02.596947 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:54:02.596927 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:54:02.597077 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:54:02.596976 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:54:02.597077 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:54:02.597025 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:54:02.602031 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:54:02.601985 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:54:02.758438 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:54:02.758405 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:55:05.167467 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.167434 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t"] Sep 17 07:55:05.169955 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.167649 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="registry" containerID="cri-o://06b2743d3574c684def768336334ab2f534334914414f39063e724f4186e4bdd" gracePeriod=30 Sep 17 07:55:05.169955 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.167688 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="ui" containerID="cri-o://814672108f149c7f64bc5ecc2fccbe1569b73e0ecef4b124cb4e3917db5c8b1d" gracePeriod=30 Sep 17 07:55:05.169955 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.167721 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="online" containerID="cri-o://8c6cc5f06a435b7c38d8bb4a704e94fa3e50da6e2c97913e54eb9fee2bb93314" gracePeriod=30 Sep 17 07:55:05.169955 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.167691 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="offline" containerID="cri-o://1de7dcebc500db1a55c63690de538f169a0c865704056b1fd4e972b4aba1105a" gracePeriod=30 Sep 17 07:55:05.709446 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.709399 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr"] Sep 17 07:55:05.713461 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.713436 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:05.716681 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.716654 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-zhpxn\"" Sep 17 07:55:05.729223 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.729188 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr"] Sep 17 07:55:05.814458 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.814416 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-crpgr\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:05.814458 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.814461 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-crpgr\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:05.814736 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.814543 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-crpgr\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:05.814736 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.814618 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-crpgr\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:05.814736 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.814664 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-crpgr\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:05.814736 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.814684 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh2kr\" (UniqueName: \"kubernetes.io/projected/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-kube-api-access-qh2kr\") pod \"feast-simple-feast-setup-57576b8fcf-crpgr\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:05.915007 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.914958 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-crpgr\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:05.915188 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.915015 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-crpgr\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:05.915188 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.915047 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh2kr\" (UniqueName: \"kubernetes.io/projected/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-kube-api-access-qh2kr\") pod \"feast-simple-feast-setup-57576b8fcf-crpgr\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:05.915188 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.915065 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-crpgr\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:05.915188 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.915080 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-crpgr\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:05.915188 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.915113 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-crpgr\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:05.915617 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.915581 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-crpgr\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:05.917783 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.917751 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-crpgr\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:05.917903 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.917812 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-crpgr\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:05.918128 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.917965 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-crpgr\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:05.918128 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.918014 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-crpgr\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:05.926434 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.926413 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh2kr\" (UniqueName: \"kubernetes.io/projected/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-kube-api-access-qh2kr\") pod \"feast-simple-feast-setup-57576b8fcf-crpgr\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:05.990347 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.990259 2579 generic.go:356] "Generic (PLEG): container finished" podID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerID="814672108f149c7f64bc5ecc2fccbe1569b73e0ecef4b124cb4e3917db5c8b1d" exitCode=0 Sep 17 07:55:05.990347 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:05.990299 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" event={"ID":"d4ca42a2-bd91-4860-9a42-a0e4815fb4da","Type":"ContainerDied","Data":"814672108f149c7f64bc5ecc2fccbe1569b73e0ecef4b124cb4e3917db5c8b1d"} Sep 17 07:55:06.022668 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:06.022638 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:06.304072 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:06.303840 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr"] Sep 17 07:55:06.307091 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:55:06.307059 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded3ad499_6df7_4d49_98a2_bde4b5d15a90.slice/crio-7c0daeb21e8991016d2642848e645f2f370de3519a1041833b124c4bd3da1f66 WatchSource:0}: Error finding container 7c0daeb21e8991016d2642848e645f2f370de3519a1041833b124c4bd3da1f66: Status 404 returned error can't find the container with id 7c0daeb21e8991016d2642848e645f2f370de3519a1041833b124c4bd3da1f66 Sep 17 07:55:06.995287 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:06.995232 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" event={"ID":"ed3ad499-6df7-4d49-98a2-bde4b5d15a90","Type":"ContainerStarted","Data":"1e58c3634535a311bb3bd7725cbd262e0effbfeb87d93e7d20b96afcc282e1de"} Sep 17 07:55:06.995287 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:06.995285 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" event={"ID":"ed3ad499-6df7-4d49-98a2-bde4b5d15a90","Type":"ContainerStarted","Data":"7c0daeb21e8991016d2642848e645f2f370de3519a1041833b124c4bd3da1f66"} Sep 17 07:55:06.998407 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:06.998381 2579 generic.go:356] "Generic (PLEG): container finished" podID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerID="8c6cc5f06a435b7c38d8bb4a704e94fa3e50da6e2c97913e54eb9fee2bb93314" exitCode=0 Sep 17 07:55:06.998539 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:06.998440 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" event={"ID":"d4ca42a2-bd91-4860-9a42-a0e4815fb4da","Type":"ContainerDied","Data":"8c6cc5f06a435b7c38d8bb4a704e94fa3e50da6e2c97913e54eb9fee2bb93314"} Sep 17 07:55:12.018723 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:12.018690 2579 generic.go:356] "Generic (PLEG): container finished" podID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerID="1e58c3634535a311bb3bd7725cbd262e0effbfeb87d93e7d20b96afcc282e1de" exitCode=0 Sep 17 07:55:12.019139 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:12.018765 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" event={"ID":"ed3ad499-6df7-4d49-98a2-bde4b5d15a90","Type":"ContainerDied","Data":"1e58c3634535a311bb3bd7725cbd262e0effbfeb87d93e7d20b96afcc282e1de"} Sep 17 07:55:12.596557 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:12.596516 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.19:8443: connect: connection refused" Sep 17 07:55:12.756169 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:12.756130 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": dial tcp 10.134.0.19:6567: connect: connection refused" Sep 17 07:55:13.026937 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:13.026905 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" event={"ID":"ed3ad499-6df7-4d49-98a2-bde4b5d15a90","Type":"ContainerStarted","Data":"b49e3c247fc70e51a57ca1cef4ab99d51dbd8b622178feb6067e00fefd6b3aa2"} Sep 17 07:55:19.055217 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:19.055179 2579 generic.go:356] "Generic (PLEG): container finished" podID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerID="b49e3c247fc70e51a57ca1cef4ab99d51dbd8b622178feb6067e00fefd6b3aa2" exitCode=0 Sep 17 07:55:19.055602 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:19.055237 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" event={"ID":"ed3ad499-6df7-4d49-98a2-bde4b5d15a90","Type":"ContainerDied","Data":"b49e3c247fc70e51a57ca1cef4ab99d51dbd8b622178feb6067e00fefd6b3aa2"} Sep 17 07:55:20.064698 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:20.064659 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" event={"ID":"ed3ad499-6df7-4d49-98a2-bde4b5d15a90","Type":"ContainerStarted","Data":"95673d98d14a5bfc73f1a7e5a9d2c22dee48d1e5aec5c92aceb5c0087d707fe6"} Sep 17 07:55:20.065199 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:20.064723 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" event={"ID":"ed3ad499-6df7-4d49-98a2-bde4b5d15a90","Type":"ContainerStarted","Data":"03e431ad824877ccc00be2d1facbff9e02c7820fee916740c1ba196e3f109a33"} Sep 17 07:55:20.065199 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:20.064743 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" event={"ID":"ed3ad499-6df7-4d49-98a2-bde4b5d15a90","Type":"ContainerStarted","Data":"f79effded7070011a92b13b694405dff52497f4a6f8c272f1933ff5a8b5fb603"} Sep 17 07:55:20.065199 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:20.064761 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" event={"ID":"ed3ad499-6df7-4d49-98a2-bde4b5d15a90","Type":"ContainerStarted","Data":"dd74f2d10a62429f043a284f012abdc31315582105e7630dc7111387ef38dab3"} Sep 17 07:55:20.108451 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:20.108384 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" podStartSLOduration=15.108364859 podStartE2EDuration="15.108364859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:55:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:55:20.107457495 +0000 UTC m=+1638.297433453" watchObservedRunningTime="2026-09-17 07:55:20.108364859 +0000 UTC m=+1638.298340804" Sep 17 07:55:21.023447 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:21.023404 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:21.023658 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:21.023475 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:21.023658 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:21.023495 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:21.023658 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:21.023513 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:21.025837 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:21.025684 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.20:6571: connect: connection refused" Sep 17 07:55:21.025837 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:21.025721 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Sep 17 07:55:21.025837 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:21.025723 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.20:8816: connect: connection refused" Sep 17 07:55:21.025837 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:21.025807 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 17 07:55:22.596851 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:22.596790 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.19:8443: connect: connection refused" Sep 17 07:55:22.755940 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:22.755892 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": dial tcp 10.134.0.19:6567: connect: connection refused" Sep 17 07:55:24.023173 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:24.023129 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.20:6571: connect: connection refused" Sep 17 07:55:24.023766 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:24.023129 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.20:8816: connect: connection refused" Sep 17 07:55:24.023868 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:24.023173 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Sep 17 07:55:24.023868 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:24.023299 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 17 07:55:26.023436 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:26.023387 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:26.023436 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:26.023460 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:26.024026 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:26.023482 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:26.024026 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:26.023502 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:27.023922 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:27.023873 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Sep 17 07:55:27.023922 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:27.023894 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 17 07:55:27.024627 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:27.024603 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:27.024794 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:27.024775 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:27.103496 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:27.103438 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:27.103693 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:27.103633 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:30.024292 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:30.024249 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:30.029647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:30.029615 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:30.114143 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:30.114110 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:30.116664 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:30.116635 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:55:32.597007 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:32.596950 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.19:8443: connect: connection refused" Sep 17 07:55:32.597426 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:32.597149 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:55:32.756269 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:32.756225 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": dial tcp 10.134.0.19:6567: connect: connection refused" Sep 17 07:55:32.756465 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:32.756371 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:55:35.814887 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:35.814863 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:55:35.940466 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:35.940368 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-feast-data\") pod \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " Sep 17 07:55:35.940466 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:35.940410 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-offline-tls\") pod \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " Sep 17 07:55:35.940466 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:35.940443 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-online-tls\") pod \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " Sep 17 07:55:35.940466 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:35.940463 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-registry-tls\") pod \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " Sep 17 07:55:35.940804 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:35.940482 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-ui-tls\") pod \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " Sep 17 07:55:35.940804 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:35.940516 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wt9tb\" (UniqueName: \"kubernetes.io/projected/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-kube-api-access-wt9tb\") pod \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\" (UID: \"d4ca42a2-bd91-4860-9a42-a0e4815fb4da\") " Sep 17 07:55:35.941029 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:35.940974 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-feast-data" (OuterVolumeSpecName: "feast-data") pod "d4ca42a2-bd91-4860-9a42-a0e4815fb4da" (UID: "d4ca42a2-bd91-4860-9a42-a0e4815fb4da"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:55:35.942796 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:35.942761 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "d4ca42a2-bd91-4860-9a42-a0e4815fb4da" (UID: "d4ca42a2-bd91-4860-9a42-a0e4815fb4da"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:55:35.943030 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:35.942959 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-kube-api-access-wt9tb" (OuterVolumeSpecName: "kube-api-access-wt9tb") pod "d4ca42a2-bd91-4860-9a42-a0e4815fb4da" (UID: "d4ca42a2-bd91-4860-9a42-a0e4815fb4da"). InnerVolumeSpecName "kube-api-access-wt9tb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:55:35.943030 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:35.942965 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "d4ca42a2-bd91-4860-9a42-a0e4815fb4da" (UID: "d4ca42a2-bd91-4860-9a42-a0e4815fb4da"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:55:35.943030 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:35.943018 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-online-tls" (OuterVolumeSpecName: "online-tls") pod "d4ca42a2-bd91-4860-9a42-a0e4815fb4da" (UID: "d4ca42a2-bd91-4860-9a42-a0e4815fb4da"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:55:35.943348 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:35.943325 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d4ca42a2-bd91-4860-9a42-a0e4815fb4da" (UID: "d4ca42a2-bd91-4860-9a42-a0e4815fb4da"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:55:36.041929 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.041890 2579 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-offline-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:55:36.041929 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.041922 2579 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-online-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:55:36.041929 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.041930 2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-registry-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:55:36.041929 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.041939 2579 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-ui-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:55:36.042221 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.041947 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wt9tb\" (UniqueName: \"kubernetes.io/projected/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-kube-api-access-wt9tb\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:55:36.042221 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.041956 2579 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d4ca42a2-bd91-4860-9a42-a0e4815fb4da-feast-data\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:55:36.136741 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.136705 2579 generic.go:356] "Generic (PLEG): container finished" podID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerID="1de7dcebc500db1a55c63690de538f169a0c865704056b1fd4e972b4aba1105a" exitCode=137 Sep 17 07:55:36.136741 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.136734 2579 generic.go:356] "Generic (PLEG): container finished" podID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerID="06b2743d3574c684def768336334ab2f534334914414f39063e724f4186e4bdd" exitCode=137 Sep 17 07:55:36.136923 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.136780 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" Sep 17 07:55:36.136923 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.136780 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" event={"ID":"d4ca42a2-bd91-4860-9a42-a0e4815fb4da","Type":"ContainerDied","Data":"1de7dcebc500db1a55c63690de538f169a0c865704056b1fd4e972b4aba1105a"} Sep 17 07:55:36.136923 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.136830 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" event={"ID":"d4ca42a2-bd91-4860-9a42-a0e4815fb4da","Type":"ContainerDied","Data":"06b2743d3574c684def768336334ab2f534334914414f39063e724f4186e4bdd"} Sep 17 07:55:36.136923 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.136846 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t" event={"ID":"d4ca42a2-bd91-4860-9a42-a0e4815fb4da","Type":"ContainerDied","Data":"7ed27c6212880d1df4d62abb86dff71f2058418cffa7e2dc69e2ed65ba4017de"} Sep 17 07:55:36.136923 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.136864 2579 scope.go:117] "RemoveContainer" containerID="814672108f149c7f64bc5ecc2fccbe1569b73e0ecef4b124cb4e3917db5c8b1d" Sep 17 07:55:36.145567 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.145549 2579 scope.go:117] "RemoveContainer" containerID="1de7dcebc500db1a55c63690de538f169a0c865704056b1fd4e972b4aba1105a" Sep 17 07:55:36.153446 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.153427 2579 scope.go:117] "RemoveContainer" containerID="8c6cc5f06a435b7c38d8bb4a704e94fa3e50da6e2c97913e54eb9fee2bb93314" Sep 17 07:55:36.160929 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.160909 2579 scope.go:117] "RemoveContainer" containerID="06b2743d3574c684def768336334ab2f534334914414f39063e724f4186e4bdd" Sep 17 07:55:36.168674 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.168658 2579 scope.go:117] "RemoveContainer" containerID="ce5e6f48958b2ae9426f2c844fa1d29d69258e98ba2f2a60353d50a0173d108b" Sep 17 07:55:36.171077 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.171054 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t"] Sep 17 07:55:36.177090 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.176914 2579 scope.go:117] "RemoveContainer" containerID="cdb26e98235ddc0251268d18bca73267d275e70575f58b7250fbe676529e7055" Sep 17 07:55:36.177159 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.177138 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-t542t"] Sep 17 07:55:36.184288 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.184265 2579 scope.go:117] "RemoveContainer" containerID="814672108f149c7f64bc5ecc2fccbe1569b73e0ecef4b124cb4e3917db5c8b1d" Sep 17 07:55:36.184587 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:55:36.184566 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"814672108f149c7f64bc5ecc2fccbe1569b73e0ecef4b124cb4e3917db5c8b1d\": container with ID starting with 814672108f149c7f64bc5ecc2fccbe1569b73e0ecef4b124cb4e3917db5c8b1d not found: ID does not exist" containerID="814672108f149c7f64bc5ecc2fccbe1569b73e0ecef4b124cb4e3917db5c8b1d" Sep 17 07:55:36.184658 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.184598 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"814672108f149c7f64bc5ecc2fccbe1569b73e0ecef4b124cb4e3917db5c8b1d"} err="failed to get container status \"814672108f149c7f64bc5ecc2fccbe1569b73e0ecef4b124cb4e3917db5c8b1d\": rpc error: code = NotFound desc = could not find container \"814672108f149c7f64bc5ecc2fccbe1569b73e0ecef4b124cb4e3917db5c8b1d\": container with ID starting with 814672108f149c7f64bc5ecc2fccbe1569b73e0ecef4b124cb4e3917db5c8b1d not found: ID does not exist" Sep 17 07:55:36.184658 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.184617 2579 scope.go:117] "RemoveContainer" containerID="1de7dcebc500db1a55c63690de538f169a0c865704056b1fd4e972b4aba1105a" Sep 17 07:55:36.184901 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:55:36.184884 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1de7dcebc500db1a55c63690de538f169a0c865704056b1fd4e972b4aba1105a\": container with ID starting with 1de7dcebc500db1a55c63690de538f169a0c865704056b1fd4e972b4aba1105a not found: ID does not exist" containerID="1de7dcebc500db1a55c63690de538f169a0c865704056b1fd4e972b4aba1105a" Sep 17 07:55:36.184938 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.184911 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1de7dcebc500db1a55c63690de538f169a0c865704056b1fd4e972b4aba1105a"} err="failed to get container status \"1de7dcebc500db1a55c63690de538f169a0c865704056b1fd4e972b4aba1105a\": rpc error: code = NotFound desc = could not find container \"1de7dcebc500db1a55c63690de538f169a0c865704056b1fd4e972b4aba1105a\": container with ID starting with 1de7dcebc500db1a55c63690de538f169a0c865704056b1fd4e972b4aba1105a not found: ID does not exist" Sep 17 07:55:36.184938 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.184927 2579 scope.go:117] "RemoveContainer" containerID="8c6cc5f06a435b7c38d8bb4a704e94fa3e50da6e2c97913e54eb9fee2bb93314" Sep 17 07:55:36.185187 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:55:36.185170 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c6cc5f06a435b7c38d8bb4a704e94fa3e50da6e2c97913e54eb9fee2bb93314\": container with ID starting with 8c6cc5f06a435b7c38d8bb4a704e94fa3e50da6e2c97913e54eb9fee2bb93314 not found: ID does not exist" containerID="8c6cc5f06a435b7c38d8bb4a704e94fa3e50da6e2c97913e54eb9fee2bb93314" Sep 17 07:55:36.185243 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.185190 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c6cc5f06a435b7c38d8bb4a704e94fa3e50da6e2c97913e54eb9fee2bb93314"} err="failed to get container status \"8c6cc5f06a435b7c38d8bb4a704e94fa3e50da6e2c97913e54eb9fee2bb93314\": rpc error: code = NotFound desc = could not find container \"8c6cc5f06a435b7c38d8bb4a704e94fa3e50da6e2c97913e54eb9fee2bb93314\": container with ID starting with 8c6cc5f06a435b7c38d8bb4a704e94fa3e50da6e2c97913e54eb9fee2bb93314 not found: ID does not exist" Sep 17 07:55:36.185243 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.185206 2579 scope.go:117] "RemoveContainer" containerID="06b2743d3574c684def768336334ab2f534334914414f39063e724f4186e4bdd" Sep 17 07:55:36.185435 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:55:36.185419 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06b2743d3574c684def768336334ab2f534334914414f39063e724f4186e4bdd\": container with ID starting with 06b2743d3574c684def768336334ab2f534334914414f39063e724f4186e4bdd not found: ID does not exist" containerID="06b2743d3574c684def768336334ab2f534334914414f39063e724f4186e4bdd" Sep 17 07:55:36.185477 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.185438 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06b2743d3574c684def768336334ab2f534334914414f39063e724f4186e4bdd"} err="failed to get container status \"06b2743d3574c684def768336334ab2f534334914414f39063e724f4186e4bdd\": rpc error: code = NotFound desc = could not find container \"06b2743d3574c684def768336334ab2f534334914414f39063e724f4186e4bdd\": container with ID starting with 06b2743d3574c684def768336334ab2f534334914414f39063e724f4186e4bdd not found: ID does not exist" Sep 17 07:55:36.185477 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.185453 2579 scope.go:117] "RemoveContainer" containerID="ce5e6f48958b2ae9426f2c844fa1d29d69258e98ba2f2a60353d50a0173d108b" Sep 17 07:55:36.185697 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:55:36.185680 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce5e6f48958b2ae9426f2c844fa1d29d69258e98ba2f2a60353d50a0173d108b\": container with ID starting with ce5e6f48958b2ae9426f2c844fa1d29d69258e98ba2f2a60353d50a0173d108b not found: ID does not exist" containerID="ce5e6f48958b2ae9426f2c844fa1d29d69258e98ba2f2a60353d50a0173d108b" Sep 17 07:55:36.185753 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.185700 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce5e6f48958b2ae9426f2c844fa1d29d69258e98ba2f2a60353d50a0173d108b"} err="failed to get container status \"ce5e6f48958b2ae9426f2c844fa1d29d69258e98ba2f2a60353d50a0173d108b\": rpc error: code = NotFound desc = could not find container \"ce5e6f48958b2ae9426f2c844fa1d29d69258e98ba2f2a60353d50a0173d108b\": container with ID starting with ce5e6f48958b2ae9426f2c844fa1d29d69258e98ba2f2a60353d50a0173d108b not found: ID does not exist" Sep 17 07:55:36.185753 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.185713 2579 scope.go:117] "RemoveContainer" containerID="cdb26e98235ddc0251268d18bca73267d275e70575f58b7250fbe676529e7055" Sep 17 07:55:36.185931 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:55:36.185918 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdb26e98235ddc0251268d18bca73267d275e70575f58b7250fbe676529e7055\": container with ID starting with cdb26e98235ddc0251268d18bca73267d275e70575f58b7250fbe676529e7055 not found: ID does not exist" containerID="cdb26e98235ddc0251268d18bca73267d275e70575f58b7250fbe676529e7055" Sep 17 07:55:36.185972 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.185934 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdb26e98235ddc0251268d18bca73267d275e70575f58b7250fbe676529e7055"} err="failed to get container status \"cdb26e98235ddc0251268d18bca73267d275e70575f58b7250fbe676529e7055\": rpc error: code = NotFound desc = could not find container \"cdb26e98235ddc0251268d18bca73267d275e70575f58b7250fbe676529e7055\": container with ID starting with cdb26e98235ddc0251268d18bca73267d275e70575f58b7250fbe676529e7055 not found: ID does not exist" Sep 17 07:55:36.185972 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.185948 2579 scope.go:117] "RemoveContainer" containerID="814672108f149c7f64bc5ecc2fccbe1569b73e0ecef4b124cb4e3917db5c8b1d" Sep 17 07:55:36.186291 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.186224 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"814672108f149c7f64bc5ecc2fccbe1569b73e0ecef4b124cb4e3917db5c8b1d"} err="failed to get container status \"814672108f149c7f64bc5ecc2fccbe1569b73e0ecef4b124cb4e3917db5c8b1d\": rpc error: code = NotFound desc = could not find container \"814672108f149c7f64bc5ecc2fccbe1569b73e0ecef4b124cb4e3917db5c8b1d\": container with ID starting with 814672108f149c7f64bc5ecc2fccbe1569b73e0ecef4b124cb4e3917db5c8b1d not found: ID does not exist" Sep 17 07:55:36.186291 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.186242 2579 scope.go:117] "RemoveContainer" containerID="1de7dcebc500db1a55c63690de538f169a0c865704056b1fd4e972b4aba1105a" Sep 17 07:55:36.186475 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.186458 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1de7dcebc500db1a55c63690de538f169a0c865704056b1fd4e972b4aba1105a"} err="failed to get container status \"1de7dcebc500db1a55c63690de538f169a0c865704056b1fd4e972b4aba1105a\": rpc error: code = NotFound desc = could not find container \"1de7dcebc500db1a55c63690de538f169a0c865704056b1fd4e972b4aba1105a\": container with ID starting with 1de7dcebc500db1a55c63690de538f169a0c865704056b1fd4e972b4aba1105a not found: ID does not exist" Sep 17 07:55:36.186526 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.186475 2579 scope.go:117] "RemoveContainer" containerID="8c6cc5f06a435b7c38d8bb4a704e94fa3e50da6e2c97913e54eb9fee2bb93314" Sep 17 07:55:36.186694 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.186673 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c6cc5f06a435b7c38d8bb4a704e94fa3e50da6e2c97913e54eb9fee2bb93314"} err="failed to get container status \"8c6cc5f06a435b7c38d8bb4a704e94fa3e50da6e2c97913e54eb9fee2bb93314\": rpc error: code = NotFound desc = could not find container \"8c6cc5f06a435b7c38d8bb4a704e94fa3e50da6e2c97913e54eb9fee2bb93314\": container with ID starting with 8c6cc5f06a435b7c38d8bb4a704e94fa3e50da6e2c97913e54eb9fee2bb93314 not found: ID does not exist" Sep 17 07:55:36.186743 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.186694 2579 scope.go:117] "RemoveContainer" containerID="06b2743d3574c684def768336334ab2f534334914414f39063e724f4186e4bdd" Sep 17 07:55:36.186889 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.186872 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06b2743d3574c684def768336334ab2f534334914414f39063e724f4186e4bdd"} err="failed to get container status \"06b2743d3574c684def768336334ab2f534334914414f39063e724f4186e4bdd\": rpc error: code = NotFound desc = could not find container \"06b2743d3574c684def768336334ab2f534334914414f39063e724f4186e4bdd\": container with ID starting with 06b2743d3574c684def768336334ab2f534334914414f39063e724f4186e4bdd not found: ID does not exist" Sep 17 07:55:36.186933 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.186889 2579 scope.go:117] "RemoveContainer" containerID="ce5e6f48958b2ae9426f2c844fa1d29d69258e98ba2f2a60353d50a0173d108b" Sep 17 07:55:36.187124 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.187108 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce5e6f48958b2ae9426f2c844fa1d29d69258e98ba2f2a60353d50a0173d108b"} err="failed to get container status \"ce5e6f48958b2ae9426f2c844fa1d29d69258e98ba2f2a60353d50a0173d108b\": rpc error: code = NotFound desc = could not find container \"ce5e6f48958b2ae9426f2c844fa1d29d69258e98ba2f2a60353d50a0173d108b\": container with ID starting with ce5e6f48958b2ae9426f2c844fa1d29d69258e98ba2f2a60353d50a0173d108b not found: ID does not exist" Sep 17 07:55:36.187124 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.187123 2579 scope.go:117] "RemoveContainer" containerID="cdb26e98235ddc0251268d18bca73267d275e70575f58b7250fbe676529e7055" Sep 17 07:55:36.187343 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.187327 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdb26e98235ddc0251268d18bca73267d275e70575f58b7250fbe676529e7055"} err="failed to get container status \"cdb26e98235ddc0251268d18bca73267d275e70575f58b7250fbe676529e7055\": rpc error: code = NotFound desc = could not find container \"cdb26e98235ddc0251268d18bca73267d275e70575f58b7250fbe676529e7055\": container with ID starting with cdb26e98235ddc0251268d18bca73267d275e70575f58b7250fbe676529e7055 not found: ID does not exist" Sep 17 07:55:36.309636 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:55:36.309605 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" path="/var/lib/kubelet/pods/d4ca42a2-bd91-4860-9a42-a0e4815fb4da/volumes" Sep 17 07:56:32.396681 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.396598 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x"] Sep 17 07:56:32.397160 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.396838 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="online" Sep 17 07:56:32.397160 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.396848 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="online" Sep 17 07:56:32.397160 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.396857 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="feast-apply" Sep 17 07:56:32.397160 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.396863 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="feast-apply" Sep 17 07:56:32.397160 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.396875 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="offline" Sep 17 07:56:32.397160 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.396880 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="offline" Sep 17 07:56:32.397160 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.396889 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="registry" Sep 17 07:56:32.397160 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.396893 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="registry" Sep 17 07:56:32.397160 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.396902 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="feast-init" Sep 17 07:56:32.397160 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.396907 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="feast-init" Sep 17 07:56:32.397160 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.396915 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="ui" Sep 17 07:56:32.397160 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.396920 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="ui" Sep 17 07:56:32.397160 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.396960 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="ui" Sep 17 07:56:32.397160 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.396967 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="online" Sep 17 07:56:32.397160 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.396973 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="offline" Sep 17 07:56:32.397160 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.396981 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4ca42a2-bd91-4860-9a42-a0e4815fb4da" containerName="registry" Sep 17 07:56:32.400222 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.400204 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:32.403463 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.403440 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 17 07:56:32.404198 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.404174 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 17 07:56:32.404396 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.404380 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 17 07:56:32.404667 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.404653 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-sl68l\"" Sep 17 07:56:32.405983 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.405963 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 17 07:56:32.406157 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.406138 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 17 07:56:32.406223 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.406138 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 17 07:56:32.418818 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.418783 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x"] Sep 17 07:56:32.440938 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.440900 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ts62x\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:32.440938 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.440939 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7ab59680-e713-4ba3-a035-68d936128d8a-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ts62x\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:32.441182 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.440971 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ts62x\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:32.441182 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.441061 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7ab59680-e713-4ba3-a035-68d936128d8a-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ts62x\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:32.441182 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.441103 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82fvb\" (UniqueName: \"kubernetes.io/projected/7ab59680-e713-4ba3-a035-68d936128d8a-kube-api-access-82fvb\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ts62x\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:32.441182 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.441132 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ts62x\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:32.541799 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.541762 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ts62x\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:32.541963 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.541815 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ts62x\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:32.541963 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.541833 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7ab59680-e713-4ba3-a035-68d936128d8a-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ts62x\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:32.541963 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.541852 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ts62x\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:32.541963 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.541871 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7ab59680-e713-4ba3-a035-68d936128d8a-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ts62x\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:32.541963 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.541898 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82fvb\" (UniqueName: \"kubernetes.io/projected/7ab59680-e713-4ba3-a035-68d936128d8a-kube-api-access-82fvb\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ts62x\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:32.541963 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:56:32.541898 2579 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found Sep 17 07:56:32.541963 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:56:32.541959 2579 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 17 07:56:32.542329 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:56:32.542032 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-offline-tls podName:7ab59680-e713-4ba3-a035-68d936128d8a nodeName:}" failed. No retries permitted until 2026-09-17 07:56:33.041987238 +0000 UTC m=+1711.231963174 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-offline-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-ts62x" (UID: "7ab59680-e713-4ba3-a035-68d936128d8a") : secret "feast-simple-feast-remote-setup-offline-tls" not found Sep 17 07:56:32.542329 ip-10-0-129-217 kubenswrapper[2579]: E0917 07:56:32.542051 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-ui-tls podName:7ab59680-e713-4ba3-a035-68d936128d8a nodeName:}" failed. No retries permitted until 2026-09-17 07:56:33.04204248 +0000 UTC m=+1711.232018731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-ui-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-ts62x" (UID: "7ab59680-e713-4ba3-a035-68d936128d8a") : secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 17 07:56:32.542329 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.542265 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7ab59680-e713-4ba3-a035-68d936128d8a-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ts62x\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:32.542564 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.542544 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7ab59680-e713-4ba3-a035-68d936128d8a-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ts62x\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:32.544338 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.544322 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ts62x\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:32.554087 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:32.554056 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82fvb\" (UniqueName: \"kubernetes.io/projected/7ab59680-e713-4ba3-a035-68d936128d8a-kube-api-access-82fvb\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ts62x\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:33.046588 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:33.046553 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ts62x\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:33.046793 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:33.046639 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ts62x\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:33.048895 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:33.048868 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ts62x\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:33.049044 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:33.048970 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ts62x\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:33.309710 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:33.309621 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:56:33.606531 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:56:33.606503 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ab59680_e713_4ba3_a035_68d936128d8a.slice/crio-a8170a54ea6be310b441a777cd074e1926a40db4af2d046fc5cfc8293826d323 WatchSource:0}: Error finding container a8170a54ea6be310b441a777cd074e1926a40db4af2d046fc5cfc8293826d323: Status 404 returned error can't find the container with id a8170a54ea6be310b441a777cd074e1926a40db4af2d046fc5cfc8293826d323 Sep 17 07:56:33.606872 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:33.606587 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x"] Sep 17 07:56:34.337569 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:34.337528 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" event={"ID":"7ab59680-e713-4ba3-a035-68d936128d8a","Type":"ContainerStarted","Data":"d91d2af251eadf562f8cc41ee68a943d78b8de3d3a65b8e21f2fc8e342d78dba"} Sep 17 07:56:34.337569 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:34.337575 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" event={"ID":"7ab59680-e713-4ba3-a035-68d936128d8a","Type":"ContainerStarted","Data":"a8170a54ea6be310b441a777cd074e1926a40db4af2d046fc5cfc8293826d323"} Sep 17 07:56:39.359805 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:39.359768 2579 generic.go:356] "Generic (PLEG): container finished" podID="7ab59680-e713-4ba3-a035-68d936128d8a" containerID="d91d2af251eadf562f8cc41ee68a943d78b8de3d3a65b8e21f2fc8e342d78dba" exitCode=0 Sep 17 07:56:39.360236 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:39.359814 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" event={"ID":"7ab59680-e713-4ba3-a035-68d936128d8a","Type":"ContainerDied","Data":"d91d2af251eadf562f8cc41ee68a943d78b8de3d3a65b8e21f2fc8e342d78dba"} Sep 17 07:56:40.365348 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:56:40.365305 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" event={"ID":"7ab59680-e713-4ba3-a035-68d936128d8a","Type":"ContainerStarted","Data":"b935c8e0ebf142a14ab6d61aa278865258093e926358e4e18d43a2110678427d"} Sep 17 07:57:06.461262 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:06.461218 2579 generic.go:356] "Generic (PLEG): container finished" podID="7ab59680-e713-4ba3-a035-68d936128d8a" containerID="b935c8e0ebf142a14ab6d61aa278865258093e926358e4e18d43a2110678427d" exitCode=0 Sep 17 07:57:06.461653 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:06.461287 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" event={"ID":"7ab59680-e713-4ba3-a035-68d936128d8a","Type":"ContainerDied","Data":"b935c8e0ebf142a14ab6d61aa278865258093e926358e4e18d43a2110678427d"} Sep 17 07:57:07.476316 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:07.476276 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" event={"ID":"7ab59680-e713-4ba3-a035-68d936128d8a","Type":"ContainerStarted","Data":"6d858e8bc38d31c18685280376c161580a6451a1f813840513f45e5ec6678ce2"} Sep 17 07:57:07.476876 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:07.476332 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" event={"ID":"7ab59680-e713-4ba3-a035-68d936128d8a","Type":"ContainerStarted","Data":"0b7521471658f63ed61934daea00ba5d54ef9a16017d4d7e5e86589a13537f16"} Sep 17 07:57:07.476876 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:07.476349 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" event={"ID":"7ab59680-e713-4ba3-a035-68d936128d8a","Type":"ContainerStarted","Data":"c5991c0cb2b8d9feab9df603905795e166e3ed20b185f810c9beff5b1f00b831"} Sep 17 07:57:07.505009 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:07.504937 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" podStartSLOduration=35.504916348 podStartE2EDuration="35.504916348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:56:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:57:07.504594783 +0000 UTC m=+1745.694570737" watchObservedRunningTime="2026-09-17 07:57:07.504916348 +0000 UTC m=+1745.694892291" Sep 17 07:57:09.310239 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:09.310202 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:57:09.310835 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:09.310801 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:57:09.310835 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:09.310835 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:57:09.311666 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:09.311632 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Sep 17 07:57:09.311780 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:09.311673 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Sep 17 07:57:09.311780 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:09.311665 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.21:8816: connect: connection refused" Sep 17 07:57:12.309961 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:12.309921 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.21:8816: connect: connection refused" Sep 17 07:57:12.310558 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:12.310031 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Sep 17 07:57:12.310558 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:12.309921 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Sep 17 07:57:13.310678 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:13.310633 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:57:13.311069 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:13.310713 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:57:13.311069 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:13.310732 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:57:15.311779 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:15.311744 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:57:15.312314 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:15.311861 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:57:15.312546 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:15.312522 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:57:15.312733 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:15.312716 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:57:16.311352 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:16.311308 2579 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": context deadline exceeded" Sep 17 07:57:19.224132 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:19.224098 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:57:20.228187 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:20.228134 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 17 07:57:30.083397 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:30.083361 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:57:34.311033 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:57:34.310985 2579 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 17 07:58:30.229158 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:30.229105 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 17 07:58:32.395509 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:32.395479 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x"] Sep 17 07:58:32.396032 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:32.395680 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="online" containerID="cri-o://c5991c0cb2b8d9feab9df603905795e166e3ed20b185f810c9beff5b1f00b831" gracePeriod=30 Sep 17 07:58:32.396032 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:32.395727 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="offline" containerID="cri-o://0b7521471658f63ed61934daea00ba5d54ef9a16017d4d7e5e86589a13537f16" gracePeriod=30 Sep 17 07:58:32.396032 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:32.395795 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="ui" containerID="cri-o://6d858e8bc38d31c18685280376c161580a6451a1f813840513f45e5ec6678ce2" gracePeriod=30 Sep 17 07:58:32.589664 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:32.589631 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr"] Sep 17 07:58:32.590096 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:32.589880 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="registry" containerID="cri-o://dd74f2d10a62429f043a284f012abdc31315582105e7630dc7111387ef38dab3" gracePeriod=30 Sep 17 07:58:32.590096 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:32.589940 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="online" containerID="cri-o://f79effded7070011a92b13b694405dff52497f4a6f8c272f1933ff5a8b5fb603" gracePeriod=30 Sep 17 07:58:32.590096 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:32.589962 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="ui" containerID="cri-o://95673d98d14a5bfc73f1a7e5a9d2c22dee48d1e5aec5c92aceb5c0087d707fe6" gracePeriod=30 Sep 17 07:58:32.590096 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:32.590030 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="offline" containerID="cri-o://03e431ad824877ccc00be2d1facbff9e02c7820fee916740c1ba196e3f109a33" gracePeriod=30 Sep 17 07:58:33.780288 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:33.780254 2579 generic.go:356] "Generic (PLEG): container finished" podID="7ab59680-e713-4ba3-a035-68d936128d8a" containerID="6d858e8bc38d31c18685280376c161580a6451a1f813840513f45e5ec6678ce2" exitCode=0 Sep 17 07:58:33.780288 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:33.780280 2579 generic.go:356] "Generic (PLEG): container finished" podID="7ab59680-e713-4ba3-a035-68d936128d8a" containerID="c5991c0cb2b8d9feab9df603905795e166e3ed20b185f810c9beff5b1f00b831" exitCode=0 Sep 17 07:58:33.780706 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:33.780321 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" event={"ID":"7ab59680-e713-4ba3-a035-68d936128d8a","Type":"ContainerDied","Data":"6d858e8bc38d31c18685280376c161580a6451a1f813840513f45e5ec6678ce2"} Sep 17 07:58:33.780706 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:33.780362 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" event={"ID":"7ab59680-e713-4ba3-a035-68d936128d8a","Type":"ContainerDied","Data":"c5991c0cb2b8d9feab9df603905795e166e3ed20b185f810c9beff5b1f00b831"} Sep 17 07:58:33.783190 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:33.783169 2579 generic.go:356] "Generic (PLEG): container finished" podID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerID="95673d98d14a5bfc73f1a7e5a9d2c22dee48d1e5aec5c92aceb5c0087d707fe6" exitCode=0 Sep 17 07:58:33.783190 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:33.783186 2579 generic.go:356] "Generic (PLEG): container finished" podID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerID="f79effded7070011a92b13b694405dff52497f4a6f8c272f1933ff5a8b5fb603" exitCode=0 Sep 17 07:58:33.783341 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:33.783219 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" event={"ID":"ed3ad499-6df7-4d49-98a2-bde4b5d15a90","Type":"ContainerDied","Data":"95673d98d14a5bfc73f1a7e5a9d2c22dee48d1e5aec5c92aceb5c0087d707fe6"} Sep 17 07:58:33.783341 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:33.783246 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" event={"ID":"ed3ad499-6df7-4d49-98a2-bde4b5d15a90","Type":"ContainerDied","Data":"f79effded7070011a92b13b694405dff52497f4a6f8c272f1933ff5a8b5fb603"} Sep 17 07:58:35.312753 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:35.312712 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Sep 17 07:58:39.224963 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:39.224925 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Sep 17 07:58:40.113638 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:40.113600 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 17 07:58:40.113804 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:40.113595 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Sep 17 07:58:45.312634 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:45.312593 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Sep 17 07:58:49.225250 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:49.225205 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Sep 17 07:58:49.225629 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:49.225317 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:58:50.113889 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:50.113846 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Sep 17 07:58:50.114086 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:50.113896 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 17 07:58:55.312557 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:55.312517 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Sep 17 07:58:55.312940 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:55.312630 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:58:59.225204 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:58:59.225153 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Sep 17 07:59:00.113791 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:00.113744 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Sep 17 07:59:00.113791 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:00.113766 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 17 07:59:00.114031 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:00.113881 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:59:00.114031 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:00.113953 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:59:02.884214 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:02.884179 2579 generic.go:356] "Generic (PLEG): container finished" podID="7ab59680-e713-4ba3-a035-68d936128d8a" containerID="0b7521471658f63ed61934daea00ba5d54ef9a16017d4d7e5e86589a13537f16" exitCode=137 Sep 17 07:59:02.884623 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:02.884249 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" event={"ID":"7ab59680-e713-4ba3-a035-68d936128d8a","Type":"ContainerDied","Data":"0b7521471658f63ed61934daea00ba5d54ef9a16017d4d7e5e86589a13537f16"} Sep 17 07:59:02.887243 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:02.887217 2579 generic.go:356] "Generic (PLEG): container finished" podID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerID="03e431ad824877ccc00be2d1facbff9e02c7820fee916740c1ba196e3f109a33" exitCode=137 Sep 17 07:59:02.887243 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:02.887242 2579 generic.go:356] "Generic (PLEG): container finished" podID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerID="dd74f2d10a62429f043a284f012abdc31315582105e7630dc7111387ef38dab3" exitCode=137 Sep 17 07:59:02.887394 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:02.887278 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" event={"ID":"ed3ad499-6df7-4d49-98a2-bde4b5d15a90","Type":"ContainerDied","Data":"03e431ad824877ccc00be2d1facbff9e02c7820fee916740c1ba196e3f109a33"} Sep 17 07:59:02.887394 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:02.887315 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" event={"ID":"ed3ad499-6df7-4d49-98a2-bde4b5d15a90","Type":"ContainerDied","Data":"dd74f2d10a62429f043a284f012abdc31315582105e7630dc7111387ef38dab3"} Sep 17 07:59:03.036463 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.036438 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:59:03.137499 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.137406 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82fvb\" (UniqueName: \"kubernetes.io/projected/7ab59680-e713-4ba3-a035-68d936128d8a-kube-api-access-82fvb\") pod \"7ab59680-e713-4ba3-a035-68d936128d8a\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " Sep 17 07:59:03.137499 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.137448 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7ab59680-e713-4ba3-a035-68d936128d8a-registry-tls\") pod \"7ab59680-e713-4ba3-a035-68d936128d8a\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " Sep 17 07:59:03.137499 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.137495 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7ab59680-e713-4ba3-a035-68d936128d8a-feast-data\") pod \"7ab59680-e713-4ba3-a035-68d936128d8a\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " Sep 17 07:59:03.137800 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.137526 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-offline-tls\") pod \"7ab59680-e713-4ba3-a035-68d936128d8a\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " Sep 17 07:59:03.137800 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.137560 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-online-tls\") pod \"7ab59680-e713-4ba3-a035-68d936128d8a\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " Sep 17 07:59:03.137800 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.137726 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-ui-tls\") pod \"7ab59680-e713-4ba3-a035-68d936128d8a\" (UID: \"7ab59680-e713-4ba3-a035-68d936128d8a\") " Sep 17 07:59:03.137962 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.137929 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ab59680-e713-4ba3-a035-68d936128d8a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7ab59680-e713-4ba3-a035-68d936128d8a" (UID: "7ab59680-e713-4ba3-a035-68d936128d8a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:59:03.138119 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.138092 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ab59680-e713-4ba3-a035-68d936128d8a-feast-data" (OuterVolumeSpecName: "feast-data") pod "7ab59680-e713-4ba3-a035-68d936128d8a" (UID: "7ab59680-e713-4ba3-a035-68d936128d8a"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:59:03.139851 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.139823 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "7ab59680-e713-4ba3-a035-68d936128d8a" (UID: "7ab59680-e713-4ba3-a035-68d936128d8a"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:59:03.139851 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.139843 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-online-tls" (OuterVolumeSpecName: "online-tls") pod "7ab59680-e713-4ba3-a035-68d936128d8a" (UID: "7ab59680-e713-4ba3-a035-68d936128d8a"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:59:03.140047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.139863 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "7ab59680-e713-4ba3-a035-68d936128d8a" (UID: "7ab59680-e713-4ba3-a035-68d936128d8a"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:59:03.140047 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.139974 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ab59680-e713-4ba3-a035-68d936128d8a-kube-api-access-82fvb" (OuterVolumeSpecName: "kube-api-access-82fvb") pod "7ab59680-e713-4ba3-a035-68d936128d8a" (UID: "7ab59680-e713-4ba3-a035-68d936128d8a"). InnerVolumeSpecName "kube-api-access-82fvb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:59:03.229915 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.229890 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:59:03.239258 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.239235 2579 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-offline-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:59:03.239356 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.239260 2579 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-online-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:59:03.239356 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.239269 2579 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7ab59680-e713-4ba3-a035-68d936128d8a-ui-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:59:03.239356 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.239277 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82fvb\" (UniqueName: \"kubernetes.io/projected/7ab59680-e713-4ba3-a035-68d936128d8a-kube-api-access-82fvb\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:59:03.239356 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.239286 2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7ab59680-e713-4ba3-a035-68d936128d8a-registry-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:59:03.239356 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.239295 2579 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7ab59680-e713-4ba3-a035-68d936128d8a-feast-data\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:59:03.339834 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.339798 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-online-tls\") pod \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " Sep 17 07:59:03.339834 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.339845 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-registry-tls\") pod \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " Sep 17 07:59:03.340153 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.339873 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-offline-tls\") pod \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " Sep 17 07:59:03.340153 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.339892 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-feast-data\") pod \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " Sep 17 07:59:03.340153 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.339922 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qh2kr\" (UniqueName: \"kubernetes.io/projected/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-kube-api-access-qh2kr\") pod \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " Sep 17 07:59:03.340153 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.339956 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-ui-tls\") pod \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\" (UID: \"ed3ad499-6df7-4d49-98a2-bde4b5d15a90\") " Sep 17 07:59:03.340546 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.340513 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-feast-data" (OuterVolumeSpecName: "feast-data") pod "ed3ad499-6df7-4d49-98a2-bde4b5d15a90" (UID: "ed3ad499-6df7-4d49-98a2-bde4b5d15a90"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:59:03.342113 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.342081 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "ed3ad499-6df7-4d49-98a2-bde4b5d15a90" (UID: "ed3ad499-6df7-4d49-98a2-bde4b5d15a90"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:59:03.342521 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.342489 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-online-tls" (OuterVolumeSpecName: "online-tls") pod "ed3ad499-6df7-4d49-98a2-bde4b5d15a90" (UID: "ed3ad499-6df7-4d49-98a2-bde4b5d15a90"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:59:03.342578 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.342552 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "ed3ad499-6df7-4d49-98a2-bde4b5d15a90" (UID: "ed3ad499-6df7-4d49-98a2-bde4b5d15a90"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:59:03.342615 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.342579 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ed3ad499-6df7-4d49-98a2-bde4b5d15a90" (UID: "ed3ad499-6df7-4d49-98a2-bde4b5d15a90"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:59:03.342615 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.342589 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-kube-api-access-qh2kr" (OuterVolumeSpecName: "kube-api-access-qh2kr") pod "ed3ad499-6df7-4d49-98a2-bde4b5d15a90" (UID: "ed3ad499-6df7-4d49-98a2-bde4b5d15a90"). InnerVolumeSpecName "kube-api-access-qh2kr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:59:03.440810 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.440769 2579 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-ui-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:59:03.440810 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.440804 2579 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-online-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:59:03.440810 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.440815 2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-registry-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:59:03.441055 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.440822 2579 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-offline-tls\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:59:03.441055 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.440830 2579 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-feast-data\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:59:03.441055 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.440838 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qh2kr\" (UniqueName: \"kubernetes.io/projected/ed3ad499-6df7-4d49-98a2-bde4b5d15a90-kube-api-access-qh2kr\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Sep 17 07:59:03.893113 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.893079 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" Sep 17 07:59:03.893594 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.893085 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x" event={"ID":"7ab59680-e713-4ba3-a035-68d936128d8a","Type":"ContainerDied","Data":"a8170a54ea6be310b441a777cd074e1926a40db4af2d046fc5cfc8293826d323"} Sep 17 07:59:03.893594 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.893237 2579 scope.go:117] "RemoveContainer" containerID="6d858e8bc38d31c18685280376c161580a6451a1f813840513f45e5ec6678ce2" Sep 17 07:59:03.896230 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.896198 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" event={"ID":"ed3ad499-6df7-4d49-98a2-bde4b5d15a90","Type":"ContainerDied","Data":"7c0daeb21e8991016d2642848e645f2f370de3519a1041833b124c4bd3da1f66"} Sep 17 07:59:03.896335 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.896233 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr" Sep 17 07:59:03.902406 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.902349 2579 scope.go:117] "RemoveContainer" containerID="0b7521471658f63ed61934daea00ba5d54ef9a16017d4d7e5e86589a13537f16" Sep 17 07:59:03.909664 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.909646 2579 scope.go:117] "RemoveContainer" containerID="c5991c0cb2b8d9feab9df603905795e166e3ed20b185f810c9beff5b1f00b831" Sep 17 07:59:03.916874 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.916830 2579 scope.go:117] "RemoveContainer" containerID="b935c8e0ebf142a14ab6d61aa278865258093e926358e4e18d43a2110678427d" Sep 17 07:59:03.917066 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.917044 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x"] Sep 17 07:59:03.922121 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.922098 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ts62x"] Sep 17 07:59:03.924355 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.924342 2579 scope.go:117] "RemoveContainer" containerID="d91d2af251eadf562f8cc41ee68a943d78b8de3d3a65b8e21f2fc8e342d78dba" Sep 17 07:59:03.930663 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.930645 2579 scope.go:117] "RemoveContainer" containerID="95673d98d14a5bfc73f1a7e5a9d2c22dee48d1e5aec5c92aceb5c0087d707fe6" Sep 17 07:59:03.934073 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.934052 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr"] Sep 17 07:59:03.937940 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.937923 2579 scope.go:117] "RemoveContainer" containerID="03e431ad824877ccc00be2d1facbff9e02c7820fee916740c1ba196e3f109a33" Sep 17 07:59:03.939610 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.939592 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-crpgr"] Sep 17 07:59:03.945162 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.945148 2579 scope.go:117] "RemoveContainer" containerID="f79effded7070011a92b13b694405dff52497f4a6f8c272f1933ff5a8b5fb603" Sep 17 07:59:03.951687 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.951672 2579 scope.go:117] "RemoveContainer" containerID="dd74f2d10a62429f043a284f012abdc31315582105e7630dc7111387ef38dab3" Sep 17 07:59:03.958412 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.958397 2579 scope.go:117] "RemoveContainer" containerID="b49e3c247fc70e51a57ca1cef4ab99d51dbd8b622178feb6067e00fefd6b3aa2" Sep 17 07:59:03.964777 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:03.964763 2579 scope.go:117] "RemoveContainer" containerID="1e58c3634535a311bb3bd7725cbd262e0effbfeb87d93e7d20b96afcc282e1de" Sep 17 07:59:04.310492 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:04.310463 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" path="/var/lib/kubelet/pods/7ab59680-e713-4ba3-a035-68d936128d8a/volumes" Sep 17 07:59:04.311008 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:04.310981 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" path="/var/lib/kubelet/pods/ed3ad499-6df7-4d49-98a2-bde4b5d15a90/volumes" Sep 17 07:59:36.153193 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153112 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tskbl/must-gather-fzp8r"] Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153401 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="ui" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153415 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="ui" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153425 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="registry" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153431 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="registry" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153440 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="online" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153447 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="online" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153453 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="online" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153457 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="online" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153463 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="ui" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153470 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="ui" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153477 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="feast-init" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153484 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="feast-init" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153490 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="feast-init" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153496 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="feast-init" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153504 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="offline" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153510 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="offline" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153520 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="feast-apply" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153525 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="feast-apply" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153532 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="offline" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153537 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="offline" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153542 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="feast-apply" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153547 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="feast-apply" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153595 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="online" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153604 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="online" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153611 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="offline" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153617 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ab59680-e713-4ba3-a035-68d936128d8a" containerName="ui" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153623 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="registry" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153628 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="offline" Sep 17 07:59:36.153647 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.153634 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed3ad499-6df7-4d49-98a2-bde4b5d15a90" containerName="ui" Sep 17 07:59:36.156362 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.156348 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tskbl/must-gather-fzp8r" Sep 17 07:59:36.168246 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.168228 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tskbl\"/\"kube-root-ca.crt\"" Sep 17 07:59:36.169283 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.169246 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tskbl\"/\"default-dockercfg-vpx4f\"" Sep 17 07:59:36.170260 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.170240 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tskbl/must-gather-fzp8r"] Sep 17 07:59:36.171395 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.171371 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tskbl\"/\"openshift-service-ca.crt\"" Sep 17 07:59:36.171788 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.171767 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjs6l\" (UniqueName: \"kubernetes.io/projected/d2244fc8-ae77-4367-be91-8777bec9600a-kube-api-access-rjs6l\") pod \"must-gather-fzp8r\" (UID: \"d2244fc8-ae77-4367-be91-8777bec9600a\") " pod="openshift-must-gather-tskbl/must-gather-fzp8r" Sep 17 07:59:36.171874 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.171849 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d2244fc8-ae77-4367-be91-8777bec9600a-must-gather-output\") pod \"must-gather-fzp8r\" (UID: \"d2244fc8-ae77-4367-be91-8777bec9600a\") " pod="openshift-must-gather-tskbl/must-gather-fzp8r" Sep 17 07:59:36.272172 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.272147 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d2244fc8-ae77-4367-be91-8777bec9600a-must-gather-output\") pod \"must-gather-fzp8r\" (UID: \"d2244fc8-ae77-4367-be91-8777bec9600a\") " pod="openshift-must-gather-tskbl/must-gather-fzp8r" Sep 17 07:59:36.272326 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.272195 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjs6l\" (UniqueName: \"kubernetes.io/projected/d2244fc8-ae77-4367-be91-8777bec9600a-kube-api-access-rjs6l\") pod \"must-gather-fzp8r\" (UID: \"d2244fc8-ae77-4367-be91-8777bec9600a\") " pod="openshift-must-gather-tskbl/must-gather-fzp8r" Sep 17 07:59:36.272445 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.272426 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d2244fc8-ae77-4367-be91-8777bec9600a-must-gather-output\") pod \"must-gather-fzp8r\" (UID: \"d2244fc8-ae77-4367-be91-8777bec9600a\") " pod="openshift-must-gather-tskbl/must-gather-fzp8r" Sep 17 07:59:36.282185 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.282161 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjs6l\" (UniqueName: \"kubernetes.io/projected/d2244fc8-ae77-4367-be91-8777bec9600a-kube-api-access-rjs6l\") pod \"must-gather-fzp8r\" (UID: \"d2244fc8-ae77-4367-be91-8777bec9600a\") " pod="openshift-must-gather-tskbl/must-gather-fzp8r" Sep 17 07:59:36.464706 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.464634 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tskbl/must-gather-fzp8r" Sep 17 07:59:36.740228 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.740151 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tskbl/must-gather-fzp8r"] Sep 17 07:59:36.743093 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:59:36.743066 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2244fc8_ae77_4367_be91_8777bec9600a.slice/crio-c079074cfe218e9f6bbda7f796b35800113784b12106def4f2f8c351ee8202a7 WatchSource:0}: Error finding container c079074cfe218e9f6bbda7f796b35800113784b12106def4f2f8c351ee8202a7: Status 404 returned error can't find the container with id c079074cfe218e9f6bbda7f796b35800113784b12106def4f2f8c351ee8202a7 Sep 17 07:59:36.744778 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:36.744759 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 07:59:37.006897 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:37.006814 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tskbl/must-gather-fzp8r" event={"ID":"d2244fc8-ae77-4367-be91-8777bec9600a","Type":"ContainerStarted","Data":"c079074cfe218e9f6bbda7f796b35800113784b12106def4f2f8c351ee8202a7"} Sep 17 07:59:38.013655 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:38.013620 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tskbl/must-gather-fzp8r" event={"ID":"d2244fc8-ae77-4367-be91-8777bec9600a","Type":"ContainerStarted","Data":"1484f075a6d729fd1321cbbaacf4ce1c3be280e5fbc7a99943a95139082efdc2"} Sep 17 07:59:38.013655 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:38.013665 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tskbl/must-gather-fzp8r" event={"ID":"d2244fc8-ae77-4367-be91-8777bec9600a","Type":"ContainerStarted","Data":"4239155445ed9fa74eebfc6bbfe4513f7daecd85ee4c20d6a30b636b9f96a2c6"} Sep 17 07:59:38.032826 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:38.032766 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tskbl/must-gather-fzp8r" podStartSLOduration=1.32558701 podStartE2EDuration="2.032743676s" totalImagesPullingTime="707.156666ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:59:38.031016156 +0000 UTC m=+1896.220992091" watchObservedRunningTime="2026-09-17 07:59:38.032743676 +0000 UTC m=+1896.222719619" Sep 17 07:59:38.910185 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:38.910156 2579 ???:1] "http: TLS handshake error from 10.0.129.217:46140: EOF" Sep 17 07:59:38.915261 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:38.915236 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mmcvp_d4580c7f-1c6c-420d-962c-0ba1a6a222fa/global-pull-secret-syncer/0.log" Sep 17 07:59:39.102397 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:39.102357 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-v46tc_073304c8-0913-4ef9-8388-920b0f7dd353/konnectivity-agent/0.log" Sep 17 07:59:39.126043 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:39.125989 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-217.ec2.internal_24d443373f3eb9148c2f63dcb77fe917/haproxy/0.log" Sep 17 07:59:43.363946 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:43.363914 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5wb9t_848dcf22-a209-4c6b-ad51-801dffe84fb0/node-exporter/0.log" Sep 17 07:59:43.395264 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:43.395234 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5wb9t_848dcf22-a209-4c6b-ad51-801dffe84fb0/kube-rbac-proxy/0.log" Sep 17 07:59:43.420936 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:43.420912 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5wb9t_848dcf22-a209-4c6b-ad51-801dffe84fb0/init-textfile/0.log" Sep 17 07:59:45.343118 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:45.343090 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-k74qf_fd1ed8f0-f91c-4789-915c-4f5dc7af25f6/networking-console-plugin/0.log" Sep 17 07:59:45.668655 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:45.668574 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8"] Sep 17 07:59:45.674050 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:45.674023 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" Sep 17 07:59:45.681912 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:45.681891 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8"] Sep 17 07:59:45.745830 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:45.745794 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de38c9f1-7c19-4718-897c-f5f0e9036d10-sys\") pod \"perf-node-gather-daemonset-wbsx8\" (UID: \"de38c9f1-7c19-4718-897c-f5f0e9036d10\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" Sep 17 07:59:45.746135 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:45.746113 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/de38c9f1-7c19-4718-897c-f5f0e9036d10-proc\") pod \"perf-node-gather-daemonset-wbsx8\" (UID: \"de38c9f1-7c19-4718-897c-f5f0e9036d10\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" Sep 17 07:59:45.746318 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:45.746301 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8l5z\" (UniqueName: \"kubernetes.io/projected/de38c9f1-7c19-4718-897c-f5f0e9036d10-kube-api-access-s8l5z\") pod \"perf-node-gather-daemonset-wbsx8\" (UID: \"de38c9f1-7c19-4718-897c-f5f0e9036d10\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" Sep 17 07:59:45.746457 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:45.746441 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de38c9f1-7c19-4718-897c-f5f0e9036d10-lib-modules\") pod \"perf-node-gather-daemonset-wbsx8\" (UID: \"de38c9f1-7c19-4718-897c-f5f0e9036d10\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" Sep 17 07:59:45.746570 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:45.746554 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/de38c9f1-7c19-4718-897c-f5f0e9036d10-podres\") pod \"perf-node-gather-daemonset-wbsx8\" (UID: \"de38c9f1-7c19-4718-897c-f5f0e9036d10\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" Sep 17 07:59:45.847413 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:45.847373 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de38c9f1-7c19-4718-897c-f5f0e9036d10-sys\") pod \"perf-node-gather-daemonset-wbsx8\" (UID: \"de38c9f1-7c19-4718-897c-f5f0e9036d10\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" Sep 17 07:59:45.847678 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:45.847659 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/de38c9f1-7c19-4718-897c-f5f0e9036d10-proc\") pod \"perf-node-gather-daemonset-wbsx8\" (UID: \"de38c9f1-7c19-4718-897c-f5f0e9036d10\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" Sep 17 07:59:45.847935 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:45.847917 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8l5z\" (UniqueName: \"kubernetes.io/projected/de38c9f1-7c19-4718-897c-f5f0e9036d10-kube-api-access-s8l5z\") pod \"perf-node-gather-daemonset-wbsx8\" (UID: \"de38c9f1-7c19-4718-897c-f5f0e9036d10\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" Sep 17 07:59:45.848489 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:45.848453 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de38c9f1-7c19-4718-897c-f5f0e9036d10-lib-modules\") pod \"perf-node-gather-daemonset-wbsx8\" (UID: \"de38c9f1-7c19-4718-897c-f5f0e9036d10\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" Sep 17 07:59:45.848583 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:45.848490 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/de38c9f1-7c19-4718-897c-f5f0e9036d10-podres\") pod \"perf-node-gather-daemonset-wbsx8\" (UID: \"de38c9f1-7c19-4718-897c-f5f0e9036d10\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" Sep 17 07:59:45.848629 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:45.848615 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de38c9f1-7c19-4718-897c-f5f0e9036d10-lib-modules\") pod \"perf-node-gather-daemonset-wbsx8\" (UID: \"de38c9f1-7c19-4718-897c-f5f0e9036d10\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" Sep 17 07:59:45.848670 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:45.848641 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/de38c9f1-7c19-4718-897c-f5f0e9036d10-podres\") pod \"perf-node-gather-daemonset-wbsx8\" (UID: \"de38c9f1-7c19-4718-897c-f5f0e9036d10\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" Sep 17 07:59:45.848670 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:45.847587 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de38c9f1-7c19-4718-897c-f5f0e9036d10-sys\") pod \"perf-node-gather-daemonset-wbsx8\" (UID: \"de38c9f1-7c19-4718-897c-f5f0e9036d10\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" Sep 17 07:59:45.848670 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:45.847831 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/de38c9f1-7c19-4718-897c-f5f0e9036d10-proc\") pod \"perf-node-gather-daemonset-wbsx8\" (UID: \"de38c9f1-7c19-4718-897c-f5f0e9036d10\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" Sep 17 07:59:45.858382 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:45.858332 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8l5z\" (UniqueName: \"kubernetes.io/projected/de38c9f1-7c19-4718-897c-f5f0e9036d10-kube-api-access-s8l5z\") pod \"perf-node-gather-daemonset-wbsx8\" (UID: \"de38c9f1-7c19-4718-897c-f5f0e9036d10\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" Sep 17 07:59:45.985154 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:45.985074 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" Sep 17 07:59:46.268626 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:46.268597 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8"] Sep 17 07:59:46.272333 ip-10-0-129-217 kubenswrapper[2579]: W0917 07:59:46.272298 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podde38c9f1_7c19_4718_897c_f5f0e9036d10.slice/crio-00122a744eb4c30c29142e26981553515004fed4fad5122cf0312a14a72508ae WatchSource:0}: Error finding container 00122a744eb4c30c29142e26981553515004fed4fad5122cf0312a14a72508ae: Status 404 returned error can't find the container with id 00122a744eb4c30c29142e26981553515004fed4fad5122cf0312a14a72508ae Sep 17 07:59:47.057203 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:47.057164 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" event={"ID":"de38c9f1-7c19-4718-897c-f5f0e9036d10","Type":"ContainerStarted","Data":"7a9f1fa98d95a2754a1f043bae9f0050875ab479e1fff49f38c2228849a0632d"} Sep 17 07:59:47.057777 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:47.057736 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" event={"ID":"de38c9f1-7c19-4718-897c-f5f0e9036d10","Type":"ContainerStarted","Data":"00122a744eb4c30c29142e26981553515004fed4fad5122cf0312a14a72508ae"} Sep 17 07:59:47.057911 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:47.057898 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" Sep 17 07:59:47.093887 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:47.093843 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" podStartSLOduration=2.093830457 podStartE2EDuration="2.093830457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:59:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:59:47.092359047 +0000 UTC m=+1905.282334990" watchObservedRunningTime="2026-09-17 07:59:47.093830457 +0000 UTC m=+1905.283806398" Sep 17 07:59:47.956336 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:47.956304 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zcjf5_1799de17-1341-4f16-9277-79056534a444/dns/0.log" Sep 17 07:59:47.986337 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:47.986312 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zcjf5_1799de17-1341-4f16-9277-79056534a444/kube-rbac-proxy/0.log" Sep 17 07:59:48.064320 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:48.064292 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dml6l_c327a988-59e8-48d2-a9fd-ff4668b66a22/dns-node-resolver/0.log" Sep 17 07:59:48.639989 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:48.639961 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gmkxm_d59a6e2b-8dd8-45ff-be5f-01ad01f02124/node-ca/0.log" Sep 17 07:59:49.979847 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:49.979796 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z6l74_c01ae36f-26da-4056-a557-7d45a7cc54f5/serve-healthcheck-canary/0.log" Sep 17 07:59:50.635416 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:50.635380 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qmmms_1187c234-335b-4635-8bee-126377c0b290/kube-rbac-proxy/0.log" Sep 17 07:59:50.660016 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:50.659974 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qmmms_1187c234-335b-4635-8bee-126377c0b290/exporter/0.log" Sep 17 07:59:50.687310 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:50.687281 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qmmms_1187c234-335b-4635-8bee-126377c0b290/extractor/0.log" Sep 17 07:59:54.081088 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:54.080319 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-wbsx8" Sep 17 07:59:55.178709 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:55.178683 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-gf8zr_ca74784d-18e9-487d-bc82-fa648083f4b3/migrator/0.log" Sep 17 07:59:55.202347 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:55.202317 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-gf8zr_ca74784d-18e9-487d-bc82-fa648083f4b3/graceful-termination/0.log" Sep 17 07:59:56.797939 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:56.797911 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rvnph_98b322c1-0c60-4b8d-b79b-32c529a63ccd/kube-multus-additional-cni-plugins/0.log" Sep 17 07:59:56.820869 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:56.820848 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rvnph_98b322c1-0c60-4b8d-b79b-32c529a63ccd/egress-router-binary-copy/0.log" Sep 17 07:59:56.843735 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:56.843714 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rvnph_98b322c1-0c60-4b8d-b79b-32c529a63ccd/cni-plugins/0.log" Sep 17 07:59:56.866737 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:56.866710 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rvnph_98b322c1-0c60-4b8d-b79b-32c529a63ccd/bond-cni-plugin/0.log" Sep 17 07:59:56.889007 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:56.888980 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rvnph_98b322c1-0c60-4b8d-b79b-32c529a63ccd/routeoverride-cni/0.log" Sep 17 07:59:56.912009 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:56.911973 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rvnph_98b322c1-0c60-4b8d-b79b-32c529a63ccd/whereabouts-cni-bincopy/0.log" Sep 17 07:59:56.933466 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:56.933444 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rvnph_98b322c1-0c60-4b8d-b79b-32c529a63ccd/whereabouts-cni/0.log" Sep 17 07:59:57.170582 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:57.170510 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-x7r9x_6fb6478a-4047-42e0-a66f-336860a6867c/kube-multus/0.log" Sep 17 07:59:57.298214 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:57.298186 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zjz9v_5eb544bb-f51e-4755-891e-ed3e8f031f0d/network-metrics-daemon/0.log" Sep 17 07:59:57.324158 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:57.324128 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zjz9v_5eb544bb-f51e-4755-891e-ed3e8f031f0d/kube-rbac-proxy/0.log" Sep 17 07:59:58.324755 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:58.324729 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nsjvb_42e9a882-f084-4752-af08-6911b3237ae2/ovn-controller/0.log" Sep 17 07:59:58.355327 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:58.355301 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nsjvb_42e9a882-f084-4752-af08-6911b3237ae2/ovn-acl-logging/0.log" Sep 17 07:59:58.383498 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:58.383471 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nsjvb_42e9a882-f084-4752-af08-6911b3237ae2/kube-rbac-proxy-node/0.log" Sep 17 07:59:58.408420 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:58.408394 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nsjvb_42e9a882-f084-4752-af08-6911b3237ae2/kube-rbac-proxy-ovn-metrics/0.log" Sep 17 07:59:58.432823 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:58.432786 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nsjvb_42e9a882-f084-4752-af08-6911b3237ae2/northd/0.log" Sep 17 07:59:58.461770 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:58.461749 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nsjvb_42e9a882-f084-4752-af08-6911b3237ae2/nbdb/0.log" Sep 17 07:59:58.486160 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:58.486138 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nsjvb_42e9a882-f084-4752-af08-6911b3237ae2/sbdb/0.log" Sep 17 07:59:58.528944 ip-10-0-129-217 kubenswrapper[2579]: I0917 07:59:58.528918 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nsjvb_42e9a882-f084-4752-af08-6911b3237ae2/ovnkube-controller/0.log" Sep 17 08:00:00.034403 ip-10-0-129-217 kubenswrapper[2579]: I0917 08:00:00.034365 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-njpsg_e7339f6e-91b9-4b00-b7aa-915f9ea79232/network-check-target-container/0.log" Sep 17 08:00:01.224454 ip-10-0-129-217 kubenswrapper[2579]: I0917 08:00:01.224423 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-pm5vm_1c554023-f031-4773-b6ef-ef864215ba80/iptables-alerter/0.log" Sep 17 08:00:02.148749 ip-10-0-129-217 kubenswrapper[2579]: I0917 08:00:02.148698 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-xfn69_c538774b-3450-4093-84a6-7cd9f9a0167e/tuned/0.log"