2026-07-29T17:33:32.369683640Z [disk usage checker] Started 2026-07-29T17:33:32.373016650Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-29T17:33:32.559122376Z Gathering data for ns/openshift-cluster-version... 2026-07-29T17:33:32.680784613Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-29T17:33:32.764521868Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-29T17:33:32.843064556Z Gathering data for ns/default... 2026-07-29T17:33:33.106720762Z Gathering data for ns/openshift... 2026-07-29T17:33:33.486181138Z Gathering data for ns/kube-system... 2026-07-29T17:33:34.163401990Z Gathering data for ns/openshift-etcd... 2026-07-29T17:33:34.437566160Z Wrote inspect data to must-gather. 2026-07-29T17:33:34.437653702Z error: inspection completed with the errors occurred while gathering data: 2026-07-29T17:33:34.437653702Z namespaces "assisted-installer" not found 2026-07-29T17:33:34.794471518Z Waiting on subprocesses to finish execution. 2026-07-29T17:33:34.802428062Z INFO: Gathering machine config daemon's old logs from all nodes 2026-07-29T17:33:34.805500774Z Executing Istio gather script 2026-07-29T17:33:34.808894522Z INFO: Gathering HAProxy config files 2026-07-29T17:33:34.811505528Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-07-29T17:33:34.812424413Z INFO: Collecting host service logs for crio 2026-07-29T17:33:34.812717820Z INFO: Collecting host service logs for kubelet 2026-07-29T17:33:34.813031018Z INFO: Collecting host service logs for rpm-ostreed 2026-07-29T17:33:34.813311894Z INFO: Collecting host service logs for ostree-finalize-staged 2026-07-29T17:33:34.813603296Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-07-29T17:33:34.814009202Z INFO: Collecting host service logs for machine-config-daemon-host 2026-07-29T17:33:34.814326583Z INFO: Collecting host service logs for NetworkManager 2026-07-29T17:33:34.814616545Z INFO: Collecting host service logs for openvswitch 2026-07-29T17:33:34.814909772Z INFO: Collecting host service logs for ovs-configuration 2026-07-29T17:33:34.815324732Z INFO: Collecting host service logs for ovsdb-server 2026-07-29T17:33:34.815628353Z INFO: Collecting host service logs for ovs-vswitchd 2026-07-29T17:33:34.815985205Z INFO: Waiting for worker host service log collection to complete ... 2026-07-29T17:33:34.819547617Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-07-29T17:33:34.833237331Z INFO: Waiting for node performance related collection to complete ... 2026-07-29T17:33:35.141655464Z No resources found in openshift-etcd namespace. 2026-07-29T17:33:35.266552126Z error: the server doesn't have a resource type "performanceprofile" 2026-07-29T17:33:35.380417824Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-xl4jn 2026-07-29T17:33:35.380467923Z insights-runtime-extractor-nbhz9 2026-07-29T17:33:35.380488129Z insights-runtime-extractor-qs29g 2026-07-29T17:33:35.380508178Z insights-runtime-extractor-rwxvt 2026-07-29T17:33:35.410733089Z INFO: Found 1 replicas - prometheus-k8s-0 2026-07-29T17:33:35.413014322Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-07-29T17:33:35.413087566Z INFO: on-disk MachineConfig config collection complete. 2026-07-29T17:33:35.735048060Z INFO: "metallb-operator" not detected. Skipping. 2026-07-29T17:33:35.851175184Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-07-29T17:33:35.896134750Z INFO: "sriov-network-operator" not detected. Skipping. 2026-07-29T17:33:35.976007236Z No resources found 2026-07-29T17:33:36.326499554Z Wrote inspect data to must-gather. 2026-07-29T17:33:36.572092772Z INFO: Worker host service log collection to complete. 2026-07-29T17:33:36.575771676Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-07-29T17:33:36.575771676Z See 'oc rsync -h' for help and examples 2026-07-29T17:33:36.720141056Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-07-29T17:33:36.849396860Z Wrote inspect data to must-gather. 2026-07-29T17:33:36.871199783Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-07-29T17:33:36.893282326Z Gathering data for ns/openshift-monitoring... 2026-07-29T17:33:36.947086499Z ERROR: No running kube-apiserver pods found 2026-07-29T17:33:36.998474157Z Wrote inspect data to must-gather. 2026-07-29T17:33:37.040560556Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-29T17:33:37.091673639Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-07-29T17:33:37.135110789Z Wrote inspect data to must-gather. 2026-07-29T17:33:37.141854362Z INFO: 'previous-logs' folder not found on ip-10-0-128-10.ec2.internal, skipping... 2026-07-29T17:33:37.142186553Z No resources found 2026-07-29T17:33:37.268362653Z INFO: Waiting for HAProxy config collection to complete ... 2026-07-29T17:33:37.279914233Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-07-29T17:33:37.290625943Z Wrote inspect data to must-gather. 2026-07-29T17:33:37.313660059Z Wrote inspect data to must-gather. 2026-07-29T17:33:37.313660059Z Wrote inspect data to must-gather. 2026-07-29T17:33:37.359643224Z error: the server doesn't have a resource type "machineconfigs" 2026-07-29T17:33:37.379136425Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-29T17:33:37.499928212Z INFO: 'previous-logs' folder not found on ip-10-0-139-26.ec2.internal, skipping... 2026-07-29T17:33:37.527433899Z tar: Removing leading `/' from member names 2026-07-29T17:33:37.533411689Z INFO: HAProxy config collection complete. 2026-07-29T17:33:37.553102933Z error: the server doesn't have a resource type "machineconfigpools" 2026-07-29T17:33:37.595750779Z Wrote inspect data to must-gather. 2026-07-29T17:33:37.668178112Z INFO: Getting rules from prometheus-k8s-0 2026-07-29T17:33:37.776743434Z INFO: 'previous-logs' folder not found on ip-10-0-142-180.ec2.internal, skipping... 2026-07-29T17:33:37.776848026Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-07-29T17:33:37.776901993Z INFO: Machine Config Daemon termination log collection complete. 2026-07-29T17:33:37.784670316Z Wrote inspect data to must-gather. 2026-07-29T17:33:37.856408355Z Wrote inspect data to must-gather. 2026-07-29T17:33:37.927716724Z error: the server doesn't have a resource type "kubeletconfigs" 2026-07-29T17:33:38.087859582Z INFO: Getting status/config from prometheus-k8s-0 2026-07-29T17:33:38.104191484Z Wrote inspect data to must-gather. 2026-07-29T17:33:38.150417607Z Wrote inspect data to must-gather. 2026-07-29T17:33:38.282379624Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-07-29T17:33:38.342680717Z Wrote inspect data to must-gather. 2026-07-29T17:33:38.376703291Z INFO: INTERCONNECT MODE 2026-07-29T17:33:38.376737625Z INFO: Gathering ovn-kubernetes DBs 2026-07-29T17:33:38.508878326Z Wrote inspect data to /must-gather/istio. 2026-07-29T17:33:38.584062286Z Wrote inspect data to must-gather. 2026-07-29T17:33:38.614129000Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-29T17:33:38.617584318Z INFO: Getting status/flags from prometheus-k8s-0 2026-07-29T17:33:38.630357941Z INFO: Gathering OVN_Northbound from ovnkube-node-94qqn... 2026-07-29T17:33:38.630576448Z INFO: Gathering OVN_Northbound from ovnkube-node-9h7jk... 2026-07-29T17:33:38.630781637Z INFO: Gathering OVN_Northbound from ovnkube-node-hsd8t... 2026-07-29T17:33:38.631072289Z INFO: Gathering OVN_Southbound from ovnkube-node-94qqn... 2026-07-29T17:33:38.631316313Z INFO: Gathering OVN_Southbound from ovnkube-node-9h7jk... 2026-07-29T17:33:38.631595995Z INFO: Gathering OVN_Southbound from ovnkube-node-hsd8t... 2026-07-29T17:33:38.836270602Z Wrote inspect data to /must-gather/istio. 2026-07-29T17:33:38.840522795Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-07-29T17:33:38.974226243Z Wrote inspect data to must-gather. 2026-07-29T17:33:39.003782894Z tar: Removing leading `/' from member names 2026-07-29T17:33:39.005241919Z tar: Removing leading `/' from member names 2026-07-29T17:33:39.010020551Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-07-29T17:33:39.016881416Z tar: Removing leading `/' from member names 2026-07-29T17:33:39.025488343Z tar: Removing leading `/' from member names 2026-07-29T17:33:39.029563258Z tar: Removing leading `/' from member names 2026-07-29T17:33:39.045513032Z tar: Removing leading `/' from member names 2026-07-29T17:33:39.145641157Z Wrote inspect data to /must-gather/istio. 2026-07-29T17:33:39.224287663Z Wrote inspect data to must-gather. 2026-07-29T17:33:39.284366830Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-07-29T17:33:39.388814864Z Wrote inspect data to /must-gather/istio. 2026-07-29T17:33:39.391792233Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-07-29T17:33:39.425930539Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-29T17:33:39.542833128Z INFO: Waiting for network log collection to complete ... 2026-07-29T17:33:39.542867117Z INFO: Waiting for ovnk database copies to complete ... 2026-07-29T17:33:39.543265523Z INFO: Copying ovnk databases complete. 2026-07-29T17:33:39.547253182Z 2.4M must-gather/network_logs/ovnk_database_store 2026-07-29T17:33:39.550101278Z ovnk_database_store/ 2026-07-29T17:33:39.550147178Z ovnk_database_store/ovnkube-node-hsd8t_sbdb 2026-07-29T17:33:39.564180223Z ovnk_database_store/ovnkube-node-94qqn_nbdb 2026-07-29T17:33:39.566155448Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-07-29T17:33:39.570131097Z ovnk_database_store/ovnkube-node-94qqn_sbdb 2026-07-29T17:33:39.576033355Z Wrote inspect data to must-gather. 2026-07-29T17:33:39.588682054Z ovnk_database_store/ovnkube-node-9h7jk_nbdb 2026-07-29T17:33:39.594125224Z ovnk_database_store/ovnkube-node-hsd8t_nbdb 2026-07-29T17:33:39.599737391Z ovnk_database_store/ovnkube-node-9h7jk_sbdb 2026-07-29T17:33:39.606684130Z Wrote inspect data to /must-gather/istio. 2026-07-29T17:33:39.621327226Z INFO: Network log collection complete. 2026-07-29T17:33:39.634356054Z Gathering data for ns/openshift-network-console... 2026-07-29T17:33:39.808089126Z Wrote inspect data to /must-gather/istio. 2026-07-29T17:33:39.811224594Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-07-29T17:33:39.872234251Z Wrote inspect data to must-gather. 2026-07-29T17:33:39.956416429Z INFO: Getting status from alertmanager-main-0 2026-07-29T17:33:40.048439628Z Wrote inspect data to /must-gather/istio. 2026-07-29T17:33:40.112481111Z Gathering data for ns/openshift-console-operator... 2026-07-29T17:33:40.148490995Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-07-29T17:33:40.151637862Z INFO: Fallback to identify the container image from release info 2026-07-29T17:33:40.251022984Z Wrote inspect data to /must-gather/istio. 2026-07-29T17:33:40.253194769Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-07-29T17:33:40.401259022Z Wrote inspect data to /must-gather/istio. 2026-07-29T17:33:40.524078788Z Gathering data for ns/openshift-console... 2026-07-29T17:33:40.579050757Z Wrote inspect data to /must-gather/istio. 2026-07-29T17:33:41.004253373Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811 2026-07-29T17:33:41.012093091Z Gathering data for ns/openshift-cluster-storage-operator... 2026-07-29T17:33:41.134340788Z daemonset.apps/perf-node-gather-daemonset created 2026-07-29T17:33:41.236354234Z Done executing Istio gather script 2026-07-29T17:33:41.251228782Z Waiting for performance profile collector pods to become ready: 1 2026-07-29T17:33:41.488184552Z Gathering data for ns/openshift-dns-operator... 2026-07-29T17:33:41.770193577Z Gathering data for ns/openshift-dns... 2026-07-29T17:33:42.303596837Z Gathering data for ns/openshift-image-registry... 2026-07-29T17:33:42.361492893Z Waiting for performance profile collector pods to become ready: 2 2026-07-29T17:33:42.386432214Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-29T17:33:42.710424608Z Gathering data for ns/openshift-ingress-operator... 2026-07-29T17:33:42.989699577Z Gathering data for ns/openshift-ingress... 2026-07-29T17:33:43.284577309Z Gathering data for ns/openshift-ingress-canary... 2026-07-29T17:33:43.481687361Z Waiting for performance profile collector pods to become ready: 3 2026-07-29T17:33:43.644902751Z Gathering data for ns/openshift-insights... 2026-07-29T17:33:44.576313428Z Waiting for performance profile collector pods to become ready: 4 2026-07-29T17:33:45.361065240Z Gathering data for ns/openshift-config... 2026-07-29T17:33:45.585549584Z Gathering data for ns/openshift-config-managed... 2026-07-29T17:33:45.690706745Z Waiting for performance profile collector pods to become ready: 5 2026-07-29T17:33:45.963900410Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-07-29T17:33:46.199655683Z Gathering data for ns/openshift-kube-apiserver... 2026-07-29T17:33:46.420511840Z Gathering data for ns/openshift-kube-controller-manager... 2026-07-29T17:33:46.664058604Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-07-29T17:33:46.796350313Z Waiting for performance profile collector pods to become ready: 6 2026-07-29T17:33:46.966883750Z Gathering data for ns/openshift-kube-scheduler... 2026-07-29T17:33:47.207647733Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-07-29T17:33:47.391242300Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-29T17:33:47.468549321Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-07-29T17:33:47.763488257Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-07-29T17:33:47.912080020Z Waiting for performance profile collector pods to become ready: 7 2026-07-29T17:33:48.067363880Z Gathering data for ns/openshift-user-workload-monitoring... 2026-07-29T17:33:48.582196223Z Gathering data for ns/openshift-multus... 2026-07-29T17:33:49.009116497Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-07-29T17:33:49.217648689Z Collecting performance related data for node ip-10-0-142-180.ec2.internal 2026-07-29T17:33:49.224421971Z Collecting performance related data for node ip-10-0-128-10.ec2.internal 2026-07-29T17:33:49.231415944Z Collecting performance related data for node ip-10-0-139-26.ec2.internal 2026-07-29T17:33:50.085198139Z Gathering data for ns/openshift-ovn-kubernetes... 2026-07-29T17:33:51.338382742Z Gathering data for ns/openshift-host-network... 2026-07-29T17:33:51.656378364Z Gathering data for ns/openshift-network-diagnostics... 2026-07-29T17:33:52.191387512Z Gathering data for ns/openshift-network-node-identity... 2026-07-29T17:33:52.395669591Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-29T17:33:52.557249771Z Gathering data for ns/openshift-network-operator... 2026-07-29T17:33:52.914379455Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-07-29T17:33:53.205879415Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-07-29T17:33:53.667129934Z Gathering data for ns/openshift-apiserver-operator... 2026-07-29T17:33:53.981122488Z Gathering data for ns/openshift-apiserver... 2026-07-29T17:33:54.342632902Z Gathering data for ns/openshift-controller-manager-operator... 2026-07-29T17:33:54.576688096Z Gathering data for ns/openshift-controller-manager... 2026-07-29T17:33:54.829099768Z Gathering data for ns/openshift-cluster-samples-operator... 2026-07-29T17:33:55.456214745Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-07-29T17:33:55.709876389Z Gathering data for ns/openshift-service-ca-operator... 2026-07-29T17:33:56.007999841Z Gathering data for ns/openshift-service-ca... 2026-07-29T17:33:56.334813813Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-07-29T17:33:57.184030954Z Wrote inspect data to must-gather. 2026-07-29T17:33:57.184071856Z error: inspection completed with the errors occurred while gathering data: 2026-07-29T17:33:57.184071856Z skipping gathering secrets/support due to error: secrets "support" not found 2026-07-29T17:33:57.401356391Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-29T17:33:57.410853634Z Collecting kubelet logs for node ip-10-0-142-180.ec2.internal 2026-07-29T17:33:57.412613406Z Collecting kubelet logs for node ip-10-0-128-10.ec2.internal 2026-07-29T17:33:57.414300695Z Collecting kubelet logs for node ip-10-0-139-26.ec2.internal 2026-07-29T17:33:57.695774974Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-f976x namespace 2026-07-29T17:33:57.700616672Z INFO: Node performance data collection complete. 2026-07-29T17:33:57.816438750Z error: the server doesn't have a resource type "clusters" 2026-07-29T17:33:57.969733208Z Caches written to disk