2026-09-17T08:23:58.261844688Z [disk usage checker] Started 2026-09-17T08:23:58.265886676Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-17T08:23:58.467993918Z Gathering data for ns/openshift-cluster-version... 2026-09-17T08:23:58.614964641Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-17T08:23:58.696183927Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-17T08:23:58.777881042Z Gathering data for ns/default... 2026-09-17T08:23:59.060210502Z Gathering data for ns/openshift... 2026-09-17T08:23:59.448893703Z Gathering data for ns/kube-system... 2026-09-17T08:24:00.143155324Z Gathering data for ns/openshift-etcd... 2026-09-17T08:24:00.426408522Z Wrote inspect data to must-gather. 2026-09-17T08:24:00.426439382Z error: inspection completed with the errors occurred while gathering data: 2026-09-17T08:24:00.426439382Z namespaces "assisted-installer" not found 2026-09-17T08:24:00.735836878Z Waiting on subprocesses to finish execution. 2026-09-17T08:24:00.739995692Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-17T08:24:00.747307809Z INFO: Gathering HAProxy config files 2026-09-17T08:24:00.750326087Z Executing Istio gather script 2026-09-17T08:24:00.755638904Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-17T08:24:00.756642354Z INFO: Collecting host service logs for crio 2026-09-17T08:24:00.756969020Z INFO: Collecting host service logs for kubelet 2026-09-17T08:24:00.757284580Z INFO: Collecting host service logs for rpm-ostreed 2026-09-17T08:24:00.757559214Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-17T08:24:00.757897484Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-17T08:24:00.758255413Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-17T08:24:00.758652167Z INFO: Collecting host service logs for NetworkManager 2026-09-17T08:24:00.758998984Z INFO: Collecting host service logs for openvswitch 2026-09-17T08:24:00.759306542Z INFO: Collecting host service logs for ovs-configuration 2026-09-17T08:24:00.759631443Z INFO: Collecting host service logs for ovsdb-server 2026-09-17T08:24:00.759952694Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-17T08:24:00.760309657Z INFO: Waiting for worker host service log collection to complete ... 2026-09-17T08:24:00.761540953Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-17T08:24:00.823342625Z INFO: Waiting for node performance related collection to complete ... 2026-09-17T08:24:01.259472358Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-17T08:24:01.297469112Z INFO: "metallb-operator" not detected. Skipping. 2026-09-17T08:24:01.688825400Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-17T08:24:01.688904886Z INFO: on-disk MachineConfig config collection complete. 2026-09-17T08:24:01.772306788Z No resources found 2026-09-17T08:24:01.875623719Z error: the server doesn't have a resource type "performanceprofile" 2026-09-17T08:24:01.894667772Z No resources found in openshift-etcd namespace. 2026-09-17T08:24:01.937539217Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-17T08:24:02.075189748Z ERROR: No running kube-apiserver pods found 2026-09-17T08:24:02.114272008Z Wrote inspect data to must-gather. 2026-09-17T08:24:02.130954428Z INFO: Collecting Insights Archives from insights-operator-5844b856db-2sbb6 2026-09-17T08:24:02.130996847Z insights-runtime-extractor-ckgsp 2026-09-17T08:24:02.131017042Z insights-runtime-extractor-q9jxl 2026-09-17T08:24:02.131037500Z insights-runtime-extractor-rqmfb 2026-09-17T08:24:02.171973804Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-17T08:24:02.172041379Z Wrote inspect data to must-gather. 2026-09-17T08:24:02.175529896Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-17T08:24:02.234505348Z INFO: Worker host service log collection to complete. 2026-09-17T08:24:02.603957113Z Wrote inspect data to must-gather. 2026-09-17T08:24:02.618056258Z INFO: 'previous-logs' folder not found on ip-10-0-131-153.ec2.internal, skipping... 2026-09-17T08:24:02.640921689Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-17T08:24:02.663548248Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-17T08:24:02.702769997Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-17T08:24:02.702769997Z See 'oc rsync -h' for help and examples 2026-09-17T08:24:02.784085467Z Gathering data for ns/openshift-monitoring... 2026-09-17T08:24:02.856060247Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-17T08:24:03.057183112Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-17T08:24:03.121278220Z Wrote inspect data to must-gather. 2026-09-17T08:24:03.223361737Z Wrote inspect data to must-gather. 2026-09-17T08:24:03.274244555Z Wrote inspect data to must-gather. 2026-09-17T08:24:03.291761285Z No resources found 2026-09-17T08:24:03.301932264Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-17T08:24:03.334480379Z Wrote inspect data to must-gather. 2026-09-17T08:24:03.392698545Z error: the server doesn't have a resource type "machineconfigs" 2026-09-17T08:24:03.456426012Z INFO: 'previous-logs' folder not found on ip-10-0-133-67.ec2.internal, skipping... 2026-09-17T08:24:03.505010465Z tar: Removing leading `/' from member names 2026-09-17T08:24:03.508860748Z INFO: HAProxy config collection complete. 2026-09-17T08:24:03.515444915Z INFO: Getting rules from prometheus-k8s-0 2026-09-17T08:24:03.529085329Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-17T08:24:03.587501316Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-17T08:24:03.598173752Z Wrote inspect data to must-gather. 2026-09-17T08:24:03.720337656Z INFO: 'previous-logs' folder not found on ip-10-0-137-245.ec2.internal, skipping... 2026-09-17T08:24:03.720448992Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-17T08:24:03.720507605Z INFO: Machine Config Daemon termination log collection complete. 2026-09-17T08:24:03.800704078Z Wrote inspect data to must-gather. 2026-09-17T08:24:03.839655061Z Wrote inspect data to must-gather. 2026-09-17T08:24:03.917570889Z INFO: Getting status/config from prometheus-k8s-0 2026-09-17T08:24:03.984593584Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-17T08:24:04.088234262Z Wrote inspect data to must-gather. 2026-09-17T08:24:04.183855612Z Wrote inspect data to must-gather. 2026-09-17T08:24:04.314862191Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-17T08:24:04.321155179Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-17T08:24:04.377586555Z INFO: INTERCONNECT MODE 2026-09-17T08:24:04.377632857Z INFO: Gathering ovn-kubernetes DBs 2026-09-17T08:24:04.404718450Z Wrote inspect data to must-gather. 2026-09-17T08:24:04.499050228Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-17T08:24:04.536463448Z Wrote inspect data to /must-gather/istio. 2026-09-17T08:24:04.542432862Z INFO: Gathering OVN_Northbound from ovnkube-node-5nkj8... 2026-09-17T08:24:04.542679967Z INFO: Gathering OVN_Northbound from ovnkube-node-9fv5v... 2026-09-17T08:24:04.543000346Z INFO: Gathering OVN_Northbound from ovnkube-node-r2mvv... 2026-09-17T08:24:04.543292545Z INFO: Gathering OVN_Southbound from ovnkube-node-5nkj8... 2026-09-17T08:24:04.543621097Z INFO: Gathering OVN_Southbound from ovnkube-node-9fv5v... 2026-09-17T08:24:04.543908668Z INFO: Gathering OVN_Southbound from ovnkube-node-r2mvv... 2026-09-17T08:24:04.610138892Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-17T08:24:04.646887280Z Wrote inspect data to must-gather. 2026-09-17T08:24:04.850242727Z tar: Removing leading `/' from member names 2026-09-17T08:24:04.860020602Z tar: Removing leading `/' from member names 2026-09-17T08:24:04.866907143Z tar: Removing leading `/' from member names 2026-09-17T08:24:04.889011978Z tar: Removing leading `/' from member names 2026-09-17T08:24:04.909354586Z tar: Removing leading `/' from member names 2026-09-17T08:24:04.919964230Z tar: Removing leading `/' from member names 2026-09-17T08:24:04.942871809Z Wrote inspect data to /must-gather/istio. 2026-09-17T08:24:04.948668401Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-17T08:24:04.997953011Z Wrote inspect data to must-gather. 2026-09-17T08:24:05.000674988Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-17T08:24:05.169306326Z Wrote inspect data to /must-gather/istio. 2026-09-17T08:24:05.243202175Z W0917 08:24:05.243155 1468 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-09-17T08:24:05.249112003Z Wrote inspect data to must-gather. 2026-09-17T08:24:05.252672384Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-17T08:24:05.359004495Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-17T08:24:05.394622459Z Wrote inspect data to /must-gather/istio. 2026-09-17T08:24:05.397750743Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-17T08:24:05.450477494Z INFO: Waiting for network log collection to complete ... 2026-09-17T08:24:05.450510060Z INFO: Waiting for ovnk database copies to complete ... 2026-09-17T08:24:05.450909724Z INFO: Copying ovnk databases complete. 2026-09-17T08:24:05.452700474Z 2.5M must-gather/network_logs/ovnk_database_store 2026-09-17T08:24:05.455639078Z ovnk_database_store/ 2026-09-17T08:24:05.455687093Z ovnk_database_store/ovnkube-node-r2mvv_sbdb 2026-09-17T08:24:05.476769677Z ovnk_database_store/ovnkube-node-5nkj8_nbdb 2026-09-17T08:24:05.483123130Z ovnk_database_store/ovnkube-node-9fv5v_sbdb 2026-09-17T08:24:05.501533001Z ovnk_database_store/ovnkube-node-5nkj8_sbdb 2026-09-17T08:24:05.518941663Z ovnk_database_store/ovnkube-node-9fv5v_nbdb 2026-09-17T08:24:05.526114293Z ovnk_database_store/ovnkube-node-r2mvv_nbdb 2026-09-17T08:24:05.536858608Z INFO: Network log collection complete. 2026-09-17T08:24:05.548435480Z Gathering data for ns/openshift-network-console... 2026-09-17T08:24:05.577962690Z Wrote inspect data to must-gather. 2026-09-17T08:24:05.609255295Z Wrote inspect data to /must-gather/istio. 2026-09-17T08:24:05.674863905Z INFO: Getting status from alertmanager-main-0 2026-09-17T08:24:05.830474560Z Wrote inspect data to /must-gather/istio. 2026-09-17T08:24:05.833160548Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-17T08:24:05.892613298Z Wrote inspect data to must-gather. 2026-09-17T08:24:06.000377330Z Gathering data for ns/openshift-console-operator... 2026-09-17T08:24:06.057413984Z Wrote inspect data to /must-gather/istio. 2026-09-17T08:24:06.163981669Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-17T08:24:06.166113329Z INFO: Fallback to identify the container image from release info 2026-09-17T08:24:06.255857425Z Wrote inspect data to /must-gather/istio. 2026-09-17T08:24:06.258209637Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-17T08:24:06.389592582Z Gathering data for ns/openshift-console... 2026-09-17T08:24:06.412747774Z Wrote inspect data to /must-gather/istio. 2026-09-17T08:24:06.583483850Z Wrote inspect data to /must-gather/istio. 2026-09-17T08:24:06.755474411Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a 2026-09-17T08:24:06.916945056Z daemonset.apps/perf-node-gather-daemonset created 2026-09-17T08:24:06.957296093Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-17T08:24:07.053307513Z Waiting for performance profile collector pods to become ready: 1 2026-09-17T08:24:07.305123276Z Done executing Istio gather script 2026-09-17T08:24:07.526161138Z Gathering data for ns/openshift-dns-operator... 2026-09-17T08:24:07.781159890Z Gathering data for ns/openshift-dns... 2026-09-17T08:24:08.163434473Z Waiting for performance profile collector pods to become ready: 2 2026-09-17T08:24:08.315804977Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-17T08:24:08.385313515Z Gathering data for ns/openshift-image-registry... 2026-09-17T08:24:08.828095060Z Gathering data for ns/openshift-ingress-operator... 2026-09-17T08:24:09.125825262Z Gathering data for ns/openshift-ingress... 2026-09-17T08:24:09.278476298Z Waiting for performance profile collector pods to become ready: 3 2026-09-17T08:24:09.433125643Z Gathering data for ns/openshift-ingress-canary... 2026-09-17T08:24:09.787426504Z Gathering data for ns/openshift-insights... 2026-09-17T08:24:10.402601133Z Waiting for performance profile collector pods to become ready: 4 2026-09-17T08:24:11.507392612Z Waiting for performance profile collector pods to become ready: 5 2026-09-17T08:24:11.645293288Z Gathering data for ns/openshift-config... 2026-09-17T08:24:11.991664491Z Gathering data for ns/openshift-config-managed... 2026-09-17T08:24:12.470103010Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-17T08:24:12.615045084Z Waiting for performance profile collector pods to become ready: 6 2026-09-17T08:24:12.891828676Z Gathering data for ns/openshift-kube-apiserver... 2026-09-17T08:24:13.238674582Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-17T08:24:13.321175986Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-17T08:24:13.606564978Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-17T08:24:13.732698773Z Waiting for performance profile collector pods to become ready: 7 2026-09-17T08:24:13.958952128Z Gathering data for ns/openshift-kube-scheduler... 2026-09-17T08:24:14.363010025Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-17T08:24:14.732186447Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-17T08:24:14.832410593Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-17T08:24:15.088566711Z Collecting performance related data for node ip-10-0-133-67.ec2.internal 2026-09-17T08:24:15.097864090Z Collecting performance related data for node ip-10-0-137-245.ec2.internal 2026-09-17T08:24:15.106988864Z Collecting performance related data for node ip-10-0-131-153.ec2.internal 2026-09-17T08:24:15.139597661Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-17T08:24:15.529111509Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-17T08:24:16.136460567Z Gathering data for ns/openshift-multus... 2026-09-17T08:24:18.035324521Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-17T08:24:18.326139089Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-17T08:24:19.388435970Z Gathering data for ns/openshift-host-network... 2026-09-17T08:24:19.756892800Z Gathering data for ns/openshift-network-diagnostics... 2026-09-17T08:24:20.314708780Z Gathering data for ns/openshift-network-node-identity... 2026-09-17T08:24:20.761812772Z Gathering data for ns/openshift-network-operator... 2026-09-17T08:24:21.160200433Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-17T08:24:21.438757077Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-17T08:24:21.929720779Z Gathering data for ns/openshift-apiserver-operator... 2026-09-17T08:24:22.232594858Z Gathering data for ns/openshift-apiserver... 2026-09-17T08:24:22.599507083Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-17T08:24:22.856629306Z Gathering data for ns/openshift-controller-manager... 2026-09-17T08:24:23.101503765Z Collecting kubelet logs for node ip-10-0-133-67.ec2.internal 2026-09-17T08:24:23.104378762Z Collecting kubelet logs for node ip-10-0-137-245.ec2.internal 2026-09-17T08:24:23.107248747Z Collecting kubelet logs for node ip-10-0-131-153.ec2.internal 2026-09-17T08:24:23.118154890Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-17T08:24:23.331338408Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-17T08:24:23.457319734Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-g6v9x namespace 2026-09-17T08:24:23.463052151Z INFO: Node performance data collection complete. 2026-09-17T08:24:23.806069388Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-17T08:24:24.068880652Z Gathering data for ns/openshift-service-ca-operator... 2026-09-17T08:24:24.367759452Z Gathering data for ns/openshift-service-ca... 2026-09-17T08:24:24.692650684Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-17T08:24:25.551461628Z Wrote inspect data to must-gather. 2026-09-17T08:24:25.551511364Z error: inspection completed with the errors occurred while gathering data: 2026-09-17T08:24:25.551511364Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-17T08:24:25.672326273Z error: the server doesn't have a resource type "clusters" 2026-09-17T08:24:25.795339215Z Caches written to disk