2026-09-16T17:52:23.560730224Z [disk usage checker] Started 2026-09-16T17:52:23.564199113Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-09-16T17:52:23.765187393Z Gathering data for ns/openshift-cluster-version... 2026-09-16T17:52:23.914146551Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-16T17:52:24.040780128Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-16T17:52:24.126098054Z Gathering data for ns/default... 2026-09-16T17:52:24.487352963Z Gathering data for ns/openshift... 2026-09-16T17:52:24.940661739Z Gathering data for ns/kube-system... 2026-09-16T17:52:25.799632729Z Gathering data for ns/openshift-etcd... 2026-09-16T17:52:26.293288835Z Wrote inspect data to must-gather. 2026-09-16T17:52:26.293334782Z error: inspection completed with the errors occurred while gathering data: 2026-09-16T17:52:26.293334782Z namespaces "assisted-installer" not found 2026-09-16T17:52:26.378000875Z Waiting on subprocesses to finish execution. 2026-09-16T17:52:26.382603815Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-16T17:52:26.385380868Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-16T17:52:26.395260987Z INFO: Gathering HAProxy config files 2026-09-16T17:52:26.396300555Z Executing Istio gather script 2026-09-16T17:52:26.409861412Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-16T17:52:26.414047828Z INFO: Collecting host service logs for crio 2026-09-16T17:52:26.414484596Z INFO: Collecting host service logs for kubelet 2026-09-16T17:52:26.414895129Z INFO: Collecting host service logs for rpm-ostreed 2026-09-16T17:52:26.415404277Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-16T17:52:26.415743332Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-16T17:52:26.416076614Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-16T17:52:26.416433778Z INFO: Collecting host service logs for NetworkManager 2026-09-16T17:52:26.416769049Z INFO: Collecting host service logs for openvswitch 2026-09-16T17:52:26.417161221Z INFO: Collecting host service logs for ovs-configuration 2026-09-16T17:52:26.417499139Z INFO: Collecting host service logs for ovsdb-server 2026-09-16T17:52:26.417832558Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-16T17:52:26.418215935Z INFO: Waiting for worker host service log collection to complete ... 2026-09-16T17:52:26.502718136Z INFO: Waiting for node performance related collection to complete ... 2026-09-16T17:52:26.909365110Z INFO: Collecting Insights Archives from insights-operator-5844b856db-pnwkv 2026-09-16T17:52:26.909403959Z insights-runtime-extractor-9mrzx 2026-09-16T17:52:26.909421143Z insights-runtime-extractor-ln7qv 2026-09-16T17:52:26.909437492Z insights-runtime-extractor-nzjxv 2026-09-16T17:52:27.184744988Z No resources found 2026-09-16T17:52:27.254082289Z Wrote inspect data to must-gather. 2026-09-16T17:52:27.388943365Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-16T17:52:27.409728881Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-16T17:52:27.410102624Z INFO: on-disk MachineConfig config collection complete. 2026-09-16T17:52:27.417807985Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-16T17:52:27.417807985Z See 'oc rsync -h' for help and examples 2026-09-16T17:52:27.468508429Z INFO: "metallb-operator" not detected. Skipping. 2026-09-16T17:52:27.558622198Z No resources found in openshift-etcd namespace. 2026-09-16T17:52:27.641419581Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-16T17:52:27.772309935Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-16T17:52:27.841045102Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-16T17:52:27.877986143Z error: the server doesn't have a resource type "performanceprofile" 2026-09-16T17:52:27.963539845Z INFO: Worker host service log collection to complete. 2026-09-16T17:52:27.982086463Z ERROR: No running kube-apiserver pods found 2026-09-16T17:52:28.173762256Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-16T17:52:28.334761539Z INFO: 'previous-logs' folder not found on ip-10-0-130-167.ec2.internal, skipping... 2026-09-16T17:52:28.475051527Z Wrote inspect data to must-gather. 2026-09-16T17:52:28.528846501Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-16T17:52:28.593741247Z No resources found 2026-09-16T17:52:28.635656963Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-16T17:52:28.680853889Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-09-16T17:52:28.681478396Z Wrote inspect data to must-gather. 2026-09-16T17:52:28.794830792Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-16T17:52:29.071050931Z Wrote inspect data to must-gather. 2026-09-16T17:52:29.085923066Z Wrote inspect data to must-gather. 2026-09-16T17:52:29.092601687Z Wrote inspect data to must-gather. 2026-09-16T17:52:29.105811663Z Wrote inspect data to must-gather. 2026-09-16T17:52:29.136318395Z tar: Removing leading `/' from member names 2026-09-16T17:52:29.144488696Z INFO: HAProxy config collection complete. 2026-09-16T17:52:29.215792730Z Gathering data for ns/openshift-monitoring... 2026-09-16T17:52:29.307935417Z INFO: 'previous-logs' folder not found on ip-10-0-139-121.ec2.internal, skipping... 2026-09-16T17:52:29.347114765Z error: the server doesn't have a resource type "machineconfigs" 2026-09-16T17:52:29.396944585Z INFO: Getting rules from prometheus-k8s-0 2026-09-16T17:52:29.448680817Z Wrote inspect data to must-gather. 2026-09-16T17:52:29.547410145Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-16T17:52:29.622969570Z INFO: 'previous-logs' folder not found on ip-10-0-142-68.ec2.internal, skipping... 2026-09-16T17:52:29.623121806Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-16T17:52:29.623182764Z INFO: Machine Config Daemon termination log collection complete. 2026-09-16T17:52:29.769732979Z Wrote inspect data to must-gather. 2026-09-16T17:52:29.826733377Z Wrote inspect data to must-gather. 2026-09-16T17:52:29.890786016Z INFO: Getting status/config from prometheus-k8s-0 2026-09-16T17:52:29.905775274Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-16T17:52:30.015441061Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-16T17:52:30.060431642Z Wrote inspect data to must-gather. 2026-09-16T17:52:30.232919545Z Wrote inspect data to must-gather. 2026-09-16T17:52:30.323652809Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-16T17:52:30.357249979Z INFO: INTERCONNECT MODE 2026-09-16T17:52:30.357282549Z INFO: Gathering ovn-kubernetes DBs 2026-09-16T17:52:30.466311864Z Wrote inspect data to must-gather. 2026-09-16T17:52:30.538939042Z INFO: Gathering OVN_Northbound from ovnkube-node-5ssfw... 2026-09-16T17:52:30.539185219Z INFO: Gathering OVN_Northbound from ovnkube-node-n5mmr... 2026-09-16T17:52:30.539401248Z INFO: Gathering OVN_Northbound from ovnkube-node-qnxgf... 2026-09-16T17:52:30.539733593Z INFO: Gathering OVN_Southbound from ovnkube-node-5ssfw... 2026-09-16T17:52:30.539912797Z INFO: Gathering OVN_Southbound from ovnkube-node-n5mmr... 2026-09-16T17:52:30.540384798Z INFO: Gathering OVN_Southbound from ovnkube-node-qnxgf... 2026-09-16T17:52:30.636612347Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-16T17:52:30.798740574Z Wrote inspect data to must-gather. 2026-09-16T17:52:30.817733968Z tar: Removing leading `/' from member names 2026-09-16T17:52:30.830035369Z tar: Removing leading `/' from member names 2026-09-16T17:52:30.853264578Z tar: Removing leading `/' from member names 2026-09-16T17:52:30.872815370Z tar: Removing leading `/' from member names 2026-09-16T17:52:30.929450412Z tar: Removing leading `/' from member names 2026-09-16T17:52:30.937241114Z tar: Removing leading `/' from member names 2026-09-16T17:52:31.023985318Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-16T17:52:31.030692340Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-16T17:52:31.065175644Z Wrote inspect data to must-gather. 2026-09-16T17:52:31.310788284Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-16T17:52:31.336933231Z Wrote inspect data to must-gather. 2026-09-16T17:52:31.349679432Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-16T17:52:31.490602461Z INFO: Waiting for network log collection to complete ... 2026-09-16T17:52:31.490650295Z INFO: Waiting for ovnk database copies to complete ... 2026-09-16T17:52:31.491040674Z INFO: Copying ovnk databases complete. 2026-09-16T17:52:31.492852234Z 2.9M must-gather/network_logs/ovnk_database_store 2026-09-16T17:52:31.495138782Z ovnk_database_store/ 2026-09-16T17:52:31.495152469Z ovnk_database_store/ovnkube-node-qnxgf_nbdb 2026-09-16T17:52:31.506323796Z ovnk_database_store/ovnkube-node-5ssfw_nbdb 2026-09-16T17:52:31.511176098Z ovnk_database_store/ovnkube-node-qnxgf_sbdb 2026-09-16T17:52:31.530614141Z ovnk_database_store/ovnkube-node-5ssfw_sbdb 2026-09-16T17:52:31.548341721Z ovnk_database_store/ovnkube-node-n5mmr_sbdb 2026-09-16T17:52:31.570371357Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:52:31.571244698Z ovnk_database_store/ovnkube-node-n5mmr_nbdb 2026-09-16T17:52:31.582799532Z INFO: Network log collection complete. 2026-09-16T17:52:31.594218727Z Wrote inspect data to must-gather. 2026-09-16T17:52:31.753838730Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-16T17:52:31.755264959Z INFO: Getting status from alertmanager-main-0 2026-09-16T17:52:31.811765287Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:52:31.815267707Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-16T17:52:31.946110591Z Wrote inspect data to must-gather. 2026-09-16T17:52:31.957148827Z Gathering data for ns/openshift-network-console... 2026-09-16T17:52:32.033377207Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:52:32.237243428Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-16T17:52:32.239521426Z INFO: Fallback to identify the container image from release info 2026-09-16T17:52:32.249031837Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:52:32.252307229Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-16T17:52:32.466556233Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:52:32.504511617Z Gathering data for ns/openshift-console-operator... 2026-09-16T17:52:32.669775267Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:52:32.672738432Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-16T17:52:32.898160551Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:52:32.940673798Z Gathering data for ns/openshift-console... 2026-09-16T17:52:32.968672941Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a 2026-09-16T17:52:33.123762732Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:52:33.124955355Z daemonset.apps/perf-node-gather-daemonset created 2026-09-16T17:52:33.126756347Z Inspecting resource crd/inferencepools.inference.networking.k8s.io 2026-09-16T17:52:33.262783213Z Waiting for performance profile collector pods to become ready: 1 2026-09-16T17:52:33.312085899Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:52:33.533992900Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:52:33.537272136Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io 2026-09-16T17:52:33.576390247Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-16T17:52:33.689466936Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-09-16T17:52:33.725325310Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:52:33.927696936Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:52:33.930966988Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-16T17:52:34.115900196Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:52:34.158379679Z Gathering data for ns/openshift-dns-operator... 2026-09-16T17:52:34.328433104Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:52:34.398749917Z Waiting for performance profile collector pods to become ready: 2 2026-09-16T17:52:34.524438225Z Gathering data for ns/openshift-dns... 2026-09-16T17:52:35.021333365Z Done executing Istio gather script 2026-09-16T17:52:35.270392637Z Gathering data for ns/openshift-image-registry... 2026-09-16T17:52:35.531751132Z Waiting for performance profile collector pods to become ready: 3 2026-09-16T17:52:35.797380934Z Gathering data for ns/openshift-ingress-operator... 2026-09-16T17:52:36.221659868Z Gathering data for ns/openshift-ingress... 2026-09-16T17:52:36.580023069Z Gathering data for ns/openshift-ingress-canary... 2026-09-16T17:52:36.652916597Z Waiting for performance profile collector pods to become ready: 4 2026-09-16T17:52:36.996981552Z Gathering data for ns/openshift-insights... 2026-09-16T17:52:37.772548373Z Waiting for performance profile collector pods to become ready: 5 2026-09-16T17:52:38.696912923Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-09-16T17:52:38.908010558Z Waiting for performance profile collector pods to become ready: 6 2026-09-16T17:52:39.514130739Z Gathering data for ns/kserve... 2026-09-16T17:52:40.039349522Z Waiting for performance profile collector pods to become ready: 7 2026-09-16T17:52:40.937419912Z Gathering data for ns/openshift-config... 2026-09-16T17:52:41.161093919Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-16T17:52:41.336779355Z Gathering data for ns/openshift-config-managed... 2026-09-16T17:52:41.439568586Z Collecting performance related data for node ip-10-0-130-167.ec2.internal 2026-09-16T17:52:41.447085136Z Collecting performance related data for node ip-10-0-139-121.ec2.internal 2026-09-16T17:52:41.454937718Z Collecting performance related data for node ip-10-0-142-68.ec2.internal 2026-09-16T17:52:41.828694969Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-16T17:52:42.207189076Z Gathering data for ns/openshift-kube-apiserver... 2026-09-16T17:52:42.584534742Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-16T17:52:42.961269515Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-16T17:52:43.333411326Z Gathering data for ns/openshift-kube-scheduler... 2026-09-16T17:52:43.633379649Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-16T17:52:43.703838706Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-09-16T17:52:44.056033367Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-16T17:52:44.512454050Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-16T17:52:44.910787184Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-16T17:52:45.506754930Z Gathering data for ns/openshift-multus... 2026-09-16T17:52:47.319150178Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-16T17:52:48.711122019Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-09-16T17:52:48.731728292Z Gathering data for ns/openshift-host-network... 2026-09-16T17:52:49.050820843Z Gathering data for ns/openshift-network-diagnostics... 2026-09-16T17:52:49.606588883Z Collecting kubelet logs for node ip-10-0-130-167.ec2.internal 2026-09-16T17:52:49.609699745Z Collecting kubelet logs for node ip-10-0-139-121.ec2.internal 2026-09-16T17:52:49.610389302Z Gathering data for ns/openshift-network-node-identity... 2026-09-16T17:52:49.611897356Z Collecting kubelet logs for node ip-10-0-142-68.ec2.internal 2026-09-16T17:52:50.001925941Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-2qsbm namespace 2026-09-16T17:52:50.006831220Z INFO: Node performance data collection complete. 2026-09-16T17:52:50.150839844Z Gathering data for ns/openshift-network-operator... 2026-09-16T17:52:50.612113622Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-16T17:52:50.939439141Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-16T17:52:51.480710988Z Gathering data for ns/openshift-apiserver-operator... 2026-09-16T17:52:51.818714861Z Gathering data for ns/openshift-apiserver... 2026-09-16T17:52:52.300624773Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-16T17:52:52.645814176Z Gathering data for ns/openshift-controller-manager... 2026-09-16T17:52:52.964687480Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-16T17:52:53.716635523Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-09-16T17:52:53.738003308Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-16T17:52:54.078318429Z Gathering data for ns/openshift-service-ca-operator... 2026-09-16T17:52:54.449428525Z Gathering data for ns/openshift-service-ca... 2026-09-16T17:52:54.882419688Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-16T17:52:55.952547827Z Wrote inspect data to must-gather. 2026-09-16T17:52:55.952593478Z error: inspection completed with the errors occurred while gathering data: 2026-09-16T17:52:55.952593478Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-16T17:52:56.088595650Z error: the server doesn't have a resource type "clusters" 2026-09-16T17:52:56.226912961Z Caches written to disk