2026-06-08T13:57:35.447629665Z [disk usage checker] Started 2026-06-08T13:57:35.451569237Z [disk usage checker] Volume usage percentage: current = 32 ; allowed = 70 2026-06-08T13:57:35.776140699Z Gathering data for ns/openshift-cluster-version... 2026-06-08T13:57:35.893818103Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-06-08T13:57:35.968470983Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-06-08T13:57:36.054581894Z Gathering data for ns/default... 2026-06-08T13:57:36.357352986Z Gathering data for ns/openshift... 2026-06-08T13:57:36.737387691Z Gathering data for ns/kube-system... 2026-06-08T13:57:37.391014801Z Gathering data for ns/openshift-etcd... 2026-06-08T13:57:37.665621804Z Wrote inspect data to must-gather. 2026-06-08T13:57:37.665654585Z error: inspection completed with the errors occurred while gathering data: 2026-06-08T13:57:37.665654585Z namespaces "assisted-installer" not found 2026-06-08T13:57:38.103964575Z Waiting on subprocesses to finish execution. 2026-06-08T13:57:38.115116196Z Executing Istio gather script 2026-06-08T13:57:38.116563418Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-06-08T13:57:38.116954581Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-06-08T13:57:38.121602052Z INFO: Gathering machine config daemon's old logs from all nodes 2026-06-08T13:57:38.129891848Z INFO: Collecting host service logs for crio 2026-06-08T13:57:38.130298580Z INFO: Collecting host service logs for kubelet 2026-06-08T13:57:38.130648812Z INFO: Collecting host service logs for rpm-ostreed 2026-06-08T13:57:38.130987794Z INFO: Collecting host service logs for ostree-finalize-staged 2026-06-08T13:57:38.131345426Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-06-08T13:57:38.131669563Z INFO: Collecting host service logs for machine-config-daemon-host 2026-06-08T13:57:38.131987326Z INFO: Collecting host service logs for NetworkManager 2026-06-08T13:57:38.132459962Z INFO: Collecting host service logs for openvswitch 2026-06-08T13:57:38.132829798Z INFO: Collecting host service logs for ovs-configuration 2026-06-08T13:57:38.133242679Z INFO: Collecting host service logs for ovsdb-server 2026-06-08T13:57:38.133648169Z INFO: Collecting host service logs for ovs-vswitchd 2026-06-08T13:57:38.134052999Z INFO: Waiting for worker host service log collection to complete ... 2026-06-08T13:57:38.170596712Z INFO: Gathering HAProxy config files 2026-06-08T13:57:38.399686154Z INFO: Waiting for node performance related collection to complete ... 2026-06-08T13:57:38.522180761Z No resources found in openshift-etcd namespace. 2026-06-08T13:57:38.702002354Z No resources found 2026-06-08T13:57:38.757738509Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-06-08T13:57:38.805547019Z INFO: Collecting Insights Archives from insights-operator-896549976-nhxw2 2026-06-08T13:57:38.805605061Z insights-runtime-extractor-596fp 2026-06-08T13:57:38.805626142Z insights-runtime-extractor-fwvzz 2026-06-08T13:57:38.805646571Z insights-runtime-extractor-wsm96 2026-06-08T13:57:38.857815048Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-06-08T13:57:38.857907027Z INFO: on-disk MachineConfig config collection complete. 2026-06-08T13:57:39.467159804Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-06-08T13:57:39.467159804Z See 'oc rsync -h' for help and examples 2026-06-08T13:57:39.531186972Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-06-08T13:57:39.590702886Z INFO: Waiting for HAProxy config collection to complete ... 2026-06-08T13:57:39.662109950Z Wrote inspect data to must-gather. 2026-06-08T13:57:39.826447466Z Gathering data for ns/openshift-monitoring... 2026-06-08T13:57:39.985007773Z No resources found 2026-06-08T13:57:40.084085298Z Wrote inspect data to must-gather. 2026-06-08T13:57:40.089822791Z INFO: Worker host service log collection to complete. 2026-06-08T13:57:40.102868928Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-06-08T13:57:40.116463928Z INFO: "metallb-operator" not detected. Skipping. 2026-06-08T13:57:40.163747212Z error: the server doesn't have a resource type "performanceprofile" 2026-06-08T13:57:40.181003206Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-06-08T13:57:40.203160856Z ERROR: No running kube-apiserver pods found 2026-06-08T13:57:40.221633627Z INFO: Found 1 replicas - prometheus-k8s-0 2026-06-08T13:57:40.236920975Z INFO: "sriov-network-operator" not detected. Skipping. 2026-06-08T13:57:40.242756239Z Wrote inspect data to must-gather. 2026-06-08T13:57:40.366557700Z tar: Removing leading `/' from member names 2026-06-08T13:57:40.375967127Z INFO: HAProxy config collection complete. 2026-06-08T13:57:40.438825522Z Wrote inspect data to must-gather. 2026-06-08T13:57:40.464779202Z [disk usage checker] Volume usage percentage: current = 32 ; allowed = 70 2026-06-08T13:57:40.478483606Z INFO: 'previous-logs' folder not found on ip-10-0-129-137.ec2.internal, skipping... 2026-06-08T13:57:40.555001526Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-06-08T13:57:40.566501290Z Wrote inspect data to must-gather. 2026-06-08T13:57:40.586737682Z Wrote inspect data to must-gather. 2026-06-08T13:57:40.591896294Z Wrote inspect data to must-gather. 2026-06-08T13:57:40.782829025Z Wrote inspect data to must-gather. 2026-06-08T13:57:40.797287707Z error: the server doesn't have a resource type "machineconfigs" 2026-06-08T13:57:40.835826997Z INFO: 'previous-logs' folder not found on ip-10-0-136-30.ec2.internal, skipping... 2026-06-08T13:57:41.006913206Z error: the server doesn't have a resource type "machineconfigpools" 2026-06-08T13:57:41.021401078Z INFO: Getting rules from prometheus-k8s-0 2026-06-08T13:57:41.074298825Z Wrote inspect data to must-gather. 2026-06-08T13:57:41.157776643Z INFO: 'previous-logs' folder not found on ip-10-0-138-32.ec2.internal, skipping... 2026-06-08T13:57:41.157810001Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-06-08T13:57:41.157845597Z INFO: Machine Config Daemon termination log collection complete. 2026-06-08T13:57:41.273379603Z Wrote inspect data to must-gather. 2026-06-08T13:57:41.363177662Z Wrote inspect data to must-gather. 2026-06-08T13:57:41.483067460Z INFO: Getting status/config from prometheus-k8s-0 2026-06-08T13:57:41.486227866Z error: the server doesn't have a resource type "kubeletconfigs" 2026-06-08T13:57:41.496679885Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-06-08T13:57:41.593497525Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-06-08T13:57:41.705682178Z INFO: INTERCONNECT MODE 2026-06-08T13:57:41.705709982Z INFO: Gathering ovn-kubernetes DBs 2026-06-08T13:57:41.744489878Z Wrote inspect data to must-gather. 2026-06-08T13:57:41.890519381Z INFO: Gathering OVN_Northbound from ovnkube-node-5fbq9... 2026-06-08T13:57:41.890734713Z INFO: Gathering OVN_Northbound from ovnkube-node-x94mm... 2026-06-08T13:57:41.890943806Z INFO: Gathering OVN_Northbound from ovnkube-node-zxqxr... 2026-06-08T13:57:41.891250844Z INFO: Gathering OVN_Southbound from ovnkube-node-5fbq9... 2026-06-08T13:57:41.891532355Z INFO: Gathering OVN_Southbound from ovnkube-node-x94mm... 2026-06-08T13:57:41.891781775Z INFO: Gathering OVN_Southbound from ovnkube-node-zxqxr... 2026-06-08T13:57:41.916341781Z INFO: Getting status/flags from prometheus-k8s-0 2026-06-08T13:57:42.036679094Z Wrote inspect data to must-gather. 2026-06-08T13:57:42.069013064Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-06-08T13:57:42.193415093Z tar: Removing leading `/' from member names 2026-06-08T13:57:42.256430801Z tar: Removing leading `/' from member names 2026-06-08T13:57:42.274823432Z tar: Removing leading `/' from member names 2026-06-08T13:57:42.286436157Z tar: Removing leading `/' from member names 2026-06-08T13:57:42.290300554Z tar: Removing leading `/' from member names 2026-06-08T13:57:42.313257007Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-06-08T13:57:42.318806722Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-06-08T13:57:42.326134730Z tar: Removing leading `/' from member names 2026-06-08T13:57:42.377983522Z Wrote inspect data to /must-gather/istio. 2026-06-08T13:57:42.495601289Z Wrote inspect data to must-gather. 2026-06-08T13:57:42.557570164Z Gathering data for ns/openshift-network-console... 2026-06-08T13:57:42.572321239Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-06-08T13:57:42.643611485Z Wrote inspect data to /must-gather/istio. 2026-06-08T13:57:42.649026813Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-06-08T13:57:42.774290336Z INFO: Waiting for network log collection to complete ... 2026-06-08T13:57:42.774322997Z INFO: Waiting for ovnk database copies to complete ... 2026-06-08T13:57:42.774804785Z INFO: Copying ovnk databases complete. 2026-06-08T13:57:42.776351363Z 2.8M must-gather/network_logs/ovnk_database_store 2026-06-08T13:57:42.778506323Z ovnk_database_store/ 2026-06-08T13:57:42.778559518Z ovnk_database_store/ovnkube-node-zxqxr_sbdb 2026-06-08T13:57:42.800707592Z ovnk_database_store/ovnkube-node-x94mm_nbdb 2026-06-08T13:57:42.809579939Z ovnk_database_store/ovnkube-node-zxqxr_nbdb 2026-06-08T13:57:42.818188095Z ovnk_database_store/ovnkube-node-5fbq9_sbdb 2026-06-08T13:57:42.837829939Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-06-08T13:57:42.839037160Z ovnk_database_store/ovnkube-node-5fbq9_nbdb 2026-06-08T13:57:42.839054444Z Wrote inspect data to must-gather. 2026-06-08T13:57:42.846395632Z ovnk_database_store/ovnkube-node-x94mm_sbdb 2026-06-08T13:57:42.871365084Z INFO: Network log collection complete. 2026-06-08T13:57:42.891841878Z Wrote inspect data to /must-gather/istio. 2026-06-08T13:57:43.068740784Z Gathering data for ns/openshift-console-operator... 2026-06-08T13:57:43.109139578Z Wrote inspect data to /must-gather/istio. 2026-06-08T13:57:43.112760447Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-06-08T13:57:43.168219556Z Wrote inspect data to must-gather. 2026-06-08T13:57:43.236062065Z INFO: Getting status from alertmanager-main-0 2026-06-08T13:57:43.322589560Z Wrote inspect data to /must-gather/istio. 2026-06-08T13:57:43.464661265Z Wrote inspect data to must-gather. 2026-06-08T13:57:43.524489398Z Gathering data for ns/openshift-console... 2026-06-08T13:57:43.539500383Z Wrote inspect data to /must-gather/istio. 2026-06-08T13:57:43.542119483Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-06-08T13:57:43.761850609Z Wrote inspect data to must-gather. 2026-06-08T13:57:43.765203049Z Wrote inspect data to /must-gather/istio. 2026-06-08T13:57:43.970555714Z Wrote inspect data to /must-gather/istio. 2026-06-08T13:57:43.974179458Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-06-08T13:57:44.032800394Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-06-08T13:57:44.037835587Z INFO: Fallback to identify the container image from release info 2026-06-08T13:57:44.080165694Z Gathering data for ns/openshift-cluster-storage-operator... 2026-06-08T13:57:44.156144878Z Wrote inspect data to /must-gather/istio. 2026-06-08T13:57:44.337470175Z Wrote inspect data to /must-gather/istio. 2026-06-08T13:57:44.571576822Z Gathering data for ns/openshift-dns-operator... 2026-06-08T13:57:44.665805703Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e 2026-06-08T13:57:44.787182488Z daemonset.apps/perf-node-gather-daemonset created 2026-06-08T13:57:44.887456165Z Gathering data for ns/openshift-dns... 2026-06-08T13:57:44.928768006Z Waiting for performance profile collector pods to become ready: 1 2026-06-08T13:57:45.025277455Z Done executing Istio gather script 2026-06-08T13:57:45.443970474Z Gathering data for ns/openshift-image-registry... 2026-06-08T13:57:45.480047317Z [disk usage checker] Volume usage percentage: current = 32 ; allowed = 70 2026-06-08T13:57:45.859475203Z Gathering data for ns/openshift-ingress-operator... 2026-06-08T13:57:46.034837401Z Waiting for performance profile collector pods to become ready: 2 2026-06-08T13:57:46.156814529Z Gathering data for ns/openshift-ingress... 2026-06-08T13:57:46.450921979Z Gathering data for ns/openshift-ingress-canary... 2026-06-08T13:57:46.793236336Z Gathering data for ns/openshift-insights... 2026-06-08T13:57:47.142491228Z Waiting for performance profile collector pods to become ready: 3 2026-06-08T13:57:48.247491923Z Waiting for performance profile collector pods to become ready: 4 2026-06-08T13:57:48.570884637Z Gathering data for ns/openshift-jobset-operator... 2026-06-08T13:57:49.300606884Z Gathering data for ns/openshift-config... 2026-06-08T13:57:49.368900246Z Waiting for performance profile collector pods to become ready: 5 2026-06-08T13:57:49.568072364Z Gathering data for ns/openshift-config-managed... 2026-06-08T13:57:49.986878483Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-06-08T13:57:50.213884206Z Gathering data for ns/openshift-kube-apiserver... 2026-06-08T13:57:50.485583193Z [disk usage checker] Volume usage percentage: current = 32 ; allowed = 70 2026-06-08T13:57:50.486668449Z Gathering data for ns/openshift-kube-controller-manager... 2026-06-08T13:57:50.487772013Z Waiting for performance profile collector pods to become ready: 6 2026-06-08T13:57:50.748477485Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-06-08T13:57:50.993191621Z Gathering data for ns/openshift-kube-scheduler... 2026-06-08T13:57:51.223071673Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-06-08T13:57:51.472565914Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-06-08T13:57:51.610780825Z Waiting for performance profile collector pods to become ready: 7 2026-06-08T13:57:51.783740569Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-06-08T13:57:52.049147388Z Gathering data for ns/openshift-user-workload-monitoring... 2026-06-08T13:57:52.563403324Z Gathering data for ns/openshift-multus... 2026-06-08T13:57:52.711363231Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-06-08T13:57:52.935371960Z Collecting performance related data for node ip-10-0-136-30.ec2.internal 2026-06-08T13:57:52.942411691Z Collecting performance related data for node ip-10-0-138-32.ec2.internal 2026-06-08T13:57:52.950481679Z Collecting performance related data for node ip-10-0-129-137.ec2.internal 2026-06-08T13:57:54.230501389Z Gathering data for ns/openshift-ovn-kubernetes... 2026-06-08T13:57:55.465470327Z Gathering data for ns/openshift-host-network... 2026-06-08T13:57:55.494968449Z [disk usage checker] Volume usage percentage: current = 32 ; allowed = 70 2026-06-08T13:57:55.785536852Z Gathering data for ns/openshift-network-diagnostics... 2026-06-08T13:57:56.264786387Z Gathering data for ns/openshift-network-node-identity... 2026-06-08T13:57:56.643422570Z Gathering data for ns/openshift-network-operator... 2026-06-08T13:57:56.993290337Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-06-08T13:57:57.249330578Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-06-08T13:57:57.701797283Z Gathering data for ns/openshift-apiserver-operator... 2026-06-08T13:57:58.009304728Z Gathering data for ns/openshift-apiserver... 2026-06-08T13:57:58.372055260Z Gathering data for ns/openshift-controller-manager-operator... 2026-06-08T13:57:58.608634966Z Gathering data for ns/openshift-controller-manager... 2026-06-08T13:57:58.877382004Z Gathering data for ns/openshift-cluster-samples-operator... 2026-06-08T13:57:59.525346994Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-06-08T13:57:59.796837853Z Gathering data for ns/openshift-service-ca-operator... 2026-06-08T13:58:00.085235757Z Gathering data for ns/openshift-service-ca... 2026-06-08T13:58:00.464843473Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-06-08T13:58:00.499906225Z [disk usage checker] Volume usage percentage: current = 32 ; allowed = 70 2026-06-08T13:58:01.395917321Z Wrote inspect data to must-gather. 2026-06-08T13:58:01.395961898Z error: inspection completed with the errors occurred while gathering data: 2026-06-08T13:58:01.395961898Z skipping gathering secrets/support due to error: secrets "support" not found 2026-06-08T13:58:01.442559727Z Collecting kubelet logs for node ip-10-0-136-30.ec2.internal 2026-06-08T13:58:01.444303466Z Collecting kubelet logs for node ip-10-0-138-32.ec2.internal 2026-06-08T13:58:01.446034889Z Collecting kubelet logs for node ip-10-0-129-137.ec2.internal 2026-06-08T13:58:01.738941637Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-67dm8 namespace 2026-06-08T13:58:01.743875035Z INFO: Node performance data collection complete. 2026-06-08T13:58:01.871709141Z error: the server doesn't have a resource type "clusters" 2026-06-08T13:58:01.999733794Z Caches written to disk