2026-09-09T20:24:37.258104726Z [disk usage checker] Started 2026-09-09T20:24:37.261350795Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-09-09T20:24:37.476228734Z Gathering data for ns/openshift-cluster-version... 2026-09-09T20:24:37.621378457Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-09T20:24:37.713479457Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-09T20:24:37.800557359Z Gathering data for ns/default... 2026-09-09T20:24:38.137667849Z Gathering data for ns/openshift... 2026-09-09T20:24:38.558254496Z Gathering data for ns/kube-system... 2026-09-09T20:24:39.266692161Z Gathering data for ns/openshift-etcd... 2026-09-09T20:24:39.580497050Z Wrote inspect data to must-gather. 2026-09-09T20:24:39.580532252Z error: inspection completed with the errors occurred while gathering data: 2026-09-09T20:24:39.580532252Z namespaces "assisted-installer" not found 2026-09-09T20:24:40.149919049Z Waiting on subprocesses to finish execution. 2026-09-09T20:24:40.156137584Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-09T20:24:40.156375544Z INFO: Gathering HAProxy config files 2026-09-09T20:24:40.158473468Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-09T20:24:40.158745223Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-09T20:24:40.177967811Z Executing Istio gather script 2026-09-09T20:24:40.183338729Z INFO: Collecting host service logs for crio 2026-09-09T20:24:40.183701960Z INFO: Collecting host service logs for kubelet 2026-09-09T20:24:40.184004032Z INFO: Collecting host service logs for rpm-ostreed 2026-09-09T20:24:40.184325621Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-09T20:24:40.184638508Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-09T20:24:40.184918484Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-09T20:24:40.185235921Z INFO: Collecting host service logs for NetworkManager 2026-09-09T20:24:40.185522685Z INFO: Collecting host service logs for openvswitch 2026-09-09T20:24:40.185806108Z INFO: Collecting host service logs for ovs-configuration 2026-09-09T20:24:40.186104867Z INFO: Collecting host service logs for ovsdb-server 2026-09-09T20:24:40.186447791Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-09T20:24:40.186794766Z INFO: Waiting for worker host service log collection to complete ... 2026-09-09T20:24:40.198980139Z INFO: Waiting for node performance related collection to complete ... 2026-09-09T20:24:40.607319819Z INFO: "metallb-operator" not detected. Skipping. 2026-09-09T20:24:40.719391081Z No resources found in openshift-etcd namespace. 2026-09-09T20:24:41.128049166Z INFO: Collecting Insights Archives from insights-operator-76bcd6ccf5-mglmh 2026-09-09T20:24:41.128049166Z insights-runtime-extractor-k6ctg 2026-09-09T20:24:41.128049166Z insights-runtime-extractor-ms748 2026-09-09T20:24:41.128049166Z insights-runtime-extractor-qnrrp 2026-09-09T20:24:41.424886578Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-09T20:24:41.439430869Z Wrote inspect data to must-gather. 2026-09-09T20:24:41.449983509Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-09T20:24:41.481425991Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-09T20:24:41.487827276Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-09T20:24:41.487915306Z INFO: on-disk MachineConfig config collection complete. 2026-09-09T20:24:41.491476940Z error: the server doesn't have a resource type "performanceprofile" 2026-09-09T20:24:41.552918947Z INFO: Worker host service log collection to complete. 2026-09-09T20:24:41.560923588Z No resources found 2026-09-09T20:24:41.615414796Z Wrote inspect data to must-gather. 2026-09-09T20:24:41.648709998Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-09T20:24:41.711240488Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-09T20:24:41.711240488Z See 'oc rsync -h' for help and examples 2026-09-09T20:24:42.020595333Z ERROR: No running kube-apiserver pods found 2026-09-09T20:24:42.044359925Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-09T20:24:42.193575766Z Wrote inspect data to must-gather. 2026-09-09T20:24:42.230737884Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-09T20:24:42.330411584Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-09-09T20:24:42.400519312Z Wrote inspect data to must-gather. 2026-09-09T20:24:42.415624867Z Wrote inspect data to must-gather. 2026-09-09T20:24:42.534624481Z No resources found 2026-09-09T20:24:42.553428214Z Wrote inspect data to must-gather. 2026-09-09T20:24:42.624620184Z INFO: 'previous-logs' folder not found on ip-10-0-135-134.ec2.internal, skipping... 2026-09-09T20:24:42.722900751Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-09T20:24:42.747123023Z Wrote inspect data to must-gather. 2026-09-09T20:24:42.768778024Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-09T20:24:42.837371798Z tar: Removing leading `/' from member names 2026-09-09T20:24:42.844108434Z INFO: HAProxy config collection complete. 2026-09-09T20:24:42.873599990Z error: the server doesn't have a resource type "machineconfigs" 2026-09-09T20:24:43.049441832Z INFO: 'previous-logs' folder not found on ip-10-0-136-183.ec2.internal, skipping... 2026-09-09T20:24:43.102750375Z Wrote inspect data to must-gather. 2026-09-09T20:24:43.103025258Z Gathering data for ns/openshift-monitoring... 2026-09-09T20:24:43.118143370Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-09T20:24:43.156225443Z INFO: Getting rules from prometheus-k8s-0 2026-09-09T20:24:43.365812632Z INFO: 'previous-logs' folder not found on ip-10-0-143-121.ec2.internal, skipping... 2026-09-09T20:24:43.365853684Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-09T20:24:43.365882755Z INFO: Machine Config Daemon termination log collection complete. 2026-09-09T20:24:43.410307649Z Wrote inspect data to must-gather. 2026-09-09T20:24:43.431800319Z Wrote inspect data to must-gather. 2026-09-09T20:24:43.605070006Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-09T20:24:43.620614444Z INFO: Getting status/config from prometheus-k8s-0 2026-09-09T20:24:43.733106153Z Wrote inspect data to must-gather. 2026-09-09T20:24:43.882199897Z Wrote inspect data to must-gather. 2026-09-09T20:24:43.920710942Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-09T20:24:44.082545717Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-09T20:24:44.093926573Z INFO: INTERCONNECT MODE 2026-09-09T20:24:44.093969920Z INFO: Gathering ovn-kubernetes DBs 2026-09-09T20:24:44.144085917Z Wrote inspect data to must-gather. 2026-09-09T20:24:44.270433774Z INFO: Gathering OVN_Northbound from ovnkube-node-4pfft... 2026-09-09T20:24:44.270696514Z INFO: Gathering OVN_Northbound from ovnkube-node-jcr9m... 2026-09-09T20:24:44.270946763Z INFO: Gathering OVN_Northbound from ovnkube-node-smhwd... 2026-09-09T20:24:44.271237128Z INFO: Gathering OVN_Southbound from ovnkube-node-4pfft... 2026-09-09T20:24:44.271490768Z INFO: Gathering OVN_Southbound from ovnkube-node-jcr9m... 2026-09-09T20:24:44.271793106Z INFO: Gathering OVN_Southbound from ovnkube-node-smhwd... 2026-09-09T20:24:44.372930325Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-09T20:24:44.487989593Z Wrote inspect data to must-gather. 2026-09-09T20:24:44.588337556Z tar: Removing leading `/' from member names 2026-09-09T20:24:44.592650637Z tar: Removing leading `/' from member names 2026-09-09T20:24:44.623615406Z tar: Removing leading `/' from member names 2026-09-09T20:24:44.650327249Z tar: Removing leading `/' from member names 2026-09-09T20:24:44.665934956Z tar: Removing leading `/' from member names 2026-09-09T20:24:44.678990162Z tar: Removing leading `/' from member names 2026-09-09T20:24:44.799906118Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-09T20:24:44.804122361Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-09T20:24:44.811896448Z Wrote inspect data to must-gather. 2026-09-09T20:24:44.854573356Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-09T20:24:45.097547374Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-09T20:24:45.101273350Z Wrote inspect data to must-gather. 2026-09-09T20:24:45.117489017Z Wrote inspect data to /must-gather/istio. 2026-09-09T20:24:45.264438033Z INFO: Waiting for network log collection to complete ... 2026-09-09T20:24:45.264477735Z INFO: Waiting for ovnk database copies to complete ... 2026-09-09T20:24:45.265057148Z INFO: Copying ovnk databases complete. 2026-09-09T20:24:45.266909236Z 3.7M must-gather/network_logs/ovnk_database_store 2026-09-09T20:24:45.269673902Z ovnk_database_store/ 2026-09-09T20:24:45.269722822Z ovnk_database_store/ovnkube-node-jcr9m_sbdb 2026-09-09T20:24:45.304250805Z ovnk_database_store/ovnkube-node-4pfft_nbdb 2026-09-09T20:24:45.314592161Z ovnk_database_store/ovnkube-node-jcr9m_nbdb 2026-09-09T20:24:45.325467693Z ovnk_database_store/ovnkube-node-4pfft_sbdb 2026-09-09T20:24:45.353506614Z ovnk_database_store/ovnkube-node-smhwd_sbdb 2026-09-09T20:24:45.359629648Z Wrote inspect data to must-gather. 2026-09-09T20:24:45.378034420Z ovnk_database_store/ovnkube-node-smhwd_nbdb 2026-09-09T20:24:45.391617186Z INFO: Network log collection complete. 2026-09-09T20:24:45.393277571Z Wrote inspect data to /must-gather/istio. 2026-09-09T20:24:45.395564241Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-09T20:24:45.519395807Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-09T20:24:45.540578941Z INFO: Getting status from alertmanager-main-0 2026-09-09T20:24:45.648180405Z Wrote inspect data to /must-gather/istio. 2026-09-09T20:24:45.699063744Z Wrote inspect data to must-gather. 2026-09-09T20:24:45.737038832Z Gathering data for ns/openshift-network-console... 2026-09-09T20:24:45.887737510Z Wrote inspect data to /must-gather/istio. 2026-09-09T20:24:45.890183245Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-09T20:24:45.990710061Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-09T20:24:45.993717318Z INFO: Fallback to identify the container image from release info 2026-09-09T20:24:46.098325010Z Wrote inspect data to /must-gather/istio. 2026-09-09T20:24:46.234816131Z Gathering data for ns/openshift-console-operator... 2026-09-09T20:24:46.324470382Z Wrote inspect data to /must-gather/istio. 2026-09-09T20:24:46.327516974Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-09T20:24:46.552460319Z Wrote inspect data to /must-gather/istio. 2026-09-09T20:24:46.627064273Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f39c1daa89e19c5e987a3b759e646ffab6c1b2ab94a97ded81ddf13e73d0399 2026-09-09T20:24:46.662031875Z Gathering data for ns/openshift-console... 2026-09-09T20:24:46.787774339Z daemonset.apps/perf-node-gather-daemonset created 2026-09-09T20:24:46.797960033Z Wrote inspect data to /must-gather/istio. 2026-09-09T20:24:46.802063501Z Inspecting resource crd/inferencepools.inference.networking.k8s.io 2026-09-09T20:24:46.935429462Z Waiting for performance profile collector pods to become ready: 1 2026-09-09T20:24:46.990860927Z Wrote inspect data to /must-gather/istio. 2026-09-09T20:24:47.219268590Z Wrote inspect data to /must-gather/istio. 2026-09-09T20:24:47.222345270Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io 2026-09-09T20:24:47.279013294Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-09T20:24:47.402455809Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-09-09T20:24:47.420311368Z Wrote inspect data to /must-gather/istio. 2026-09-09T20:24:47.640969012Z Wrote inspect data to /must-gather/istio. 2026-09-09T20:24:47.643674811Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-09T20:24:47.832864146Z Wrote inspect data to /must-gather/istio. 2026-09-09T20:24:47.910893293Z Gathering data for ns/openshift-dns-operator... 2026-09-09T20:24:48.054773013Z Wrote inspect data to /must-gather/istio. 2026-09-09T20:24:48.087277430Z Waiting for performance profile collector pods to become ready: 2 2026-09-09T20:24:48.266001780Z Gathering data for ns/openshift-dns... 2026-09-09T20:24:48.754011630Z Done executing Istio gather script 2026-09-09T20:24:48.936139476Z Gathering data for ns/openshift-image-registry... 2026-09-09T20:24:49.216789954Z Waiting for performance profile collector pods to become ready: 3 2026-09-09T20:24:49.445987030Z Gathering data for ns/openshift-ingress-operator... 2026-09-09T20:24:49.786190914Z Gathering data for ns/openshift-ingress... 2026-09-09T20:24:50.151961067Z Gathering data for ns/openshift-ingress-canary... 2026-09-09T20:24:50.359628479Z Waiting for performance profile collector pods to become ready: 4 2026-09-09T20:24:50.576696150Z Gathering data for ns/openshift-insights... 2026-09-09T20:24:51.480340609Z Waiting for performance profile collector pods to become ready: 5 2026-09-09T20:24:52.407295452Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-09-09T20:24:52.603408952Z Waiting for performance profile collector pods to become ready: 6 2026-09-09T20:24:53.005420827Z Gathering data for ns/kserve... 2026-09-09T20:24:53.750292999Z Waiting for performance profile collector pods to become ready: 7 2026-09-09T20:24:54.748802062Z Gathering data for ns/openshift-config... 2026-09-09T20:24:54.880122808Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-09T20:24:55.085595354Z Gathering data for ns/openshift-config-managed... 2026-09-09T20:24:55.161456708Z Collecting performance related data for node ip-10-0-136-183.ec2.internal 2026-09-09T20:24:55.171389044Z Collecting performance related data for node ip-10-0-135-134.ec2.internal 2026-09-09T20:24:55.180591563Z Collecting performance related data for node ip-10-0-143-121.ec2.internal 2026-09-09T20:24:55.607367237Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-09T20:24:55.964465182Z Gathering data for ns/openshift-kube-apiserver... 2026-09-09T20:24:56.335255715Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-09T20:24:56.712388292Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-09T20:24:57.060453520Z Gathering data for ns/openshift-kube-scheduler... 2026-09-09T20:24:57.341936199Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-09T20:24:57.413838106Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-09-09T20:24:57.717253964Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-09T20:24:58.133561484Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-09T20:24:58.493435559Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-09T20:24:59.128846733Z Gathering data for ns/openshift-multus... 2026-09-09T20:25:00.660532761Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-09T20:25:01.951523390Z Gathering data for ns/openshift-host-network... 2026-09-09T20:25:02.283076881Z Gathering data for ns/openshift-network-diagnostics... 2026-09-09T20:25:02.419327726Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-09-09T20:25:02.893778285Z Gathering data for ns/openshift-network-node-identity... 2026-09-09T20:25:03.356271160Z Collecting kubelet logs for node ip-10-0-136-183.ec2.internal 2026-09-09T20:25:03.358194773Z Collecting kubelet logs for node ip-10-0-135-134.ec2.internal 2026-09-09T20:25:03.360116324Z Collecting kubelet logs for node ip-10-0-143-121.ec2.internal 2026-09-09T20:25:03.376761829Z Gathering data for ns/openshift-network-operator... 2026-09-09T20:25:03.749357556Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-cwwj7 namespace 2026-09-09T20:25:03.755957117Z INFO: Node performance data collection complete. 2026-09-09T20:25:03.812141309Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-09T20:25:04.138424450Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-09T20:25:04.643381784Z Gathering data for ns/openshift-apiserver-operator... 2026-09-09T20:25:04.957428145Z Gathering data for ns/openshift-apiserver... 2026-09-09T20:25:05.365687233Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-09T20:25:05.640565776Z Gathering data for ns/openshift-controller-manager... 2026-09-09T20:25:05.919211854Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-09T20:25:06.586944089Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-09T20:25:06.901970559Z Gathering data for ns/openshift-service-ca-operator... 2026-09-09T20:25:07.249980402Z Gathering data for ns/openshift-service-ca... 2026-09-09T20:25:07.426038098Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-09-09T20:25:07.636851021Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-09T20:25:08.574292100Z Wrote inspect data to must-gather. 2026-09-09T20:25:08.574342774Z error: inspection completed with the errors occurred while gathering data: 2026-09-09T20:25:08.574342774Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-09T20:25:08.713335023Z error: the server doesn't have a resource type "clusters" 2026-09-09T20:25:08.942968137Z Caches written to disk