2026-09-18T20:47:02.861130859Z [disk usage checker] Started 2026-09-18T20:47:02.864322869Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-18T20:47:03.064888931Z Gathering data for ns/openshift-cluster-version... 2026-09-18T20:47:03.213726304Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-18T20:47:03.320154760Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-18T20:47:03.405604618Z Gathering data for ns/default... 2026-09-18T20:47:03.709908376Z Gathering data for ns/openshift... 2026-09-18T20:47:04.128921441Z Gathering data for ns/kube-system... 2026-09-18T20:47:04.896597057Z Gathering data for ns/openshift-etcd... 2026-09-18T20:47:05.233335933Z Wrote inspect data to must-gather. 2026-09-18T20:47:05.233369750Z error: inspection completed with the errors occurred while gathering data: 2026-09-18T20:47:05.233369750Z namespaces "assisted-installer" not found 2026-09-18T20:47:05.626295072Z Waiting on subprocesses to finish execution. 2026-09-18T20:47:05.640100343Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-18T20:47:05.640433492Z INFO: Waiting for node performance related collection to complete ... 2026-09-18T20:47:05.645966804Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-18T20:47:05.651372757Z INFO: Gathering HAProxy config files 2026-09-18T20:47:05.654910192Z INFO: Collecting host service logs for crio 2026-09-18T20:47:05.655239480Z INFO: Collecting host service logs for kubelet 2026-09-18T20:47:05.657255152Z INFO: Collecting host service logs for rpm-ostreed 2026-09-18T20:47:05.657255152Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-18T20:47:05.657255152Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-18T20:47:05.657255152Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-18T20:47:05.657255152Z INFO: Collecting host service logs for NetworkManager 2026-09-18T20:47:05.657255152Z INFO: Collecting host service logs for openvswitch 2026-09-18T20:47:05.657296544Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-18T20:47:05.657308404Z INFO: Collecting host service logs for ovs-configuration 2026-09-18T20:47:05.657665611Z INFO: Collecting host service logs for ovsdb-server 2026-09-18T20:47:05.657968575Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-18T20:47:05.658315536Z INFO: Waiting for worker host service log collection to complete ... 2026-09-18T20:47:05.676462588Z Executing Istio gather script 2026-09-18T20:47:06.090214345Z No resources found in openshift-etcd namespace. 2026-09-18T20:47:06.376626428Z Wrote inspect data to must-gather. 2026-09-18T20:47:06.466801117Z INFO: Collecting Insights Archives from insights-operator-6f9c644dd9-42gzv 2026-09-18T20:47:06.466846891Z insights-runtime-extractor-jlz2j 2026-09-18T20:47:06.466868175Z insights-runtime-extractor-knkx4 2026-09-18T20:47:06.466888267Z insights-runtime-extractor-ltk7w 2026-09-18T20:47:06.739985989Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-18T20:47:06.749723854Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-18T20:47:06.788493183Z error: the server doesn't have a resource type "performanceprofile" 2026-09-18T20:47:06.933706041Z No resources found 2026-09-18T20:47:07.001800530Z INFO: "metallb-operator" not detected. Skipping. 2026-09-18T20:47:07.160052297Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-18T20:47:07.287419101Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-18T20:47:07.287419101Z See 'oc rsync -h' for help and examples 2026-09-18T20:47:07.339659077Z Wrote inspect data to must-gather. 2026-09-18T20:47:07.406539780Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-18T20:47:07.406634626Z INFO: on-disk MachineConfig config collection complete. 2026-09-18T20:47:07.457380100Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-18T20:47:07.471530959Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-18T20:47:07.479962102Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-18T20:47:07.573901255Z INFO: Worker host service log collection to complete. 2026-09-18T20:47:07.769368618Z tar: Removing leading `/' from member names 2026-09-18T20:47:07.786053651Z INFO: HAProxy config collection complete. 2026-09-18T20:47:07.837912779Z No resources found 2026-09-18T20:47:07.855883831Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-18T20:47:07.917233556Z Wrote inspect data to must-gather. 2026-09-18T20:47:07.968540865Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-18T20:47:08.002696544Z ERROR: No running kube-apiserver pods found 2026-09-18T20:47:08.005154749Z Wrote inspect data to must-gather. 2026-09-18T20:47:08.117291248Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-18T20:47:08.144692521Z Wrote inspect data to must-gather. 2026-09-18T20:47:08.202635863Z INFO: 'previous-logs' folder not found on ip-10-0-138-158.ec2.internal, skipping... 2026-09-18T20:47:08.258564604Z Wrote inspect data to must-gather. 2026-09-18T20:47:08.269283765Z Wrote inspect data to must-gather. 2026-09-18T20:47:08.365049431Z error: the server doesn't have a resource type "machineconfigs" 2026-09-18T20:47:08.416764048Z Gathering data for ns/openshift-monitoring... 2026-09-18T20:47:08.559352031Z INFO: 'previous-logs' folder not found on ip-10-0-138-28.ec2.internal, skipping... 2026-09-18T20:47:08.583207352Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-18T20:47:08.598913061Z Wrote inspect data to must-gather. 2026-09-18T20:47:08.695057735Z INFO: Getting rules from prometheus-k8s-0 2026-09-18T20:47:08.872592001Z Wrote inspect data to must-gather. 2026-09-18T20:47:08.913038814Z Wrote inspect data to must-gather. 2026-09-18T20:47:08.918959293Z INFO: 'previous-logs' folder not found on ip-10-0-143-192.ec2.internal, skipping... 2026-09-18T20:47:08.919028910Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-18T20:47:08.919070372Z INFO: Machine Config Daemon termination log collection complete. 2026-09-18T20:47:09.048721961Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-18T20:47:09.182110297Z INFO: Getting status/config from prometheus-k8s-0 2026-09-18T20:47:09.214961407Z Wrote inspect data to must-gather. 2026-09-18T20:47:09.323197428Z Wrote inspect data to must-gather. 2026-09-18T20:47:09.466941730Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-18T20:47:09.580337137Z Wrote inspect data to must-gather. 2026-09-18T20:47:09.590793351Z INFO: INTERCONNECT MODE 2026-09-18T20:47:09.590822953Z INFO: Gathering ovn-kubernetes DBs 2026-09-18T20:47:09.668936605Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-18T20:47:09.775300677Z INFO: Gathering OVN_Northbound from ovnkube-node-dklfc... 2026-09-18T20:47:09.775726351Z INFO: Gathering OVN_Northbound from ovnkube-node-k9tlw... 2026-09-18T20:47:09.776004267Z INFO: Gathering OVN_Northbound from ovnkube-node-m5vk2... 2026-09-18T20:47:09.776416449Z INFO: Gathering OVN_Southbound from ovnkube-node-dklfc... 2026-09-18T20:47:09.776796060Z INFO: Gathering OVN_Southbound from ovnkube-node-k9tlw... 2026-09-18T20:47:09.777044626Z INFO: Gathering OVN_Southbound from ovnkube-node-m5vk2... 2026-09-18T20:47:09.840751413Z Wrote inspect data to must-gather. 2026-09-18T20:47:10.007399060Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-18T20:47:10.112013843Z tar: Removing leading `/' from member names 2026-09-18T20:47:10.136610995Z tar: Removing leading `/' from member names 2026-09-18T20:47:10.162924298Z tar: Removing leading `/' from member names 2026-09-18T20:47:10.168871706Z tar: Removing leading `/' from member names 2026-09-18T20:47:10.175964928Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-18T20:47:10.176188950Z tar: Removing leading `/' from member names 2026-09-18T20:47:10.228904722Z tar: Removing leading `/' from member names 2026-09-18T20:47:10.264809785Z Wrote inspect data to must-gather. 2026-09-18T20:47:10.338856098Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-18T20:47:10.349268239Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-18T20:47:10.545892206Z Wrote inspect data to must-gather. 2026-09-18T20:47:10.591780894Z Wrote inspect data to /must-gather/istio. 2026-09-18T20:47:10.652463362Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-18T20:47:10.743745482Z INFO: Waiting for network log collection to complete ... 2026-09-18T20:47:10.743859392Z INFO: Waiting for ovnk database copies to complete ... 2026-09-18T20:47:10.744368078Z INFO: Copying ovnk databases complete. 2026-09-18T20:47:10.747930399Z 3.7M must-gather/network_logs/ovnk_database_store 2026-09-18T20:47:10.750848334Z ovnk_database_store/ 2026-09-18T20:47:10.750848334Z ovnk_database_store/ovnkube-node-m5vk2_nbdb 2026-09-18T20:47:10.763850169Z ovnk_database_store/ovnkube-node-dklfc_sbdb 2026-09-18T20:47:10.789673982Z ovnk_database_store/ovnkube-node-k9tlw_nbdb 2026-09-18T20:47:10.797096371Z ovnk_database_store/ovnkube-node-dklfc_nbdb 2026-09-18T20:47:10.806237966Z W0918 20:47:10.806206 1460 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-09-18T20:47:10.809736693Z ovnk_database_store/ovnkube-node-k9tlw_sbdb 2026-09-18T20:47:10.810987046Z Wrote inspect data to must-gather. 2026-09-18T20:47:10.832353331Z ovnk_database_store/ovnkube-node-m5vk2_sbdb 2026-09-18T20:47:10.851499307Z Wrote inspect data to /must-gather/istio. 2026-09-18T20:47:10.854823009Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-18T20:47:10.865494385Z INFO: Network log collection complete. 2026-09-18T20:47:10.877132129Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-18T20:47:11.082739397Z Wrote inspect data to /must-gather/istio. 2026-09-18T20:47:11.084063158Z Gathering data for ns/openshift-network-console... 2026-09-18T20:47:11.112794776Z INFO: Getting status from alertmanager-main-0 2026-09-18T20:47:11.153572974Z Wrote inspect data to must-gather. 2026-09-18T20:47:11.309079737Z Wrote inspect data to /must-gather/istio. 2026-09-18T20:47:11.311670735Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-18T20:47:11.463913551Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-18T20:47:11.466563530Z INFO: Fallback to identify the container image from release info 2026-09-18T20:47:11.524465637Z Wrote inspect data to /must-gather/istio. 2026-09-18T20:47:11.587958260Z Gathering data for ns/openshift-console-operator... 2026-09-18T20:47:11.733164775Z Wrote inspect data to /must-gather/istio. 2026-09-18T20:47:11.735491966Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-18T20:47:11.959763042Z Wrote inspect data to /must-gather/istio. 2026-09-18T20:47:11.995762493Z Gathering data for ns/openshift-console... 2026-09-18T20:47:12.028515344Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0720c8b078cc0cebb01b4c5cde21180aee7c2bb1a0d322b21890c0d3707406ea 2026-09-18T20:47:12.179433620Z daemonset.apps/perf-node-gather-daemonset created 2026-09-18T20:47:12.195289540Z Wrote inspect data to /must-gather/istio. 2026-09-18T20:47:12.197688211Z Inspecting resource crd/inferencepools.inference.networking.k8s.io 2026-09-18T20:47:12.322769564Z Waiting for performance profile collector pods to become ready: 1 2026-09-18T20:47:12.390172078Z Wrote inspect data to /must-gather/istio. 2026-09-18T20:47:12.597446093Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-18T20:47:12.607956849Z Wrote inspect data to /must-gather/istio. 2026-09-18T20:47:12.611417525Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io 2026-09-18T20:47:12.792194460Z Wrote inspect data to /must-gather/istio. 2026-09-18T20:47:12.998590947Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-18T20:47:13.013081453Z Wrote inspect data to /must-gather/istio. 2026-09-18T20:47:13.017393292Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-18T20:47:13.147370332Z Gathering data for ns/openshift-dns-operator... 2026-09-18T20:47:13.208780989Z Wrote inspect data to /must-gather/istio. 2026-09-18T20:47:13.416366745Z Wrote inspect data to /must-gather/istio. 2026-09-18T20:47:13.478261704Z Waiting for performance profile collector pods to become ready: 2 2026-09-18T20:47:13.485192761Z Gathering data for ns/openshift-dns... 2026-09-18T20:47:14.088153825Z Done executing Istio gather script 2026-09-18T20:47:14.127153333Z Gathering data for ns/openshift-image-registry... 2026-09-18T20:47:14.609417396Z Waiting for performance profile collector pods to become ready: 3 2026-09-18T20:47:14.609887464Z Gathering data for ns/openshift-ingress-operator... 2026-09-18T20:47:14.932864327Z Gathering data for ns/openshift-ingress... 2026-09-18T20:47:15.276961692Z Gathering data for ns/openshift-ingress-canary... 2026-09-18T20:47:15.718997239Z Gathering data for ns/openshift-insights... 2026-09-18T20:47:15.740032892Z Waiting for performance profile collector pods to become ready: 4 2026-09-18T20:47:16.854053197Z Waiting for performance profile collector pods to become ready: 5 2026-09-18T20:47:17.998828181Z Waiting for performance profile collector pods to become ready: 6 2026-09-18T20:47:18.002985928Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-18T20:47:18.203949694Z Gathering data for ns/kserve... 2026-09-18T20:47:19.131189936Z Waiting for performance profile collector pods to become ready: 7 2026-09-18T20:47:19.883559013Z Gathering data for ns/openshift-config... 2026-09-18T20:47:20.250085833Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-18T20:47:20.279942129Z Gathering data for ns/openshift-config-managed... 2026-09-18T20:47:20.520251340Z Collecting performance related data for node ip-10-0-138-158.ec2.internal 2026-09-18T20:47:20.527089307Z Collecting performance related data for node ip-10-0-138-28.ec2.internal 2026-09-18T20:47:20.534286166Z Collecting performance related data for node ip-10-0-143-192.ec2.internal 2026-09-18T20:47:20.775153336Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-18T20:47:21.127727014Z Gathering data for ns/openshift-kube-apiserver... 2026-09-18T20:47:21.466496579Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-18T20:47:21.885020607Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-18T20:47:22.275677981Z Gathering data for ns/openshift-kube-scheduler... 2026-09-18T20:47:22.564769506Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-18T20:47:22.889855378Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-18T20:47:23.009292256Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-18T20:47:23.269703692Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-18T20:47:23.625623085Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-18T20:47:24.140521076Z Gathering data for ns/openshift-multus... 2026-09-18T20:47:25.773237530Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-18T20:47:27.157076172Z Gathering data for ns/openshift-host-network... 2026-09-18T20:47:27.460073835Z Gathering data for ns/openshift-network-diagnostics... 2026-09-18T20:47:28.015557228Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-18T20:47:28.114853904Z Gathering data for ns/openshift-network-node-identity... 2026-09-18T20:47:28.564897532Z Gathering data for ns/openshift-network-operator... 2026-09-18T20:47:28.983400450Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-18T20:47:29.122393278Z Collecting kubelet logs for node ip-10-0-138-158.ec2.internal 2026-09-18T20:47:29.124172484Z Collecting kubelet logs for node ip-10-0-138-28.ec2.internal 2026-09-18T20:47:29.125704293Z Collecting kubelet logs for node ip-10-0-143-192.ec2.internal 2026-09-18T20:47:29.333028274Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-18T20:47:29.534285806Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-dcbdm namespace 2026-09-18T20:47:29.539034555Z INFO: Node performance data collection complete. 2026-09-18T20:47:29.849775052Z Gathering data for ns/openshift-apiserver-operator... 2026-09-18T20:47:30.141595891Z Gathering data for ns/openshift-apiserver... 2026-09-18T20:47:30.521772519Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-18T20:47:30.778872834Z Gathering data for ns/openshift-controller-manager... 2026-09-18T20:47:31.036525671Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-18T20:47:31.653359303Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-18T20:47:31.934939462Z Gathering data for ns/openshift-service-ca-operator... 2026-09-18T20:47:32.247413159Z Gathering data for ns/openshift-service-ca... 2026-09-18T20:47:32.599129032Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-18T20:47:33.020164938Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-18T20:47:33.499263577Z Wrote inspect data to must-gather. 2026-09-18T20:47:33.499304874Z error: inspection completed with the errors occurred while gathering data: 2026-09-18T20:47:33.499304874Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-18T20:47:33.625420794Z error: the server doesn't have a resource type "clusters" 2026-09-18T20:47:33.765390471Z Caches written to disk