2026-09-18T12:24:52.043945014Z [disk usage checker] Started
2026-09-18T12:24:52.047565659Z [disk usage checker] Volume usage percentage: current = 31 ; allowed = 70
2026-09-18T12:24:52.720991480Z Gathering data for ns/openshift-cluster-version...
2026-09-18T12:24:52.863860411Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-18T12:24:52.953671961Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-18T12:24:53.038141035Z Gathering data for ns/default...
2026-09-18T12:24:53.339099530Z Gathering data for ns/openshift...
2026-09-18T12:24:53.747846543Z Gathering data for ns/kube-system...
2026-09-18T12:24:54.451906806Z Gathering data for ns/openshift-etcd...
2026-09-18T12:24:54.758926278Z Wrote inspect data to must-gather.
2026-09-18T12:24:54.758958676Z error: inspection completed with the errors occurred while gathering data:
2026-09-18T12:24:54.758958676Z     namespaces "assisted-installer" not found
2026-09-18T12:24:55.131720559Z Waiting on subprocesses to finish execution.
2026-09-18T12:24:55.139537782Z INFO: Gathering HAProxy config files
2026-09-18T12:24:55.139675665Z Executing Istio gather script
2026-09-18T12:24:55.143157308Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-18T12:24:55.151289719Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-18T12:24:55.156426426Z INFO: Collecting host service logs for crio
2026-09-18T12:24:55.156905076Z INFO: Collecting host service logs for kubelet
2026-09-18T12:24:55.157268680Z INFO: Collecting host service logs for rpm-ostreed
2026-09-18T12:24:55.157633519Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-18T12:24:55.158019745Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-18T12:24:55.158382699Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-18T12:24:55.158771051Z INFO: Collecting host service logs for NetworkManager
2026-09-18T12:24:55.159142999Z INFO: Collecting host service logs for openvswitch
2026-09-18T12:24:55.159532340Z INFO: Collecting host service logs for ovs-configuration
2026-09-18T12:24:55.159957661Z INFO: Collecting host service logs for ovsdb-server
2026-09-18T12:24:55.160344897Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-18T12:24:55.160808772Z INFO: Waiting for worker host service log collection to complete ...
2026-09-18T12:24:55.188377764Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-18T12:24:55.205898993Z INFO: Waiting for node performance related collection to complete ...
2026-09-18T12:24:55.485530679Z INFO: "metallb-operator" not detected. Skipping.
2026-09-18T12:24:55.626679363Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-18T12:24:55.626805406Z INFO: on-disk MachineConfig config collection complete.
2026-09-18T12:24:55.694478172Z error: the server doesn't have a resource type "performanceprofile"
2026-09-18T12:24:55.962515613Z No resources found in openshift-etcd namespace.
2026-09-18T12:24:56.274996071Z Wrote inspect data to must-gather.
2026-09-18T12:24:56.321667966Z INFO: Collecting Insights Archives from insights-operator-6f9c644dd9-2qfsn
2026-09-18T12:24:56.321725112Z insights-runtime-extractor-992hx
2026-09-18T12:24:56.321745067Z insights-runtime-extractor-n697n
2026-09-18T12:24:56.321765028Z insights-runtime-extractor-pxsvt
2026-09-18T12:24:56.490668810Z No resources found
2026-09-18T12:24:56.808767086Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-18T12:24:57.086173608Z No resources found
2026-09-18T12:24:57.162125340Z [disk usage checker] Volume usage percentage: current = 31 ; allowed = 70
2026-09-18T12:24:57.171726119Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-18T12:24:57.171726119Z See 'oc rsync -h' for help and examples
2026-09-18T12:24:57.217230925Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-18T12:24:57.227546581Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-18T12:24:57.308824102Z Wrote inspect data to must-gather.
2026-09-18T12:24:57.327043969Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-18T12:24:57.354633548Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-18T12:24:57.391581631Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-18T12:24:57.427865740Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-18T12:24:57.439760608Z Wrote inspect data to must-gather.
2026-09-18T12:24:57.467143937Z ERROR: No running kube-apiserver pods found
2026-09-18T12:24:57.513920622Z Wrote inspect data to must-gather.
2026-09-18T12:24:57.520105845Z Wrote inspect data to must-gather.
2026-09-18T12:24:57.565102811Z Wrote inspect data to must-gather.
2026-09-18T12:24:57.578257245Z INFO: Worker host service log collection to complete.
2026-09-18T12:24:57.734075046Z Wrote inspect data to must-gather.
2026-09-18T12:24:57.834024359Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-18T12:24:57.848427420Z tar: Removing leading `/' from member names
2026-09-18T12:24:57.855687505Z INFO: HAProxy config collection complete.
2026-09-18T12:24:57.858005818Z error: the server doesn't have a resource type "machineconfigs"
2026-09-18T12:24:57.884736545Z INFO: 'previous-logs' folder not found on ip-10-0-132-9.ec2.internal, skipping...
2026-09-18T12:24:57.916610566Z Gathering data for ns/openshift-monitoring...
2026-09-18T12:24:57.917842343Z Wrote inspect data to must-gather.
2026-09-18T12:24:58.061688698Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-18T12:24:58.206752885Z INFO: 'previous-logs' folder not found on ip-10-0-136-137.ec2.internal, skipping...
2026-09-18T12:24:58.226780551Z Wrote inspect data to must-gather.
2026-09-18T12:24:58.318298226Z Wrote inspect data to must-gather.
2026-09-18T12:24:58.369347577Z INFO: Getting rules from prometheus-k8s-0
2026-09-18T12:24:58.501947858Z INFO: 'previous-logs' folder not found on ip-10-0-139-113.ec2.internal, skipping...
2026-09-18T12:24:58.501984130Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-18T12:24:58.502011928Z INFO: Machine Config Daemon termination log collection complete.
2026-09-18T12:24:58.528597719Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-18T12:24:58.538716259Z Wrote inspect data to must-gather.
2026-09-18T12:24:58.719950962Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-18T12:24:58.760427301Z Wrote inspect data to must-gather.
2026-09-18T12:24:58.809701073Z INFO: Getting status/config from prometheus-k8s-0
2026-09-18T12:24:58.846865194Z INFO: INTERCONNECT MODE
2026-09-18T12:24:58.846899040Z INFO: Gathering ovn-kubernetes DBs
2026-09-18T12:24:58.988786143Z Wrote inspect data to must-gather.
2026-09-18T12:24:59.018338467Z INFO: Gathering OVN_Northbound from ovnkube-node-5l7rg...
2026-09-18T12:24:59.018645727Z INFO: Gathering OVN_Northbound from ovnkube-node-j2tbf...
2026-09-18T12:24:59.018893648Z INFO: Gathering OVN_Northbound from ovnkube-node-rxcxq...
2026-09-18T12:24:59.019356264Z INFO: Gathering OVN_Southbound from ovnkube-node-5l7rg...
2026-09-18T12:24:59.019628678Z INFO: Gathering OVN_Southbound from ovnkube-node-j2tbf...
2026-09-18T12:24:59.019923045Z INFO: Gathering OVN_Southbound from ovnkube-node-rxcxq...
2026-09-18T12:24:59.331881452Z tar: Removing leading `/' from member names
2026-09-18T12:24:59.334294088Z tar: Removing leading `/' from member names
2026-09-18T12:24:59.352052645Z Wrote inspect data to must-gather.
2026-09-18T12:24:59.355186968Z tar: Removing leading `/' from member names
2026-09-18T12:24:59.375942451Z tar: Removing leading `/' from member names
2026-09-18T12:24:59.387075032Z tar: Removing leading `/' from member names
2026-09-18T12:24:59.401293925Z tar: Removing leading `/' from member names
2026-09-18T12:24:59.428758929Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-18T12:24:59.488856445Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-18T12:24:59.618951825Z Wrote inspect data to must-gather.
2026-09-18T12:24:59.695688544Z Wrote inspect data to /must-gather/istio.
2026-09-18T12:24:59.706248201Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-18T12:24:59.795679460Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-18T12:24:59.878397203Z Wrote inspect data to must-gather.
2026-09-18T12:24:59.879849572Z INFO: Waiting for network log collection to complete ...
2026-09-18T12:24:59.879903440Z INFO: Waiting for ovnk database copies to complete ...
2026-09-18T12:24:59.880474478Z INFO: Copying ovnk databases complete.
2026-09-18T12:24:59.882330053Z 3.3M	must-gather/network_logs/ovnk_database_store
2026-09-18T12:24:59.884879130Z ovnk_database_store/
2026-09-18T12:24:59.884930883Z ovnk_database_store/ovnkube-node-5l7rg_nbdb
2026-09-18T12:24:59.895750220Z ovnk_database_store/ovnkube-node-rxcxq_sbdb
2026-09-18T12:24:59.914625288Z ovnk_database_store/ovnkube-node-rxcxq_nbdb
2026-09-18T12:24:59.925627762Z ovnk_database_store/ovnkube-node-j2tbf_nbdb
2026-09-18T12:24:59.926081644Z Wrote inspect data to /must-gather/istio.
2026-09-18T12:24:59.929683286Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-18T12:24:59.935102647Z ovnk_database_store/ovnkube-node-j2tbf_sbdb
2026-09-18T12:24:59.956348717Z ovnk_database_store/ovnkube-node-5l7rg_sbdb
2026-09-18T12:24:59.980775236Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-18T12:24:59.982530859Z INFO: Network log collection complete.
2026-09-18T12:25:00.111460195Z Wrote inspect data to must-gather.
2026-09-18T12:25:00.138306577Z Wrote inspect data to /must-gather/istio.
2026-09-18T12:25:00.254245057Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-18T12:25:00.327913350Z W0918 12:25:00.327864    1525 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-18T12:25:00.335512306Z Wrote inspect data to must-gather.
2026-09-18T12:25:00.361716662Z Wrote inspect data to /must-gather/istio.
2026-09-18T12:25:00.366781045Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-18T12:25:00.431860627Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-18T12:25:00.563072725Z Wrote inspect data to /must-gather/istio.
2026-09-18T12:25:00.634039894Z Gathering data for ns/openshift-network-console...
2026-09-18T12:25:00.644014185Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-18T12:25:00.646947138Z INFO: Fallback to identify the container image from release info
2026-09-18T12:25:00.665905736Z INFO: Getting status from alertmanager-main-0
2026-09-18T12:25:00.799956339Z Wrote inspect data to /must-gather/istio.
2026-09-18T12:25:00.803085055Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-18T12:25:01.013652565Z Wrote inspect data to /must-gather/istio.
2026-09-18T12:25:01.102275373Z Gathering data for ns/openshift-console-operator...
2026-09-18T12:25:01.232231409Z Wrote inspect data to /must-gather/istio.
2026-09-18T12:25:01.236954952Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-18T12:25:01.246838562Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0720c8b078cc0cebb01b4c5cde21180aee7c2bb1a0d322b21890c0d3707406ea
2026-09-18T12:25:01.373568233Z daemonset.apps/perf-node-gather-daemonset created
2026-09-18T12:25:01.403889426Z Wrote inspect data to /must-gather/istio.
2026-09-18T12:25:01.522397674Z Waiting for performance profile collector pods to become ready: 1
2026-09-18T12:25:01.580150821Z Gathering data for ns/openshift-console...
2026-09-18T12:25:01.608891746Z Wrote inspect data to /must-gather/istio.
2026-09-18T12:25:02.152184435Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-18T12:25:02.179280113Z [disk usage checker] Volume usage percentage: current = 31 ; allowed = 70
2026-09-18T12:25:02.294106717Z Done executing Istio gather script
2026-09-18T12:25:02.652232777Z Waiting for performance profile collector pods to become ready: 2
2026-09-18T12:25:02.665754211Z Gathering data for ns/openshift-dns-operator...
2026-09-18T12:25:02.998990765Z Gathering data for ns/openshift-dns...
2026-09-18T12:25:03.675077167Z Gathering data for ns/openshift-image-registry...
2026-09-18T12:25:03.768690349Z Waiting for performance profile collector pods to become ready: 3
2026-09-18T12:25:04.295244848Z Gathering data for ns/openshift-ingress-operator...
2026-09-18T12:25:04.698787579Z Gathering data for ns/openshift-ingress...
2026-09-18T12:25:04.897644475Z Waiting for performance profile collector pods to become ready: 4
2026-09-18T12:25:05.138932632Z Gathering data for ns/openshift-ingress-canary...
2026-09-18T12:25:05.550072315Z Gathering data for ns/openshift-insights...
2026-09-18T12:25:06.004461732Z Waiting for performance profile collector pods to become ready: 5
2026-09-18T12:25:07.117217048Z Waiting for performance profile collector pods to become ready: 6
2026-09-18T12:25:07.190331880Z [disk usage checker] Volume usage percentage: current = 31 ; allowed = 70
2026-09-18T12:25:07.507639755Z Gathering data for ns/openshift-jobset-operator...
2026-09-18T12:25:08.225840654Z Waiting for performance profile collector pods to become ready: 7
2026-09-18T12:25:08.353522309Z Gathering data for ns/openshift-config...
2026-09-18T12:25:08.648386955Z Gathering data for ns/openshift-config-managed...
2026-09-18T12:25:09.073908177Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-18T12:25:09.343492977Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-18T12:25:09.399971462Z Gathering data for ns/openshift-kube-apiserver...
2026-09-18T12:25:09.597149283Z Collecting performance related data for node ip-10-0-139-113.ec2.internal
2026-09-18T12:25:09.604609700Z Collecting performance related data for node ip-10-0-132-9.ec2.internal
2026-09-18T12:25:09.615354552Z Collecting performance related data for node ip-10-0-136-137.ec2.internal
2026-09-18T12:25:09.718213301Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-18T12:25:10.076900252Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-18T12:25:10.424119918Z Gathering data for ns/openshift-kube-scheduler...
2026-09-18T12:25:10.758083685Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-18T12:25:11.128261187Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-18T12:25:11.530183013Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-18T12:25:11.877033129Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-18T12:25:12.197770771Z [disk usage checker] Volume usage percentage: current = 31 ; allowed = 70
2026-09-18T12:25:12.401946147Z Gathering data for ns/openshift-multus...
2026-09-18T12:25:13.953206187Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-18T12:25:15.523695335Z Gathering data for ns/openshift-host-network...
2026-09-18T12:25:15.901054942Z Gathering data for ns/openshift-network-diagnostics...
2026-09-18T12:25:16.473995291Z Gathering data for ns/openshift-network-node-identity...
2026-09-18T12:25:16.935391605Z Gathering data for ns/openshift-network-operator...
2026-09-18T12:25:17.205088399Z [disk usage checker] Volume usage percentage: current = 31 ; allowed = 70
2026-09-18T12:25:17.369306627Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-18T12:25:17.709729034Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-18T12:25:18.205626526Z Collecting kubelet logs for node ip-10-0-139-113.ec2.internal
2026-09-18T12:25:18.207521990Z Collecting kubelet logs for node ip-10-0-132-9.ec2.internal
2026-09-18T12:25:18.211116165Z Collecting kubelet logs for node ip-10-0-136-137.ec2.internal
2026-09-18T12:25:18.230803337Z Gathering data for ns/openshift-apiserver-operator...
2026-09-18T12:25:18.576983111Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-ngr5f namespace
2026-09-18T12:25:18.584047825Z INFO: Node performance data collection complete.
2026-09-18T12:25:18.641287154Z Gathering data for ns/openshift-apiserver...
2026-09-18T12:25:19.082791237Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-18T12:25:19.403603286Z Gathering data for ns/openshift-controller-manager...
2026-09-18T12:25:19.687291918Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-18T12:25:20.338457394Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-18T12:25:20.638627757Z Gathering data for ns/openshift-service-ca-operator...
2026-09-18T12:25:21.016488486Z Gathering data for ns/openshift-service-ca...
2026-09-18T12:25:21.407655783Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-18T12:25:22.210323661Z [disk usage checker] Volume usage percentage: current = 31 ; allowed = 70
2026-09-18T12:25:22.345414366Z Wrote inspect data to must-gather.
2026-09-18T12:25:22.345482333Z error: inspection completed with the errors occurred while gathering data:
2026-09-18T12:25:22.345482333Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-18T12:25:22.474552793Z error: the server doesn't have a resource type "clusters"
2026-09-18T12:25:22.609038712Z Caches written to disk