2026-07-09T18:37:46.625459413Z [disk usage checker] Started
2026-07-09T18:37:46.629153771Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-09T18:37:46.810858049Z Gathering data for ns/openshift-cluster-version...
2026-07-09T18:37:46.950935141Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-09T18:37:47.061866398Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-09T18:37:47.209322135Z Gathering data for ns/default...
2026-07-09T18:37:47.650586652Z Gathering data for ns/openshift...
2026-07-09T18:37:48.061066756Z Gathering data for ns/kube-system...
2026-07-09T18:37:48.830263797Z Gathering data for ns/openshift-etcd...
2026-07-09T18:37:49.045502578Z Waiting on subprocesses to finish execution.
2026-07-09T18:37:49.053877314Z INFO: Collecting host service logs for crio
2026-07-09T18:37:49.054175893Z INFO: Collecting host service logs for kubelet
2026-07-09T18:37:49.054527270Z INFO: Collecting host service logs for rpm-ostreed
2026-07-09T18:37:49.054856334Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-09T18:37:49.055185297Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-09T18:37:49.055579346Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-09T18:37:49.055903348Z INFO: Collecting host service logs for NetworkManager
2026-07-09T18:37:49.056221530Z INFO: Collecting host service logs for openvswitch
2026-07-09T18:37:49.056553701Z INFO: Collecting host service logs for ovs-configuration
2026-07-09T18:37:49.056897542Z INFO: Collecting host service logs for ovsdb-server
2026-07-09T18:37:49.057208660Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-09T18:37:49.057606899Z INFO: Waiting for worker host service log collection to complete ...
2026-07-09T18:37:49.059494855Z INFO: Gathering HAProxy config files
2026-07-09T18:37:49.065080194Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-09T18:37:49.070269636Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-09T18:37:49.095329452Z Executing Istio gather script
2026-07-09T18:37:49.099855064Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-09T18:37:49.135603756Z INFO: Waiting for node performance related collection to complete ...
2026-07-09T18:37:49.473626523Z error: the server doesn't have a resource type "performanceprofile"
2026-07-09T18:37:49.573591583Z No resources found in openshift-etcd namespace.
2026-07-09T18:37:49.711366420Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-09T18:37:49.741028158Z Wrote inspect data to must-gather.
2026-07-09T18:37:49.741061664Z error: inspection completed with the errors occurred while gathering data:
2026-07-09T18:37:49.741061664Z     namespaces "assisted-installer" not found
2026-07-09T18:37:49.866342770Z INFO: Collecting Insights Archives from insights-operator-68d9dd94cf-5gbcj
2026-07-09T18:37:49.866462023Z insights-runtime-extractor-mmdw6
2026-07-09T18:37:49.866503231Z insights-runtime-extractor-sgqtd
2026-07-09T18:37:49.866529317Z insights-runtime-extractor-xm8kj
2026-07-09T18:37:49.952905083Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-09T18:37:49.972335514Z Wrote inspect data to must-gather.
2026-07-09T18:37:50.135418456Z Wrote inspect data to must-gather.
2026-07-09T18:37:50.158691317Z INFO: "metallb-operator" not detected. Skipping.
2026-07-09T18:37:50.284959219Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-09T18:37:50.285000271Z INFO: on-disk MachineConfig config collection complete.
2026-07-09T18:37:50.303818242Z Wrote inspect data to must-gather.
2026-07-09T18:37:50.473853285Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-09T18:37:50.473853285Z See 'oc rsync -h' for help and examples
2026-07-09T18:37:50.498783623Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-09T18:37:50.512754510Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-09T18:37:50.518745715Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-09T18:37:50.527139758Z INFO: Worker host service log collection to complete.
2026-07-09T18:37:50.581307429Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-09T18:37:50.632949425Z No resources found
2026-07-09T18:37:50.741884675Z ERROR: No running kube-apiserver pods found
2026-07-09T18:37:50.860425631Z error: the server doesn't have a resource type "machineconfigs"
2026-07-09T18:37:51.036087314Z tar: Removing leading `/' from member names
2026-07-09T18:37:51.042280677Z INFO: HAProxy config collection complete.
2026-07-09T18:37:51.129030890Z No resources found
2026-07-09T18:37:51.171958976Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-09T18:37:51.403936634Z INFO: 'previous-logs' folder not found on ip-10-0-131-1.ec2.internal, skipping...
2026-07-09T18:37:51.447834447Z Wrote inspect data to must-gather.
2026-07-09T18:37:51.471966120Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-09T18:37:51.512829634Z Wrote inspect data to must-gather.
2026-07-09T18:37:51.553333305Z Gathering data for ns/openshift-monitoring...
2026-07-09T18:37:51.571651634Z Wrote inspect data to must-gather.
2026-07-09T18:37:51.577418446Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-09T18:37:51.641584278Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-09T18:37:51.656169551Z Wrote inspect data to must-gather.
2026-07-09T18:37:51.686860877Z INFO: Getting rules from prometheus-k8s-0
2026-07-09T18:37:51.764270040Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-09T18:37:51.824200122Z INFO: 'previous-logs' folder not found on ip-10-0-131-146.ec2.internal, skipping...
2026-07-09T18:37:51.874713190Z Wrote inspect data to must-gather.
2026-07-09T18:37:51.887156964Z Wrote inspect data to must-gather.
2026-07-09T18:37:52.040024925Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-09T18:37:52.074592222Z INFO: 'previous-logs' folder not found on ip-10-0-141-5.ec2.internal, skipping...
2026-07-09T18:37:52.074683926Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-09T18:37:52.074700226Z INFO: Machine Config Daemon termination log collection complete.
2026-07-09T18:37:52.139713181Z Wrote inspect data to must-gather.
2026-07-09T18:37:52.146493629Z INFO: Getting status/config from prometheus-k8s-0
2026-07-09T18:37:52.254943889Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-09T18:37:52.260823964Z Wrote inspect data to must-gather.
2026-07-09T18:37:52.374881253Z Wrote inspect data to must-gather.
2026-07-09T18:37:52.456432394Z Wrote inspect data to /must-gather/istio.
2026-07-09T18:37:52.512551334Z Wrote inspect data to must-gather.
2026-07-09T18:37:52.592995769Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-09T18:37:52.683586255Z Wrote inspect data to /must-gather/istio.
2026-07-09T18:37:52.686112300Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-09T18:37:52.691319801Z INFO: INTERCONNECT MODE
2026-07-09T18:37:52.691416462Z INFO: Gathering ovn-kubernetes DBs
2026-07-09T18:37:52.733772396Z Wrote inspect data to must-gather.
2026-07-09T18:37:52.842624110Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-09T18:37:52.851561475Z INFO: Gathering OVN_Northbound from ovnkube-node-d7qsz...
2026-07-09T18:37:52.851775144Z INFO: Gathering OVN_Northbound from ovnkube-node-s5wzq...
2026-07-09T18:37:52.851956828Z INFO: Gathering OVN_Northbound from ovnkube-node-z8gzd...
2026-07-09T18:37:52.852235675Z INFO: Gathering OVN_Southbound from ovnkube-node-d7qsz...
2026-07-09T18:37:52.852461582Z INFO: Gathering OVN_Southbound from ovnkube-node-s5wzq...
2026-07-09T18:37:52.852781435Z INFO: Gathering OVN_Southbound from ovnkube-node-z8gzd...
2026-07-09T18:37:52.890983040Z Wrote inspect data to /must-gather/istio.
2026-07-09T18:37:52.965298890Z Wrote inspect data to must-gather.
2026-07-09T18:37:53.128617382Z tar: Removing leading `/' from member names
2026-07-09T18:37:53.167617966Z tar: Removing leading `/' from member names
2026-07-09T18:37:53.192185567Z tar: Removing leading `/' from member names
2026-07-09T18:37:53.217112061Z tar: Removing leading `/' from member names
2026-07-09T18:37:53.218005046Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-09T18:37:53.219069620Z tar: Removing leading `/' from member names
2026-07-09T18:37:53.231153334Z Wrote inspect data to /must-gather/istio.
2026-07-09T18:37:53.234544873Z tar: Removing leading `/' from member names
2026-07-09T18:37:53.234872243Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-09T18:37:53.238891514Z Wrote inspect data to must-gather.
2026-07-09T18:37:53.455722832Z Wrote inspect data to /must-gather/istio.
2026-07-09T18:37:53.487004851Z Wrote inspect data to must-gather.
2026-07-09T18:37:53.494396577Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-09T18:37:53.647659914Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-09T18:37:53.665720235Z INFO: Waiting for network log collection to complete ...
2026-07-09T18:37:53.665818540Z INFO: Waiting for ovnk database copies to complete ...
2026-07-09T18:37:53.666210365Z INFO: Copying ovnk databases complete.
2026-07-09T18:37:53.667814779Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-07-09T18:37:53.669861979Z ovnk_database_store/
2026-07-09T18:37:53.669861979Z ovnk_database_store/ovnkube-node-z8gzd_nbdb
2026-07-09T18:37:53.679344889Z ovnk_database_store/ovnkube-node-z8gzd_sbdb
2026-07-09T18:37:53.679732710Z Wrote inspect data to /must-gather/istio.
2026-07-09T18:37:53.681681385Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-09T18:37:53.692862397Z ovnk_database_store/ovnkube-node-s5wzq_sbdb
2026-07-09T18:37:53.711087885Z ovnk_database_store/ovnkube-node-s5wzq_nbdb
2026-07-09T18:37:53.715007858Z Wrote inspect data to must-gather.
2026-07-09T18:37:53.718545980Z ovnk_database_store/ovnkube-node-d7qsz_nbdb
2026-07-09T18:37:53.725287462Z ovnk_database_store/ovnkube-node-d7qsz_sbdb
2026-07-09T18:37:53.742221369Z INFO: Network log collection complete.
2026-07-09T18:37:53.894079970Z INFO: Getting status from alertmanager-main-0
2026-07-09T18:37:53.911084571Z Wrote inspect data to /must-gather/istio.
2026-07-09T18:37:53.981661061Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-09T18:37:53.984184214Z INFO: Fallback to identify the container image from release info
2026-07-09T18:37:54.104551481Z Wrote inspect data to /must-gather/istio.
2026-07-09T18:37:54.106988770Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-09T18:37:54.234892237Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-09T18:37:54.261107358Z Wrote inspect data to /must-gather/istio.
2026-07-09T18:37:54.395495025Z Gathering data for ns/openshift-network-console...
2026-07-09T18:37:54.428551007Z Wrote inspect data to /must-gather/istio.
2026-07-09T18:37:54.532859957Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d8d5dc4ea635b13544e6f92103089219e732369ae30557b7beee2e8a11bf500d
2026-07-09T18:37:54.645631926Z daemonset.apps/perf-node-gather-daemonset created
2026-07-09T18:37:54.774260069Z Waiting for performance profile collector pods to become ready: 1
2026-07-09T18:37:54.868519331Z Gathering data for ns/openshift-console-operator...
2026-07-09T18:37:55.065705491Z Done executing Istio gather script
2026-07-09T18:37:55.269961127Z Gathering data for ns/openshift-console...
2026-07-09T18:37:55.777024630Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-09T18:37:55.893204476Z Waiting for performance profile collector pods to become ready: 2
2026-07-09T18:37:56.286715254Z Gathering data for ns/openshift-dns-operator...
2026-07-09T18:37:56.620204816Z Gathering data for ns/openshift-dns...
2026-07-09T18:37:56.646433843Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-09T18:37:57.005770286Z Waiting for performance profile collector pods to become ready: 3
2026-07-09T18:37:57.270682424Z Gathering data for ns/openshift-image-registry...
2026-07-09T18:37:57.798701110Z Gathering data for ns/openshift-ingress-operator...
2026-07-09T18:37:58.120764459Z Waiting for performance profile collector pods to become ready: 4
2026-07-09T18:37:58.156744978Z Gathering data for ns/openshift-ingress...
2026-07-09T18:37:58.522134896Z Gathering data for ns/openshift-ingress-canary...
2026-07-09T18:37:58.991518986Z Gathering data for ns/openshift-insights...
2026-07-09T18:37:59.214458285Z Waiting for performance profile collector pods to become ready: 5
2026-07-09T18:38:00.317227015Z Waiting for performance profile collector pods to become ready: 6
2026-07-09T18:38:00.930230611Z Gathering data for ns/openshift-config...
2026-07-09T18:38:01.215345030Z Gathering data for ns/openshift-config-managed...
2026-07-09T18:38:01.417073544Z Waiting for performance profile collector pods to become ready: 7
2026-07-09T18:38:01.645965731Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-09T18:38:01.650805776Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-09T18:38:01.927055020Z Gathering data for ns/openshift-kube-apiserver...
2026-07-09T18:38:02.211137227Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-09T18:38:02.528614479Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-09T18:38:02.558841874Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-09T18:38:02.759265064Z Collecting performance related data for node ip-10-0-131-1.ec2.internal
2026-07-09T18:38:02.767317884Z Collecting performance related data for node ip-10-0-141-5.ec2.internal
2026-07-09T18:38:02.776651995Z Collecting performance related data for node ip-10-0-131-146.ec2.internal
2026-07-09T18:38:02.918439620Z Gathering data for ns/openshift-kube-scheduler...
2026-07-09T18:38:03.238014191Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-09T18:38:03.628138924Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-09T18:38:04.079999101Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-09T18:38:04.462754224Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-09T18:38:05.012444246Z Gathering data for ns/openshift-multus...
2026-07-09T18:38:06.656596756Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-09T18:38:06.749978245Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-09T18:38:08.191248550Z Gathering data for ns/openshift-host-network...
2026-07-09T18:38:08.550221069Z Gathering data for ns/openshift-network-diagnostics...
2026-07-09T18:38:09.154978828Z Gathering data for ns/openshift-network-node-identity...
2026-07-09T18:38:09.586935433Z Gathering data for ns/openshift-network-operator...
2026-07-09T18:38:10.067809050Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-09T18:38:10.352512893Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-09T18:38:10.511748352Z Collecting kubelet logs for node ip-10-0-131-1.ec2.internal
2026-07-09T18:38:10.513546103Z Collecting kubelet logs for node ip-10-0-141-5.ec2.internal
2026-07-09T18:38:10.515124363Z Collecting kubelet logs for node ip-10-0-131-146.ec2.internal
2026-07-09T18:38:10.824649871Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-d6fzq namespace
2026-07-09T18:38:10.830902471Z INFO: Node performance data collection complete.
2026-07-09T18:38:10.918081805Z Gathering data for ns/openshift-apiserver-operator...
2026-07-09T18:38:11.237320512Z Gathering data for ns/openshift-apiserver...
2026-07-09T18:38:11.631444601Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-09T18:38:11.662153192Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-09T18:38:11.950447383Z Gathering data for ns/openshift-controller-manager...
2026-07-09T18:38:12.255533484Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-09T18:38:12.893708234Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-09T18:38:13.182537248Z Gathering data for ns/openshift-service-ca-operator...
2026-07-09T18:38:13.504238961Z Gathering data for ns/openshift-service-ca...
2026-07-09T18:38:13.871612107Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-09T18:38:14.839623778Z Wrote inspect data to must-gather.
2026-07-09T18:38:14.839965852Z error: inspection completed with the errors occurred while gathering data:
2026-07-09T18:38:14.839965852Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-09T18:38:14.955640251Z error: the server doesn't have a resource type "clusters"
2026-07-09T18:38:15.056034274Z Caches written to disk