2026-11-04T09:51:40.022121285Z [disk usage checker] Started
2026-11-04T09:51:40.025502735Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:51:40.912929037Z Gathering data for ns/openshift-cluster-version...
2026-11-04T09:51:41.765686702Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable no sooner than v6.0+
2026-11-04T09:51:42.302135864Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-11-04T09:51:42.780651636Z Gathering data for ns/default...
2026-11-04T09:51:44.565653585Z Gathering data for ns/openshift...
2026-11-04T09:51:45.031426371Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:51:46.457019347Z Gathering data for ns/kube-system...
2026-11-04T09:51:46.675953229Z Waiting on subprocesses to finish execution.
2026-11-04T09:51:46.680833743Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-11-04T09:51:46.682715873Z INFO: Gathering machine config daemon's old logs from all nodes
2026-11-04T09:51:46.689318440Z Executing Istio gather script
2026-11-04T09:51:46.694887530Z INFO: Gathering HAProxy config files
2026-11-04T09:51:46.701272588Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-11-04T09:51:46.721493235Z INFO: Waiting for node performance related collection to complete ...
2026-11-04T09:51:46.728093402Z INFO: Collecting host service logs for crio
2026-11-04T09:51:46.728406598Z INFO: Collecting host service logs for kubelet
2026-11-04T09:51:46.728684079Z INFO: Collecting host service logs for rpm-ostreed
2026-11-04T09:51:46.728966184Z INFO: Collecting host service logs for ostree-finalize-staged
2026-11-04T09:51:46.729252516Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-11-04T09:51:46.729540083Z INFO: Collecting host service logs for machine-config-daemon-host
2026-11-04T09:51:46.729862503Z INFO: Collecting host service logs for NetworkManager
2026-11-04T09:51:46.730223659Z INFO: Collecting host service logs for openvswitch
2026-11-04T09:51:46.730635064Z INFO: Collecting host service logs for ovs-configuration
2026-11-04T09:51:46.731095995Z INFO: Collecting host service logs for ovsdb-server
2026-11-04T09:51:46.731464193Z INFO: Collecting host service logs for ovs-vswitchd
2026-11-04T09:51:46.731881058Z INFO: Waiting for worker host service log collection to complete ...
2026-11-04T09:51:47.593770968Z No resources found in openshift-etcd namespace.
2026-11-04T09:51:47.717188370Z INFO: "sriov-network-operator" not detected. Skipping.
2026-11-04T09:51:47.782935562Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-11-04T09:51:47.782993786Z INFO: on-disk MachineConfig config collection complete.
2026-11-04T09:51:47.847731367Z INFO: Collecting Insights Archives from insights-operator-7c4986d449-nnmpp
2026-11-04T09:51:47.848333968Z insights-runtime-extractor-lgqzn
2026-11-04T09:51:47.848366414Z insights-runtime-extractor-lmfnf
2026-11-04T09:51:47.848389001Z insights-runtime-extractor-zrp8k
2026-11-04T09:51:48.030788251Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-11-04T09:51:48.125544399Z No resources found
2026-11-04T09:51:48.148713465Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-11-04T09:51:48.197206255Z INFO: "metallb-operator" not detected. Skipping.
2026-11-04T09:51:48.210428401Z INFO: "nfd" not detected. Skipping.
2026-11-04T09:51:48.440874623Z error: the server doesn't have a resource type "performanceprofile"
2026-11-04T09:51:48.531053609Z INFO: Found 1 replicas - prometheus-k8s-0
2026-11-04T09:51:48.538588333Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-11-04T09:51:48.538588333Z See 'oc rsync -h' for help and examples
2026-11-04T09:51:48.754604185Z INFO: Worker host service log collection to complete.
2026-11-04T09:51:48.813418976Z Wrote inspect data to must-gather.
2026-11-04T09:51:48.854642723Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-11-04T09:51:48.883108739Z INFO: Waiting for HAProxy config collection to complete ...
2026-11-04T09:51:48.956328380Z Wrote inspect data to must-gather.
2026-11-04T09:51:48.968520680Z ERROR: No running kube-apiserver pods found
2026-11-04T09:51:48.999239434Z Wrote inspect data to must-gather.
2026-11-04T09:51:49.111057130Z No resources found
2026-11-04T09:51:49.126540730Z Wrote inspect data to must-gather.
2026-11-04T09:51:49.142180830Z INFO: 'previous-logs' folder not found on ip-10-0-132-94.us-west-1.compute.internal, skipping...
2026-11-04T09:51:49.271938666Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-11-04T09:51:49.281403758Z Wrote inspect data to must-gather.
2026-11-04T09:51:49.323035305Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-11-04T09:51:49.433241095Z Wrote inspect data to must-gather.
2026-11-04T09:51:49.441624576Z Wrote inspect data to must-gather.
2026-11-04T09:51:49.589022709Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-11-04T09:51:49.593937692Z INFO: 'previous-logs' folder not found on ip-10-0-139-242.us-west-1.compute.internal, skipping...
2026-11-04T09:51:49.744253465Z tar: Removing leading `/' from member names
2026-11-04T09:51:49.755644130Z error: the server doesn't have a resource type "machineconfigs"
2026-11-04T09:51:49.845732567Z INFO: HAProxy config collection complete.
2026-11-04T09:51:49.939502144Z INFO: 'previous-logs' folder not found on ip-10-0-140-85.us-west-1.compute.internal, skipping...
2026-11-04T09:51:49.939540958Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-11-04T09:51:49.939540958Z INFO: Machine Config Daemon termination log collection complete.
2026-11-04T09:51:50.036125275Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:51:50.070777486Z Wrote inspect data to must-gather.
2026-11-04T09:51:50.173972739Z error: the server doesn't have a resource type "machineconfigpools"
2026-11-04T09:51:50.654872395Z INFO: Getting rules from prometheus-k8s-0
2026-11-04T09:51:50.709259377Z Wrote inspect data to must-gather.
2026-11-04T09:51:50.723911932Z Wrote inspect data to must-gather.
2026-11-04T09:51:50.884802493Z Inspecting resource crd/backendtlspolicies.gateway.networking.k8s.io
2026-11-04T09:51:51.146194927Z error: the server doesn't have a resource type "kubeletconfigs"
2026-11-04T09:51:51.374968210Z Wrote inspect data to must-gather.
2026-11-04T09:51:51.437406265Z Gathering data for ns/openshift-cloud-credential-operator...
2026-11-04T09:51:51.447764861Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:51:51.573171872Z Wrote inspect data to must-gather.
2026-11-04T09:51:51.980961090Z Wrote inspect data to must-gather.
2026-11-04T09:51:52.059630776Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:51:52.063096309Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-11-04T09:51:52.066080840Z INFO: LEGACY MODE
2026-11-04T09:51:52.066184553Z INFO: Gathering ovn-kubernetes master data
2026-11-04T09:51:52.121766009Z INFO: Getting status/config from prometheus-k8s-0
2026-11-04T09:51:52.193874595Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable no sooner than v6.0+
2026-11-04T09:51:52.394129311Z Wrote inspect data to must-gather.
2026-11-04T09:51:52.523730212Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:51:52.788664534Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-11-04T09:51:52.816439117Z Wrote inspect data to must-gather.
2026-11-04T09:51:53.198508667Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:51:53.201395303Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-11-04T09:51:53.349180970Z Wrote inspect data to must-gather.
2026-11-04T09:51:53.535815692Z INFO: Getting status/flags from prometheus-k8s-0
2026-11-04T09:51:53.740511971Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:51:53.778119838Z Wrote inspect data to must-gather.
2026-11-04T09:51:53.968109416Z INFO: Waiting for network log collection to complete ...
2026-11-04T09:51:53.968148686Z INFO: Waiting for ovnk database copies to complete ...
2026-11-04T09:51:53.968568850Z INFO: Copying ovnk databases complete.
2026-11-04T09:51:53.970310076Z du: cannot access 'must-gather/network_logs/ovnk_database_store': No such file or directory
2026-11-04T09:51:53.970816075Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 703 is not a child of this shell
2026-11-04T09:51:53.970816075Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 704 is not a child of this shell
2026-11-04T09:51:53.970816075Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 705 is not a child of this shell
2026-11-04T09:51:53.970816075Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 706 is not a child of this shell
2026-11-04T09:51:53.970816075Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 707 is not a child of this shell
2026-11-04T09:51:53.970816075Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 708 is not a child of this shell
2026-11-04T09:51:53.970816075Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 709 is not a child of this shell
2026-11-04T09:51:53.970845454Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 710 is not a child of this shell
2026-11-04T09:51:53.970845454Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 748 is not a child of this shell
2026-11-04T09:51:53.970845454Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 749 is not a child of this shell
2026-11-04T09:51:53.970845454Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 750 is not a child of this shell
2026-11-04T09:51:53.970845454Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 751 is not a child of this shell
2026-11-04T09:51:53.970845454Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 752 is not a child of this shell
2026-11-04T09:51:53.974185277Z tar: ovnk_database_store: Cannot stat: No such file or directory
2026-11-04T09:51:53.977825496Z tar: Exiting with failure status due to previous errors
2026-11-04T09:51:53.978345864Z INFO: Network log collection complete.
2026-11-04T09:51:54.321910509Z Wrote inspect data to must-gather.
2026-11-04T09:51:54.351096399Z Gathering data for ns/openshift-monitoring...
2026-11-04T09:51:54.372257443Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:51:54.374817048Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-11-04T09:51:54.420555979Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-11-04T09:51:54.479607027Z Gathering data for ns/openshift-etcd...
2026-11-04T09:51:54.898845832Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:51:55.040818861Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:51:55.112994251Z INFO: Hosted cluster detected, get NTO container image from the TuneD pod
2026-11-04T09:51:55.295901973Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-11-04T09:51:55.476499491Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:36f60e56e0594610b09d7a64639223d0a6a314315bca9313914f95c902a1597c
2026-11-04T09:51:55.498398044Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:51:55.501025570Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-11-04T09:51:55.834957525Z daemonset.apps/perf-node-gather-daemonset created
2026-11-04T09:51:56.030143551Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:51:56.095891685Z Waiting for performance profile collector pods to become ready: 1
2026-11-04T09:51:56.281726557Z Wrote inspect data to must-gather.
2026-11-04T09:51:56.281778540Z error: inspection completed with the errors occurred while gathering data:
2026-11-04T09:51:56.281778540Z     namespaces "assisted-installer" not found
2026-11-04T09:51:56.353608484Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-11-04T09:51:56.711482026Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:51:56.714245101Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-11-04T09:51:57.138823503Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:51:57.358637146Z Waiting for performance profile collector pods to become ready: 2
2026-11-04T09:51:57.574196666Z INFO: Getting status from alertmanager-main-0
2026-11-04T09:51:57.750817001Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:51:58.608261243Z Waiting for performance profile collector pods to become ready: 3
2026-11-04T09:51:59.476331238Z Done executing Istio gather script
2026-11-04T09:51:59.854759896Z Waiting for performance profile collector pods to become ready: 4
2026-11-04T09:52:00.045418712Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:52:01.100667143Z Waiting for performance profile collector pods to become ready: 5
2026-11-04T09:52:02.347351395Z Waiting for performance profile collector pods to become ready: 6
2026-11-04T09:52:03.601405011Z Waiting for performance profile collector pods to become ready: 7
2026-11-04T09:52:04.848495992Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-11-04T09:52:05.050036543Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:52:05.405016048Z Collecting performance related data for node ip-10-0-132-94.us-west-1.compute.internal
2026-11-04T09:52:05.411885460Z Collecting performance related data for node ip-10-0-140-85.us-west-1.compute.internal
2026-11-04T09:52:05.419253656Z Collecting performance related data for node ip-10-0-139-242.us-west-1.compute.internal
2026-11-04T09:52:10.054887559Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:52:15.061905490Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:52:15.369546420Z Gathering data for ns/openshift-network-console...
2026-11-04T09:52:17.990251420Z Gathering data for ns/openshift-console-operator...
2026-11-04T09:52:19.036002324Z Collecting kubelet logs for node ip-10-0-132-94.us-west-1.compute.internal
2026-11-04T09:52:19.037821014Z Collecting kubelet logs for node ip-10-0-140-85.us-west-1.compute.internal
2026-11-04T09:52:19.039570276Z Collecting kubelet logs for node ip-10-0-139-242.us-west-1.compute.internal
2026-11-04T09:52:19.864904198Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-h6r4f namespace
2026-11-04T09:52:19.919187469Z INFO: Node performance data collection complete.
2026-11-04T09:52:20.066690229Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:52:20.308964890Z Gathering data for ns/openshift-console...
2026-11-04T09:52:23.370087986Z Gathering data for ns/openshift-cluster-storage-operator...
2026-11-04T09:52:25.071447182Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:52:26.450687551Z Gathering data for ns/openshift-dns-operator...
2026-11-04T09:52:28.504949201Z Gathering data for ns/openshift-dns...
2026-11-04T09:52:30.076227656Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:52:34.482474511Z Gathering data for ns/openshift-image-registry...
2026-11-04T09:52:35.080821869Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:52:38.050871491Z Gathering data for ns/openshift-ingress-operator...
2026-11-04T09:52:40.003682594Z Gathering data for ns/openshift-ingress...
2026-11-04T09:52:40.085398212Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:52:42.377454798Z Gathering data for ns/openshift-ingress-canary...
2026-11-04T09:52:45.089990195Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:52:45.585833908Z Gathering data for ns/openshift-insights...
2026-11-04T09:52:50.094768663Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:52:53.577951696Z Gathering data for ns/openshift-jobset-operator...
2026-11-04T09:52:55.099417668Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:52:56.597147478Z Gathering data for ns/openshift-config...
2026-11-04T09:52:58.412829424Z Gathering data for ns/openshift-config-managed...
2026-11-04T09:53:00.106412124Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:53:00.391287468Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-11-04T09:53:02.218986191Z Gathering data for ns/openshift-kube-apiserver...
2026-11-04T09:53:04.023258109Z Gathering data for ns/openshift-kube-controller-manager...
2026-11-04T09:53:05.111057463Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:53:05.853462833Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-11-04T09:53:07.772640955Z Gathering data for ns/openshift-kube-scheduler...
2026-11-04T09:53:09.580239599Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-11-04T09:53:10.115733309Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:53:11.508319071Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-11-04T09:53:14.207967768Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-11-04T09:53:15.120310870Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:53:16.462671192Z Gathering data for ns/openshift-user-workload-monitoring...
2026-11-04T09:53:19.115493633Z Gathering data for ns/openshift-multus...
2026-11-04T09:53:20.125025989Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:53:25.129715844Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:53:30.134560065Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:53:35.139356787Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:53:35.586997066Z Gathering data for ns/openshift-ovn-kubernetes...
2026-11-04T09:53:40.144069840Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:53:45.148622861Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:53:49.189378487Z Gathering data for ns/openshift-host-network...
2026-11-04T09:53:50.153263046Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:53:51.278560660Z Gathering data for ns/openshift-network-diagnostics...
2026-11-04T09:53:55.157706392Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:53:55.816533802Z Gathering data for ns/openshift-network-node-identity...
2026-11-04T09:53:58.809123903Z Gathering data for ns/openshift-network-operator...
2026-11-04T09:54:00.162269019Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:54:01.949968523Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-11-04T09:54:03.754888916Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-11-04T09:54:05.169827286Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:54:07.282619885Z Gathering data for ns/openshift-apiserver-operator...
2026-11-04T09:54:09.098390006Z Gathering data for ns/openshift-apiserver...
2026-11-04T09:54:10.174654656Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:54:11.594837904Z Gathering data for ns/openshift-controller-manager-operator...
2026-11-04T09:54:13.386760870Z Gathering data for ns/openshift-controller-manager...
2026-11-04T09:54:15.179128601Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:54:15.242055447Z Gathering data for ns/openshift-cluster-samples-operator...
2026-11-04T09:54:18.331122460Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-11-04T09:54:20.183649727Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:54:20.193042210Z Gathering data for ns/openshift-service-ca-operator...
2026-11-04T09:54:22.424098958Z Gathering data for ns/openshift-service-ca...
2026-11-04T09:54:24.987934564Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-11-04T09:54:25.188141452Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:54:30.192589658Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:54:30.992903215Z Wrote inspect data to must-gather.
2026-11-04T09:54:30.992955189Z error: inspection completed with the errors occurred while gathering data:
2026-11-04T09:54:30.992955189Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-11-04T09:54:31.412003159Z error: the server doesn't have a resource type "clusters"
2026-11-04T09:54:31.454669047Z Caches written to disk