2026-09-16T19:13:37.819693522Z [disk usage checker] Started
2026-09-16T19:13:37.823079821Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-09-16T19:13:38.040637427Z Gathering data for ns/openshift-cluster-version...
2026-09-16T19:13:38.211872921Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-16T19:13:38.338948829Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T19:13:38.433326451Z Gathering data for ns/default...
2026-09-16T19:13:38.885623235Z Gathering data for ns/openshift...
2026-09-16T19:13:39.598778836Z Gathering data for ns/kube-system...
2026-09-16T19:13:40.634578635Z Waiting on subprocesses to finish execution.
2026-09-16T19:13:40.643697165Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-16T19:13:40.648130104Z INFO: Gathering HAProxy config files
2026-09-16T19:13:40.650838205Z INFO: Collecting host service logs for crio
2026-09-16T19:13:40.651161672Z INFO: Collecting host service logs for kubelet
2026-09-16T19:13:40.651542610Z INFO: Collecting host service logs for rpm-ostreed
2026-09-16T19:13:40.651880646Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-16T19:13:40.652184958Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-16T19:13:40.652536488Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-16T19:13:40.652869105Z INFO: Collecting host service logs for NetworkManager
2026-09-16T19:13:40.653217296Z INFO: Collecting host service logs for openvswitch
2026-09-16T19:13:40.653624011Z INFO: Collecting host service logs for ovs-configuration
2026-09-16T19:13:40.653978519Z INFO: Collecting host service logs for ovsdb-server
2026-09-16T19:13:40.654385931Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-16T19:13:40.654870744Z INFO: Waiting for worker host service log collection to complete ...
2026-09-16T19:13:40.663937829Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-16T19:13:40.685676592Z Executing Istio gather script
2026-09-16T19:13:40.691989907Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-16T19:13:40.815508976Z INFO: Waiting for node performance related collection to complete ...
2026-09-16T19:13:41.012819381Z No resources found
2026-09-16T19:13:41.143129642Z Wrote inspect data to must-gather.
2026-09-16T19:13:41.300766262Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-16T19:13:41.602017036Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-16T19:13:41.726532097Z error: the server doesn't have a resource type "performanceprofile"
2026-09-16T19:13:41.819157547Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-16T19:13:41.933403628Z INFO: Collecting Insights Archives from insights-operator-5844b856db-2hfjx
2026-09-16T19:13:41.933446734Z insights-runtime-extractor-c6jcz
2026-09-16T19:13:41.933465811Z insights-runtime-extractor-cnnk5
2026-09-16T19:13:41.933485891Z insights-runtime-extractor-m7k5m
2026-09-16T19:13:41.975747376Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-16T19:13:41.975779618Z INFO: on-disk MachineConfig config collection complete.
2026-09-16T19:13:41.991914111Z INFO: "metallb-operator" not detected. Skipping.
2026-09-16T19:13:42.055920713Z No resources found in openshift-etcd namespace.
2026-09-16T19:13:42.112298891Z Gathering data for ns/openshift-etcd...
2026-09-16T19:13:42.147669975Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-16T19:13:42.475177097Z INFO: Worker host service log collection to complete.
2026-09-16T19:13:42.522599544Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-16T19:13:42.539310016Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-16T19:13:42.556669871Z Wrote inspect data to must-gather.
2026-09-16T19:13:42.668044567Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-16T19:13:42.668044567Z See 'oc rsync -h' for help and examples
2026-09-16T19:13:42.866015526Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-09-16T19:13:42.868700768Z No resources found
2026-09-16T19:13:42.962720738Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T19:13:42.977199558Z Wrote inspect data to must-gather.
2026-09-16T19:13:43.092660246Z ERROR: No running kube-apiserver pods found
2026-09-16T19:13:43.120515845Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-16T19:13:43.173331864Z Wrote inspect data to must-gather.
2026-09-16T19:13:43.235549543Z Wrote inspect data to must-gather.
2026-09-16T19:13:43.270560605Z INFO: 'previous-logs' folder not found on ip-10-0-136-255.ec2.internal, skipping...
2026-09-16T19:13:43.297509727Z Wrote inspect data to must-gather.
2026-09-16T19:13:43.423292431Z Wrote inspect data to must-gather.
2026-09-16T19:13:43.451438380Z tar: Removing leading `/' from member names
2026-09-16T19:13:43.465864259Z INFO: HAProxy config collection complete.
2026-09-16T19:13:43.497055766Z error: the server doesn't have a resource type "machineconfigs"
2026-09-16T19:13:43.625247338Z Wrote inspect data to must-gather.
2026-09-16T19:13:43.625346037Z error: inspection completed with the errors occurred while gathering data:
2026-09-16T19:13:43.625346037Z     namespaces "assisted-installer" not found
2026-09-16T19:13:43.673858614Z INFO: 'previous-logs' folder not found on ip-10-0-138-120.ec2.internal, skipping...
2026-09-16T19:13:43.688898844Z Wrote inspect data to must-gather.
2026-09-16T19:13:43.706639022Z Gathering data for ns/openshift-monitoring...
2026-09-16T19:13:43.748857434Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-16T19:13:43.862898992Z INFO: Getting rules from prometheus-k8s-0
2026-09-16T19:13:43.989327890Z Wrote inspect data to must-gather.
2026-09-16T19:13:43.990690265Z INFO: 'previous-logs' folder not found on ip-10-0-143-174.ec2.internal, skipping...
2026-09-16T19:13:43.990711216Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-16T19:13:43.990735449Z INFO: Machine Config Daemon termination log collection complete.
2026-09-16T19:13:44.028463581Z Wrote inspect data to must-gather.
2026-09-16T19:13:44.181907494Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-16T19:13:44.239774001Z Wrote inspect data to must-gather.
2026-09-16T19:13:44.418128894Z INFO: Getting status/config from prometheus-k8s-0
2026-09-16T19:13:44.420503438Z Wrote inspect data to must-gather.
2026-09-16T19:13:44.496411687Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-16T19:13:44.555628445Z INFO: INTERCONNECT MODE
2026-09-16T19:13:44.555658047Z INFO: Gathering ovn-kubernetes DBs
2026-09-16T19:13:44.639403302Z Wrote inspect data to must-gather.
2026-09-16T19:13:44.722099432Z INFO: Gathering OVN_Northbound from ovnkube-node-6v8b7...
2026-09-16T19:13:44.722362128Z INFO: Gathering OVN_Northbound from ovnkube-node-8s6d2...
2026-09-16T19:13:44.722658343Z INFO: Gathering OVN_Northbound from ovnkube-node-nsg7q...
2026-09-16T19:13:44.722995269Z INFO: Gathering OVN_Southbound from ovnkube-node-6v8b7...
2026-09-16T19:13:44.723254256Z INFO: Gathering OVN_Southbound from ovnkube-node-8s6d2...
2026-09-16T19:13:44.723563551Z INFO: Gathering OVN_Southbound from ovnkube-node-nsg7q...
2026-09-16T19:13:44.993795982Z Wrote inspect data to must-gather.
2026-09-16T19:13:45.064197412Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-16T19:13:45.212127178Z tar: Removing leading `/' from member names
2026-09-16T19:13:45.228085119Z tar: Removing leading `/' from member names
2026-09-16T19:13:45.239413380Z tar: Removing leading `/' from member names
2026-09-16T19:13:45.240910824Z tar: Removing leading `/' from member names
2026-09-16T19:13:45.251834044Z tar: Removing leading `/' from member names
2026-09-16T19:13:45.273148011Z tar: Removing leading `/' from member names
2026-09-16T19:13:45.273645044Z Wrote inspect data to must-gather.
2026-09-16T19:13:45.394830254Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-16T19:13:45.526588076Z Wrote inspect data to must-gather.
2026-09-16T19:13:45.674614452Z INFO: Waiting for network log collection to complete ...
2026-09-16T19:13:45.674647059Z INFO: Waiting for ovnk database copies to complete ...
2026-09-16T19:13:45.675015505Z INFO: Copying ovnk databases complete.
2026-09-16T19:13:45.676841504Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-09-16T19:13:45.678952274Z ovnk_database_store/
2026-09-16T19:13:45.678952274Z ovnk_database_store/ovnkube-node-nsg7q_sbdb
2026-09-16T19:13:45.699411371Z ovnk_database_store/ovnkube-node-6v8b7_nbdb
2026-09-16T19:13:45.707491184Z ovnk_database_store/ovnkube-node-nsg7q_nbdb
2026-09-16T19:13:45.713131797Z ovnk_database_store/ovnkube-node-6v8b7_sbdb
2026-09-16T19:13:45.728957709Z ovnk_database_store/ovnkube-node-8s6d2_sbdb
2026-09-16T19:13:45.743874691Z ovnk_database_store/ovnkube-node-8s6d2_nbdb
2026-09-16T19:13:45.753633988Z INFO: Network log collection complete.
2026-09-16T19:13:45.755061380Z Wrote inspect data to must-gather.
2026-09-16T19:13:45.769658767Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-16T19:13:45.930823705Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-16T19:13:45.974837155Z Wrote inspect data to must-gather.
2026-09-16T19:13:46.066430259Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-16T19:13:46.115635510Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:13:46.259273037Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-16T19:13:46.261286084Z INFO: Fallback to identify the container image from release info
2026-09-16T19:13:46.347930758Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:13:46.351169847Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-16T19:13:46.502167749Z INFO: Getting status from alertmanager-main-0
2026-09-16T19:13:46.519142955Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-16T19:13:46.555748559Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:13:46.764988168Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:13:46.767731487Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-16T19:13:46.899032586Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a
2026-09-16T19:13:46.983183884Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:13:47.036522647Z daemonset.apps/perf-node-gather-daemonset created
2026-09-16T19:13:47.128972139Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T19:13:47.190263048Z Waiting for performance profile collector pods to become ready: 1
2026-09-16T19:13:47.213007430Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:13:47.215695178Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-16T19:13:47.341308237Z Gathering data for ns/openshift-network-console...
2026-09-16T19:13:47.458967007Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:13:47.673120604Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:13:47.675532169Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-09-16T19:13:47.852585520Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:13:48.026337290Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-09-16T19:13:48.079749923Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:13:48.084015467Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-09-16T19:13:48.085407103Z Gathering data for ns/openshift-console-operator...
2026-09-16T19:13:48.260642681Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:13:48.336743223Z Waiting for performance profile collector pods to become ready: 2
2026-09-16T19:13:48.459340625Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:13:48.461921438Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-16T19:13:48.621876893Z Gathering data for ns/openshift-console...
2026-09-16T19:13:48.639937902Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:13:48.839170035Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:13:49.365741641Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-16T19:13:49.466614307Z Waiting for performance profile collector pods to become ready: 3
2026-09-16T19:13:49.538294786Z Done executing Istio gather script
2026-09-16T19:13:50.033325333Z Gathering data for ns/openshift-dns-operator...
2026-09-16T19:13:50.409916139Z Gathering data for ns/openshift-dns...
2026-09-16T19:13:50.579987827Z Waiting for performance profile collector pods to become ready: 4
2026-09-16T19:13:51.479875324Z Gathering data for ns/openshift-image-registry...
2026-09-16T19:13:51.704609570Z Waiting for performance profile collector pods to become ready: 5
2026-09-16T19:13:52.120101286Z Gathering data for ns/openshift-ingress-operator...
2026-09-16T19:13:52.553515389Z Gathering data for ns/openshift-ingress...
2026-09-16T19:13:52.825152757Z Waiting for performance profile collector pods to become ready: 6
2026-09-16T19:13:53.032902351Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-09-16T19:13:53.060993546Z Gathering data for ns/openshift-ingress-canary...
2026-09-16T19:13:53.747335249Z Gathering data for ns/openshift-insights...
2026-09-16T19:13:53.944857900Z Waiting for performance profile collector pods to become ready: 7
2026-09-16T19:13:55.072109917Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-16T19:13:55.322626825Z Collecting performance related data for node ip-10-0-136-255.ec2.internal
2026-09-16T19:13:55.329741359Z Collecting performance related data for node ip-10-0-143-174.ec2.internal
2026-09-16T19:13:55.337244604Z Collecting performance related data for node ip-10-0-138-120.ec2.internal
2026-09-16T19:13:57.204910806Z Gathering data for ns/kserve...
2026-09-16T19:13:58.037947925Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-09-16T19:13:58.861251607Z Gathering data for ns/openshift-config...
2026-09-16T19:13:59.334086238Z Gathering data for ns/openshift-config-managed...
2026-09-16T19:13:59.928036603Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-16T19:14:00.371849014Z Gathering data for ns/openshift-kube-apiserver...
2026-09-16T19:14:00.775586205Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-16T19:14:01.209080954Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-16T19:14:01.641581009Z Gathering data for ns/openshift-kube-scheduler...
2026-09-16T19:14:01.980946953Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-16T19:14:02.357531765Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-16T19:14:02.867523121Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-16T19:14:03.042441786Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-09-16T19:14:03.337009680Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-16T19:14:03.873485762Z Collecting kubelet logs for node ip-10-0-136-255.ec2.internal
2026-09-16T19:14:03.875440210Z Collecting kubelet logs for node ip-10-0-143-174.ec2.internal
2026-09-16T19:14:03.877212680Z Collecting kubelet logs for node ip-10-0-138-120.ec2.internal
2026-09-16T19:14:04.067039820Z Gathering data for ns/openshift-multus...
2026-09-16T19:14:04.264206084Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-84wls namespace
2026-09-16T19:14:04.270169427Z INFO: Node performance data collection complete.
2026-09-16T19:14:07.681034323Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-16T19:14:08.046948246Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-09-16T19:14:10.044118319Z Gathering data for ns/openshift-host-network...
2026-09-16T19:14:10.525290292Z Gathering data for ns/openshift-network-diagnostics...
2026-09-16T19:14:11.361324005Z Gathering data for ns/openshift-network-node-identity...
2026-09-16T19:14:11.866535188Z Gathering data for ns/openshift-network-operator...
2026-09-16T19:14:12.480831136Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-16T19:14:12.927143480Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-16T19:14:13.051509662Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-09-16T19:14:13.652681760Z Gathering data for ns/openshift-apiserver-operator...
2026-09-16T19:14:14.102596784Z Gathering data for ns/openshift-apiserver...
2026-09-16T19:14:14.698398466Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-16T19:14:15.155085938Z Gathering data for ns/openshift-controller-manager...
2026-09-16T19:14:15.587338339Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-16T19:14:16.479759744Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-16T19:14:16.949472473Z Gathering data for ns/openshift-service-ca-operator...
2026-09-16T19:14:17.535918192Z Gathering data for ns/openshift-service-ca...
2026-09-16T19:14:18.055900351Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-09-16T19:14:18.204801764Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-16T19:14:19.594393346Z Wrote inspect data to must-gather.
2026-09-16T19:14:19.594434508Z error: inspection completed with the errors occurred while gathering data:
2026-09-16T19:14:19.594434508Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-16T19:14:19.734868339Z error: the server doesn't have a resource type "clusters"
2026-09-16T19:14:19.875147061Z Caches written to disk