2026-05-21T16:12:09.299626494Z [disk usage checker] Started
2026-05-21T16:12:09.302758129Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-21T16:12:09.529812431Z Gathering data for ns/openshift-cluster-version...
2026-05-21T16:12:09.700476118Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-21T16:12:09.839801365Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-21T16:12:09.944205129Z Gathering data for ns/default...
2026-05-21T16:12:10.315479534Z Gathering data for ns/openshift...
2026-05-21T16:12:10.818246729Z Gathering data for ns/kube-system...
2026-05-21T16:12:11.554982239Z Gathering data for ns/openshift-etcd...
2026-05-21T16:12:11.726009849Z Waiting on subprocesses to finish execution.
2026-05-21T16:12:11.732852763Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-21T16:12:11.735030126Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-21T16:12:11.737223884Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-21T16:12:11.741301567Z INFO: Gathering HAProxy config files
2026-05-21T16:12:11.763692097Z INFO: Collecting host service logs for crio
2026-05-21T16:12:11.764029396Z INFO: Collecting host service logs for kubelet
2026-05-21T16:12:11.764361489Z INFO: Collecting host service logs for rpm-ostreed
2026-05-21T16:12:11.764702072Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-21T16:12:11.765206392Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-21T16:12:11.765526931Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-21T16:12:11.765848518Z INFO: Collecting host service logs for NetworkManager
2026-05-21T16:12:11.766137352Z INFO: Collecting host service logs for openvswitch
2026-05-21T16:12:11.766479915Z INFO: Collecting host service logs for ovs-configuration
2026-05-21T16:12:11.766805631Z INFO: Collecting host service logs for ovsdb-server
2026-05-21T16:12:11.767110805Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-21T16:12:11.767454284Z INFO: Waiting for worker host service log collection to complete ...
2026-05-21T16:12:11.784751584Z INFO: Waiting for node performance related collection to complete ...
2026-05-21T16:12:12.331723517Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-21T16:12:12.331825471Z INFO: on-disk MachineConfig config collection complete.
2026-05-21T16:12:12.343759992Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-21T16:12:12.511528765Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-21T16:12:12.551710740Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-21T16:12:12.673440435Z No resources found in openshift-etcd namespace.
2026-05-21T16:12:12.789402061Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-21T16:12:12.867347947Z No resources found
2026-05-21T16:12:12.903536286Z Wrote inspect data to must-gather.
2026-05-21T16:12:12.931785855Z INFO: "metallb-operator" not detected. Skipping.
2026-05-21T16:12:13.054721044Z Wrote inspect data to must-gather.
2026-05-21T16:12:13.054815136Z error: inspection completed with the errors occurred while gathering data:
2026-05-21T16:12:13.054815136Z     namespaces "assisted-installer" not found
2026-05-21T16:12:13.320800832Z error: the server doesn't have a resource type "performanceprofile"
2026-05-21T16:12:13.334915597Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-pwpzw
2026-05-21T16:12:13.334961190Z insights-runtime-extractor-4mmst
2026-05-21T16:12:13.335028508Z insights-runtime-extractor-n62hc
2026-05-21T16:12:13.335052656Z insights-runtime-extractor-td64q
2026-05-21T16:12:13.517127231Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-21T16:12:13.578334564Z No resources found
2026-05-21T16:12:13.764680328Z ERROR: No running kube-apiserver pods found
2026-05-21T16:12:13.806331387Z INFO: Worker host service log collection to complete.
2026-05-21T16:12:14.022739208Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-21T16:12:14.074517819Z INFO: 'previous-logs' folder not found on ip-10-0-129-52.ec2.internal, skipping...
2026-05-21T16:12:14.333798847Z Wrote inspect data to must-gather.
2026-05-21T16:12:14.420684370Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-21T16:12:14.492134841Z Gathering data for ns/kuadrant-system...
2026-05-21T16:12:14.513690835Z Wrote inspect data to must-gather.
2026-05-21T16:12:14.562744840Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-21T16:12:14.571252249Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-21T16:12:14.571252249Z See 'oc rsync -h' for help and examples
2026-05-21T16:12:14.587102187Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-21T16:12:14.656351320Z Wrote inspect data to must-gather.
2026-05-21T16:12:14.675719846Z Wrote inspect data to must-gather.
2026-05-21T16:12:14.687995969Z Wrote inspect data to must-gather.
2026-05-21T16:12:14.838848564Z INFO: Getting rules from prometheus-k8s-0
2026-05-21T16:12:14.850105177Z tar: Removing leading `/' from member names
2026-05-21T16:12:14.855870221Z INFO: HAProxy config collection complete.
2026-05-21T16:12:14.856559969Z INFO: 'previous-logs' folder not found on ip-10-0-136-0.ec2.internal, skipping...
2026-05-21T16:12:14.906046661Z error: the server doesn't have a resource type "machineconfigs"
2026-05-21T16:12:15.092893930Z Wrote inspect data to must-gather.
2026-05-21T16:12:15.301114795Z INFO: 'previous-logs' folder not found on ip-10-0-136-55.ec2.internal, skipping...
2026-05-21T16:12:15.301235976Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-21T16:12:15.301317717Z INFO: Machine Config Daemon termination log collection complete.
2026-05-21T16:12:15.321019867Z Wrote inspect data to must-gather.
2026-05-21T16:12:15.420194645Z INFO: Getting status/config from prometheus-k8s-0
2026-05-21T16:12:15.531053005Z Wrote inspect data to must-gather.
2026-05-21T16:12:15.653442282Z Wrote inspect data to must-gather.
2026-05-21T16:12:15.779019196Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-21T16:12:15.875266862Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-21T16:12:15.940461286Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-21T16:12:15.940762831Z Wrote inspect data to must-gather.
2026-05-21T16:12:16.200093441Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-21T16:12:16.203897699Z Wrote inspect data to must-gather.
2026-05-21T16:12:16.330840686Z INFO: INTERCONNECT MODE
2026-05-21T16:12:16.330885887Z INFO: Gathering ovn-kubernetes DBs
2026-05-21T16:12:16.344890019Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-21T16:12:16.531975459Z INFO: Gathering OVN_Northbound from ovnkube-node-gkx7z...
2026-05-21T16:12:16.532235656Z INFO: Gathering OVN_Northbound from ovnkube-node-j2t49...
2026-05-21T16:12:16.532479182Z INFO: Gathering OVN_Northbound from ovnkube-node-v9w2m...
2026-05-21T16:12:16.532847910Z INFO: Gathering OVN_Southbound from ovnkube-node-gkx7z...
2026-05-21T16:12:16.533120356Z INFO: Gathering OVN_Southbound from ovnkube-node-j2t49...
2026-05-21T16:12:16.533382511Z INFO: Gathering OVN_Southbound from ovnkube-node-v9w2m...
2026-05-21T16:12:16.566044553Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-21T16:12:16.590455778Z Gathering data for ns/openshift-monitoring...
2026-05-21T16:12:16.631571096Z Wrote inspect data to must-gather.
2026-05-21T16:12:16.856367671Z tar: Removing leading `/' from member names
2026-05-21T16:12:16.926861714Z tar: Removing leading `/' from member names
2026-05-21T16:12:16.938948733Z tar: Removing leading `/' from member names
2026-05-21T16:12:16.940897731Z tar: Removing leading `/' from member names
2026-05-21T16:12:16.978043746Z tar: Removing leading `/' from member names
2026-05-21T16:12:16.999846192Z tar: Removing leading `/' from member names
2026-05-21T16:12:17.014669473Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-21T16:12:17.188231362Z Wrote inspect data to must-gather.
2026-05-21T16:12:17.462745567Z INFO: Getting status from alertmanager-main-0
2026-05-21T16:12:17.623425797Z Wrote inspect data to must-gather.
2026-05-21T16:12:17.655123481Z INFO: Waiting for network log collection to complete ...
2026-05-21T16:12:17.655160983Z INFO: Waiting for ovnk database copies to complete ...
2026-05-21T16:12:17.655670326Z INFO: Copying ovnk databases complete.
2026-05-21T16:12:17.657256861Z 4.1M	must-gather/network_logs/ovnk_database_store
2026-05-21T16:12:17.659415856Z ovnk_database_store/
2026-05-21T16:12:17.659468229Z ovnk_database_store/ovnkube-node-gkx7z_sbdb
2026-05-21T16:12:17.735280226Z ovnk_database_store/ovnkube-node-v9w2m_nbdb
2026-05-21T16:12:17.744914139Z ovnk_database_store/ovnkube-node-v9w2m_sbdb
2026-05-21T16:12:17.769262699Z ovnk_database_store/ovnkube-node-gkx7z_nbdb
2026-05-21T16:12:17.783536732Z ovnk_database_store/ovnkube-node-j2t49_nbdb
2026-05-21T16:12:17.793840838Z ovnk_database_store/ovnkube-node-j2t49_sbdb
2026-05-21T16:12:17.827742121Z INFO: Network log collection complete.
2026-05-21T16:12:17.975890095Z Wrote inspect data to must-gather.
2026-05-21T16:12:18.253270959Z Wrote inspect data to must-gather.
2026-05-21T16:12:18.527052421Z Wrote inspect data to must-gather.
2026-05-21T16:12:18.785034075Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-21T16:12:18.787685413Z INFO: Fallback to identify the container image from release info
2026-05-21T16:12:18.909096143Z Gathering data for ns/openshift-network-console...
2026-05-21T16:12:19.411707585Z Gathering data for ns/openshift-console-operator...
2026-05-21T16:12:19.413417967Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda
2026-05-21T16:12:19.437962329Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-21T16:12:19.579467152Z daemonset.apps/perf-node-gather-daemonset created
2026-05-21T16:12:19.715185078Z Waiting for performance profile collector pods to become ready: 1
2026-05-21T16:12:19.964323219Z Gathering data for ns/openshift-console...
2026-05-21T16:12:20.517747437Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-21T16:12:20.835914976Z Waiting for performance profile collector pods to become ready: 2
2026-05-21T16:12:20.981963550Z Gathering data for ns/openshift-dns-operator...
2026-05-21T16:12:21.327797936Z Gathering data for ns/openshift-dns...
2026-05-21T16:12:21.397250034Z Wrote inspect data to must-gather.
2026-05-21T16:12:21.947559214Z Gathering data for ns/openshift-image-registry...
2026-05-21T16:12:21.967664232Z Waiting for performance profile collector pods to become ready: 3
2026-05-21T16:12:22.488652503Z Gathering data for ns/openshift-ingress-operator...
2026-05-21T16:12:22.841074928Z Gathering data for ns/openshift-ingress...
2026-05-21T16:12:23.076272166Z Waiting for performance profile collector pods to become ready: 4
2026-05-21T16:12:23.619723069Z Gathering data for ns/openshift-ingress-canary...
2026-05-21T16:12:24.078276256Z Gathering data for ns/openshift-insights...
2026-05-21T16:12:24.183935144Z Waiting for performance profile collector pods to become ready: 5
2026-05-21T16:12:24.442688626Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-21T16:12:25.300646518Z Waiting for performance profile collector pods to become ready: 6
2026-05-21T16:12:26.108266199Z Gathering data for ns/opendatahub...
2026-05-21T16:12:26.422174136Z Waiting for performance profile collector pods to become ready: 7
2026-05-21T16:12:27.543141043Z Gathering data for ns/openshift-lws-operator...
2026-05-21T16:12:27.544536677Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-21T16:12:27.801094791Z Collecting performance related data for node ip-10-0-129-52.ec2.internal
2026-05-21T16:12:27.807895766Z Collecting performance related data for node ip-10-0-136-55.ec2.internal
2026-05-21T16:12:27.815442500Z Collecting performance related data for node ip-10-0-136-0.ec2.internal
2026-05-21T16:12:29.069753500Z Gathering data for ns/openshift-config...
2026-05-21T16:12:29.436157531Z Gathering data for ns/openshift-config-managed...
2026-05-21T16:12:29.448299776Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-21T16:12:29.899761545Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-21T16:12:30.244877652Z Gathering data for ns/openshift-kube-apiserver...
2026-05-21T16:12:30.630680915Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-21T16:12:31.041752246Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-21T16:12:31.453384076Z Gathering data for ns/openshift-kube-scheduler...
2026-05-21T16:12:31.759309759Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-21T16:12:32.123513678Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-21T16:12:32.497904287Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-21T16:12:32.865817388Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-21T16:12:33.487731212Z Gathering data for ns/openshift-multus...
2026-05-21T16:12:34.453639830Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-21T16:12:35.294715215Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-21T16:12:36.149026356Z Collecting kubelet logs for node ip-10-0-129-52.ec2.internal
2026-05-21T16:12:36.151022292Z Collecting kubelet logs for node ip-10-0-136-55.ec2.internal
2026-05-21T16:12:36.153199024Z Collecting kubelet logs for node ip-10-0-136-0.ec2.internal
2026-05-21T16:12:36.513660893Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-21T16:12:36.518710071Z INFO: Node performance data collection complete.
2026-05-21T16:12:36.982976065Z Gathering data for ns/openshift-host-network...
2026-05-21T16:12:37.338719996Z Gathering data for ns/openshift-network-diagnostics...
2026-05-21T16:12:37.912296092Z Gathering data for ns/openshift-network-node-identity...
2026-05-21T16:12:38.391043107Z Gathering data for ns/openshift-network-operator...
2026-05-21T16:12:38.788245618Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-21T16:12:39.106992287Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-21T16:12:39.458056658Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-21T16:12:39.595484068Z Gathering data for ns/openshift-apiserver-operator...
2026-05-21T16:12:39.926626162Z Gathering data for ns/openshift-apiserver...
2026-05-21T16:12:40.372924243Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-21T16:12:40.698322602Z Gathering data for ns/openshift-controller-manager...
2026-05-21T16:12:41.005707852Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-21T16:12:41.728785156Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-21T16:12:42.062061453Z Gathering data for ns/openshift-service-ca-operator...
2026-05-21T16:12:42.403388643Z Gathering data for ns/openshift-service-ca...
2026-05-21T16:12:42.851281928Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-21T16:12:43.878312283Z Wrote inspect data to must-gather.
2026-05-21T16:12:43.878353664Z error: inspection completed with the errors occurred while gathering data:
2026-05-21T16:12:43.878353664Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-21T16:12:44.009604723Z error: the server doesn't have a resource type "clusters"
2026-05-21T16:12:44.165407907Z Caches written to disk