2026-05-26T16:44:15.343959758Z [disk usage checker] Started
2026-05-26T16:44:15.347822323Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-05-26T16:44:15.582103456Z Gathering data for ns/openshift-cluster-version...
2026-05-26T16:44:15.750941181Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-26T16:44:15.854236614Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-26T16:44:15.963146848Z Gathering data for ns/default...
2026-05-26T16:44:16.326329956Z Gathering data for ns/openshift...
2026-05-26T16:44:16.800981946Z Gathering data for ns/kube-system...
2026-05-26T16:44:17.599269430Z Gathering data for ns/openshift-etcd...
2026-05-26T16:44:17.891098281Z Waiting on subprocesses to finish execution.
2026-05-26T16:44:17.893797204Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-26T16:44:17.899243676Z INFO: Gathering HAProxy config files
2026-05-26T16:44:17.909366191Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-26T16:44:17.911558424Z INFO: Collecting host service logs for crio
2026-05-26T16:44:17.911915231Z INFO: Collecting host service logs for kubelet
2026-05-26T16:44:17.912309584Z INFO: Collecting host service logs for rpm-ostreed
2026-05-26T16:44:17.912632517Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-26T16:44:17.912968245Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-26T16:44:17.913279433Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-26T16:44:17.913623226Z INFO: Collecting host service logs for NetworkManager
2026-05-26T16:44:17.913951656Z INFO: Collecting host service logs for openvswitch
2026-05-26T16:44:17.914273063Z INFO: Collecting host service logs for ovs-configuration
2026-05-26T16:44:17.914618391Z INFO: Collecting host service logs for ovsdb-server
2026-05-26T16:44:17.914967194Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-26T16:44:17.915353395Z INFO: Waiting for worker host service log collection to complete ...
2026-05-26T16:44:17.948205356Z INFO: Waiting for node performance related collection to complete ...
2026-05-26T16:44:17.978091476Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-26T16:44:18.339775605Z Wrote inspect data to must-gather.
2026-05-26T16:44:18.339813670Z error: inspection completed with the errors occurred while gathering data:
2026-05-26T16:44:18.339813670Z     namespaces "assisted-installer" not found
2026-05-26T16:44:18.667928392Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-26T16:44:18.883252910Z No resources found
2026-05-26T16:44:18.963615375Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-q64z9
2026-05-26T16:44:18.963730081Z insights-runtime-extractor-6zn9f
2026-05-26T16:44:18.963752965Z insights-runtime-extractor-mrvlm
2026-05-26T16:44:18.963772188Z insights-runtime-extractor-w6ml6
2026-05-26T16:44:19.020736537Z INFO: "metallb-operator" not detected. Skipping.
2026-05-26T16:44:19.199123286Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-26T16:44:19.276460758Z Wrote inspect data to must-gather.
2026-05-26T16:44:19.280034197Z No resources found in openshift-etcd namespace.
2026-05-26T16:44:19.302915869Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-26T16:44:19.305393512Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-26T16:44:19.332570517Z INFO: Worker host service log collection to complete.
2026-05-26T16:44:19.352409708Z error: the server doesn't have a resource type "performanceprofile"
2026-05-26T16:44:19.355871633Z ERROR: No running kube-apiserver pods found
2026-05-26T16:44:19.433743060Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-26T16:44:19.433786593Z INFO: on-disk MachineConfig config collection complete.
2026-05-26T16:44:19.458160245Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-26T16:44:19.995874546Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-26T16:44:19.995874546Z See 'oc rsync -h' for help and examples
2026-05-26T16:44:20.048854635Z INFO: 'previous-logs' folder not found on ip-10-0-132-148.ec2.internal, skipping...
2026-05-26T16:44:20.158490532Z Wrote inspect data to must-gather.
2026-05-26T16:44:20.337944482Z tar: Removing leading `/' from member names
2026-05-26T16:44:20.368794739Z INFO: HAProxy config collection complete.
2026-05-26T16:44:20.399452041Z No resources found
2026-05-26T16:44:20.486946441Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-05-26T16:44:20.541162705Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-26T16:44:20.578745122Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-26T16:44:20.599464446Z Wrote inspect data to must-gather.
2026-05-26T16:44:20.624705033Z Wrote inspect data to must-gather.
2026-05-26T16:44:20.627820462Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-26T16:44:20.638001874Z Wrote inspect data to must-gather.
2026-05-26T16:44:20.727632251Z Wrote inspect data to must-gather.
2026-05-26T16:44:20.758103555Z INFO: 'previous-logs' folder not found on ip-10-0-133-107.ec2.internal, skipping...
2026-05-26T16:44:20.967272736Z error: the server doesn't have a resource type "machineconfigs"
2026-05-26T16:44:21.024872934Z Gathering data for ns/kuadrant-system...
2026-05-26T16:44:21.043681236Z Wrote inspect data to must-gather.
2026-05-26T16:44:21.110544775Z INFO: Getting rules from prometheus-k8s-0
2026-05-26T16:44:21.113957441Z INFO: 'previous-logs' folder not found on ip-10-0-142-102.ec2.internal, skipping...
2026-05-26T16:44:21.113998560Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-26T16:44:21.114029322Z INFO: Machine Config Daemon termination log collection complete.
2026-05-26T16:44:21.276495369Z Wrote inspect data to must-gather.
2026-05-26T16:44:21.398798424Z Wrote inspect data to must-gather.
2026-05-26T16:44:21.553014904Z Wrote inspect data to must-gather.
2026-05-26T16:44:21.618194836Z INFO: Getting status/config from prometheus-k8s-0
2026-05-26T16:44:21.771100391Z Wrote inspect data to must-gather.
2026-05-26T16:44:21.781598699Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-26T16:44:22.154037713Z Wrote inspect data to must-gather.
2026-05-26T16:44:22.193922442Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-26T16:44:22.195728361Z INFO: INTERCONNECT MODE
2026-05-26T16:44:22.195758239Z INFO: Gathering ovn-kubernetes DBs
2026-05-26T16:44:22.234741434Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-26T16:44:22.398673035Z INFO: Gathering OVN_Northbound from ovnkube-node-6pnf2...
2026-05-26T16:44:22.399014122Z INFO: Gathering OVN_Northbound from ovnkube-node-7xvh4...
2026-05-26T16:44:22.399275977Z INFO: Gathering OVN_Northbound from ovnkube-node-hbstd...
2026-05-26T16:44:22.399648870Z INFO: Gathering OVN_Southbound from ovnkube-node-6pnf2...
2026-05-26T16:44:22.400026048Z INFO: Gathering OVN_Southbound from ovnkube-node-7xvh4...
2026-05-26T16:44:22.400495633Z INFO: Gathering OVN_Southbound from ovnkube-node-hbstd...
2026-05-26T16:44:22.540334367Z Wrote inspect data to must-gather.
2026-05-26T16:44:22.548918352Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-26T16:44:22.783961786Z tar: Removing leading `/' from member names
2026-05-26T16:44:22.805641453Z tar: Removing leading `/' from member names
2026-05-26T16:44:22.813783055Z tar: Removing leading `/' from member names
2026-05-26T16:44:22.850747833Z tar: Removing leading `/' from member names
2026-05-26T16:44:22.858845588Z tar: Removing leading `/' from member names
2026-05-26T16:44:22.893391447Z tar: Removing leading `/' from member names
2026-05-26T16:44:22.953891735Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-26T16:44:23.042233261Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-26T16:44:23.156547141Z Wrote inspect data to must-gather.
2026-05-26T16:44:23.243724463Z Gathering data for ns/openshift-monitoring...
2026-05-26T16:44:23.353254847Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-26T16:44:23.587667480Z Wrote inspect data to must-gather.
2026-05-26T16:44:23.695559282Z INFO: Waiting for network log collection to complete ...
2026-05-26T16:44:23.695678077Z INFO: Waiting for ovnk database copies to complete ...
2026-05-26T16:44:23.696181643Z INFO: Copying ovnk databases complete.
2026-05-26T16:44:23.698377158Z 4.2M	must-gather/network_logs/ovnk_database_store
2026-05-26T16:44:23.701197252Z ovnk_database_store/
2026-05-26T16:44:23.701197252Z ovnk_database_store/ovnkube-node-hbstd_nbdb
2026-05-26T16:44:23.748384727Z ovnk_database_store/ovnkube-node-hbstd_sbdb
2026-05-26T16:44:23.778866800Z ovnk_database_store/ovnkube-node-6pnf2_sbdb
2026-05-26T16:44:23.811616964Z ovnk_database_store/ovnkube-node-7xvh4_sbdb
2026-05-26T16:44:23.838004855Z ovnk_database_store/ovnkube-node-6pnf2_nbdb
2026-05-26T16:44:23.851999239Z ovnk_database_store/ovnkube-node-7xvh4_nbdb
2026-05-26T16:44:23.862056017Z INFO: Getting status from alertmanager-main-0
2026-05-26T16:44:23.865973061Z INFO: Network log collection complete.
2026-05-26T16:44:23.993669953Z Wrote inspect data to must-gather.
2026-05-26T16:44:24.279864053Z Wrote inspect data to must-gather.
2026-05-26T16:44:24.548210891Z Wrote inspect data to must-gather.
2026-05-26T16:44:24.822157839Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-26T16:44:24.824748346Z INFO: Fallback to identify the container image from release info
2026-05-26T16:44:25.513727200Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-05-26T16:44:25.551768319Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda
2026-05-26T16:44:25.708194820Z daemonset.apps/perf-node-gather-daemonset created
2026-05-26T16:44:25.754016697Z Gathering data for ns/openshift-network-console...
2026-05-26T16:44:25.878010748Z Waiting for performance profile collector pods to become ready: 1
2026-05-26T16:44:26.345343165Z Gathering data for ns/openshift-console-operator...
2026-05-26T16:44:26.818907764Z Gathering data for ns/openshift-console...
2026-05-26T16:44:27.038010703Z Waiting for performance profile collector pods to become ready: 2
2026-05-26T16:44:27.389161962Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-26T16:44:27.460839538Z Wrote inspect data to must-gather.
2026-05-26T16:44:27.804710284Z Gathering data for ns/openshift-dns-operator...
2026-05-26T16:44:28.170379238Z Waiting for performance profile collector pods to become ready: 3
2026-05-26T16:44:28.178735942Z Gathering data for ns/openshift-dns...
2026-05-26T16:44:28.824566093Z Gathering data for ns/openshift-image-registry...
2026-05-26T16:44:29.295023780Z Waiting for performance profile collector pods to become ready: 4
2026-05-26T16:44:29.356806445Z Gathering data for ns/openshift-ingress-operator...
2026-05-26T16:44:29.780146285Z Gathering data for ns/openshift-ingress...
2026-05-26T16:44:30.422177627Z Waiting for performance profile collector pods to become ready: 5
2026-05-26T16:44:30.518181341Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-05-26T16:44:30.641275381Z Gathering data for ns/openshift-ingress-canary...
2026-05-26T16:44:31.114391853Z Gathering data for ns/openshift-insights...
2026-05-26T16:44:31.531478109Z Waiting for performance profile collector pods to become ready: 6
2026-05-26T16:44:32.652142063Z Waiting for performance profile collector pods to become ready: 7
2026-05-26T16:44:33.203530493Z Gathering data for ns/opendatahub...
2026-05-26T16:44:33.766019843Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-26T16:44:34.024366384Z Collecting performance related data for node ip-10-0-132-148.ec2.internal
2026-05-26T16:44:34.035066986Z Collecting performance related data for node ip-10-0-133-107.ec2.internal
2026-05-26T16:44:34.044357231Z Collecting performance related data for node ip-10-0-142-102.ec2.internal
2026-05-26T16:44:34.827502507Z Gathering data for ns/openshift-lws-operator...
2026-05-26T16:44:35.524725212Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-05-26T16:44:36.234955330Z Gathering data for ns/openshift-config...
2026-05-26T16:44:36.604863754Z Gathering data for ns/openshift-config-managed...
2026-05-26T16:44:37.126557915Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-26T16:44:37.461245324Z Gathering data for ns/openshift-kube-apiserver...
2026-05-26T16:44:37.767723591Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-26T16:44:38.121458857Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-26T16:44:38.486185041Z Gathering data for ns/openshift-kube-scheduler...
2026-05-26T16:44:38.806844647Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-26T16:44:39.159998230Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-26T16:44:39.545208309Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-26T16:44:39.908628953Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-26T16:44:40.515458793Z Gathering data for ns/openshift-multus...
2026-05-26T16:44:40.529717086Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-05-26T16:44:42.059628878Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-26T16:44:42.433420844Z Collecting kubelet logs for node ip-10-0-132-148.ec2.internal
2026-05-26T16:44:42.435123748Z Collecting kubelet logs for node ip-10-0-133-107.ec2.internal
2026-05-26T16:44:42.436798322Z Collecting kubelet logs for node ip-10-0-142-102.ec2.internal
2026-05-26T16:44:42.780579965Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-26T16:44:42.785920805Z INFO: Node performance data collection complete.
2026-05-26T16:44:43.617761885Z Gathering data for ns/openshift-host-network...
2026-05-26T16:44:44.003581940Z Gathering data for ns/openshift-network-diagnostics...
2026-05-26T16:44:44.633679383Z Gathering data for ns/openshift-network-node-identity...
2026-05-26T16:44:45.097834314Z Gathering data for ns/openshift-network-operator...
2026-05-26T16:44:45.521823881Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-26T16:44:45.536314262Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-05-26T16:44:45.844742365Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-26T16:44:46.384713020Z Gathering data for ns/openshift-apiserver-operator...
2026-05-26T16:44:46.719112422Z Gathering data for ns/openshift-apiserver...
2026-05-26T16:44:47.179450195Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-26T16:44:47.510595863Z Gathering data for ns/openshift-controller-manager...
2026-05-26T16:44:47.828772738Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-26T16:44:48.542325758Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-26T16:44:48.875576843Z Gathering data for ns/openshift-service-ca-operator...
2026-05-26T16:44:49.224837811Z Gathering data for ns/openshift-service-ca...
2026-05-26T16:44:49.637397709Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-26T16:44:50.540906857Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-05-26T16:44:50.665949682Z Wrote inspect data to must-gather.
2026-05-26T16:44:50.665996262Z error: inspection completed with the errors occurred while gathering data:
2026-05-26T16:44:50.665996262Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-26T16:44:50.805258447Z error: the server doesn't have a resource type "clusters"
2026-05-26T16:44:50.993863461Z Caches written to disk