2026-04-16T23:19:13.816148910Z [disk usage checker] Started
2026-04-16T23:19:13.850255203Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-16T23:19:14.015040049Z Gathering data for ns/openshift-cluster-version...
2026-04-16T23:19:14.139350862Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T23:19:14.228187843Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T23:19:14.301338839Z Gathering data for ns/default...
2026-04-16T23:19:14.615429766Z Gathering data for ns/openshift...
2026-04-16T23:19:15.000472505Z Gathering data for ns/kube-system...
2026-04-16T23:19:15.671546920Z Gathering data for ns/openshift-etcd...
2026-04-16T23:19:15.952500892Z Wrote inspect data to must-gather.
2026-04-16T23:19:15.952532409Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T23:19:15.952532409Z     namespaces "assisted-installer" not found
2026-04-16T23:19:16.195237468Z Waiting on subprocesses to finish execution.
2026-04-16T23:19:16.202830512Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-16T23:19:16.206480899Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-16T23:19:16.207189105Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-16T23:19:16.211859529Z INFO: Gathering HAProxy config files
2026-04-16T23:19:16.225824800Z INFO: Collecting host service logs for crio
2026-04-16T23:19:16.226154652Z INFO: Collecting host service logs for kubelet
2026-04-16T23:19:16.226583946Z INFO: Collecting host service logs for rpm-ostreed
2026-04-16T23:19:16.226908312Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-16T23:19:16.227224881Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-16T23:19:16.227519596Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-16T23:19:16.227873124Z INFO: Collecting host service logs for NetworkManager
2026-04-16T23:19:16.228172418Z INFO: Collecting host service logs for openvswitch
2026-04-16T23:19:16.228467930Z INFO: Collecting host service logs for ovs-configuration
2026-04-16T23:19:16.228792061Z INFO: Collecting host service logs for ovsdb-server
2026-04-16T23:19:16.229088387Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-16T23:19:16.229542304Z INFO: Waiting for worker host service log collection to complete ...
2026-04-16T23:19:16.312963685Z INFO: Waiting for node performance related collection to complete ...
2026-04-16T23:19:16.547321075Z No resources found
2026-04-16T23:19:16.837831613Z No resources found in openshift-etcd namespace.
2026-04-16T23:19:16.909545448Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-16T23:19:17.013594171Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-16T23:19:17.013702775Z INFO: on-disk MachineConfig config collection complete.
2026-04-16T23:19:17.081903352Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-87gkq
2026-04-16T23:19:17.081997054Z insights-runtime-extractor-8wkch
2026-04-16T23:19:17.082019656Z insights-runtime-extractor-t9crn
2026-04-16T23:19:17.082039353Z insights-runtime-extractor-xwqzs
2026-04-16T23:19:17.150548548Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-16T23:19:17.367425455Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-16T23:19:17.461774830Z error: the server doesn't have a resource type "performanceprofile"
2026-04-16T23:19:17.582283305Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-16T23:19:17.582283305Z See 'oc rsync -h' for help and examples
2026-04-16T23:19:17.683615447Z Wrote inspect data to must-gather.
2026-04-16T23:19:17.739438284Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-16T23:19:17.882947617Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-16T23:19:17.961037529Z Wrote inspect data to must-gather.
2026-04-16T23:19:18.083764804Z Wrote inspect data to must-gather.
2026-04-16T23:19:18.109883847Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T23:19:18.178994017Z INFO: "metallb-operator" not detected. Skipping.
2026-04-16T23:19:18.204169381Z No resources found
2026-04-16T23:19:18.234170845Z INFO: 'previous-logs' folder not found on ip-10-0-130-16.ec2.internal, skipping...
2026-04-16T23:19:18.306664411Z INFO: Worker host service log collection to complete.
2026-04-16T23:19:18.324016681Z Wrote inspect data to must-gather.
2026-04-16T23:19:18.327474545Z Wrote inspect data to must-gather.
2026-04-16T23:19:18.428923559Z Wrote inspect data to must-gather.
2026-04-16T23:19:18.430538845Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-16T23:19:18.433638541Z ERROR: No running kube-apiserver pods found
2026-04-16T23:19:18.463190687Z Wrote inspect data to must-gather.
2026-04-16T23:19:18.476062768Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-16T23:19:18.681981629Z error: the server doesn't have a resource type "machineconfigs"
2026-04-16T23:19:18.690467080Z tar: Removing leading `/' from member names
2026-04-16T23:19:18.696759924Z INFO: HAProxy config collection complete.
2026-04-16T23:19:18.705475812Z INFO: Getting rules from prometheus-k8s-0
2026-04-16T23:19:18.736407652Z INFO: 'previous-logs' folder not found on ip-10-0-133-183.ec2.internal, skipping...
2026-04-16T23:19:18.757175708Z Wrote inspect data to must-gather.
2026-04-16T23:19:18.860845363Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-16T23:19:18.920570889Z Wrote inspect data to must-gather.
2026-04-16T23:19:18.961449590Z Gathering data for ns/openshift-monitoring...
2026-04-16T23:19:19.004577117Z Wrote inspect data to must-gather.
2026-04-16T23:19:19.021429271Z INFO: 'previous-logs' folder not found on ip-10-0-136-39.ec2.internal, skipping...
2026-04-16T23:19:19.021532982Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-16T23:19:19.021583055Z INFO: Machine Config Daemon termination log collection complete.
2026-04-16T23:19:19.156882869Z Wrote inspect data to must-gather.
2026-04-16T23:19:19.163998701Z INFO: Getting status/config from prometheus-k8s-0
2026-04-16T23:19:19.254016030Z Wrote inspect data to must-gather.
2026-04-16T23:19:19.311648621Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-16T23:19:19.516790215Z Wrote inspect data to must-gather.
2026-04-16T23:19:19.546088467Z INFO: INTERCONNECT MODE
2026-04-16T23:19:19.546198871Z INFO: Gathering ovn-kubernetes DBs
2026-04-16T23:19:19.555520452Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-16T23:19:19.710720482Z INFO: Gathering OVN_Northbound from ovnkube-node-88hfq...
2026-04-16T23:19:19.710997904Z INFO: Gathering OVN_Northbound from ovnkube-node-l8m6g...
2026-04-16T23:19:19.711243352Z INFO: Gathering OVN_Northbound from ovnkube-node-z4npz...
2026-04-16T23:19:19.711504330Z INFO: Gathering OVN_Southbound from ovnkube-node-88hfq...
2026-04-16T23:19:19.711748020Z INFO: Gathering OVN_Southbound from ovnkube-node-l8m6g...
2026-04-16T23:19:19.711989434Z INFO: Gathering OVN_Southbound from ovnkube-node-z4npz...
2026-04-16T23:19:19.730426169Z Wrote inspect data to must-gather.
2026-04-16T23:19:19.832399146Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-16T23:19:19.976495624Z tar: Removing leading `/' from member names
2026-04-16T23:19:19.983332223Z tar: Removing leading `/' from member names
2026-04-16T23:19:20.000282525Z tar: Removing leading `/' from member names
2026-04-16T23:19:20.000938630Z tar: Removing leading `/' from member names
2026-04-16T23:19:20.017244987Z tar: Removing leading `/' from member names
2026-04-16T23:19:20.021867299Z tar: Removing leading `/' from member names
2026-04-16T23:19:20.057053318Z Wrote inspect data to must-gather.
2026-04-16T23:19:20.134689350Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-16T23:19:20.293938523Z Wrote inspect data to must-gather.
2026-04-16T23:19:20.384320946Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-16T23:19:20.493081962Z INFO: Waiting for network log collection to complete ...
2026-04-16T23:19:20.493202237Z INFO: Waiting for ovnk database copies to complete ...
2026-04-16T23:19:20.493768275Z INFO: Copying ovnk databases complete.
2026-04-16T23:19:20.495673328Z 4.3M	must-gather/network_logs/ovnk_database_store
2026-04-16T23:19:20.497816264Z ovnk_database_store/
2026-04-16T23:19:20.497816264Z ovnk_database_store/ovnkube-node-l8m6g_sbdb
2026-04-16T23:19:20.519802040Z Wrote inspect data to must-gather.
2026-04-16T23:19:20.524664941Z ovnk_database_store/ovnkube-node-88hfq_sbdb
2026-04-16T23:19:20.541370300Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T23:19:20.549202871Z ovnk_database_store/ovnkube-node-z4npz_nbdb
2026-04-16T23:19:20.561259704Z ovnk_database_store/ovnkube-node-z4npz_sbdb
2026-04-16T23:19:20.588890682Z ovnk_database_store/ovnkube-node-l8m6g_nbdb
2026-04-16T23:19:20.601529466Z ovnk_database_store/ovnkube-node-88hfq_nbdb
2026-04-16T23:19:20.612121110Z INFO: Network log collection complete.
2026-04-16T23:19:20.733881057Z Wrote inspect data to must-gather.
2026-04-16T23:19:20.771898674Z INFO: Getting status from alertmanager-main-0
2026-04-16T23:19:20.929097717Z Wrote inspect data to must-gather.
2026-04-16T23:19:21.051049027Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T23:19:21.183915194Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-16T23:19:21.185948583Z INFO: Fallback to identify the container image from release info
2026-04-16T23:19:21.204319574Z Gathering data for ns/openshift-network-console...
2026-04-16T23:19:21.560845853Z Gathering data for ns/openshift-console-operator...
2026-04-16T23:19:21.786600572Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-16T23:19:21.908290175Z Gathering data for ns/openshift-console...
2026-04-16T23:19:21.916055299Z daemonset.apps/perf-node-gather-daemonset created
2026-04-16T23:19:22.045621915Z Waiting for performance profile collector pods to become ready: 1
2026-04-16T23:19:22.365035777Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-16T23:19:22.710217523Z Gathering data for ns/openshift-dns-operator...
2026-04-16T23:19:23.008867146Z Gathering data for ns/openshift-dns...
2026-04-16T23:19:23.157237079Z Waiting for performance profile collector pods to become ready: 2
2026-04-16T23:19:23.557621573Z Gathering data for ns/openshift-image-registry...
2026-04-16T23:19:23.865428407Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-16T23:19:23.978296409Z Gathering data for ns/openshift-ingress-operator...
2026-04-16T23:19:24.287425041Z Waiting for performance profile collector pods to become ready: 3
2026-04-16T23:19:24.330430980Z Gathering data for ns/openshift-ingress...
2026-04-16T23:19:24.639171329Z Gathering data for ns/openshift-ingress-canary...
2026-04-16T23:19:24.987969025Z Gathering data for ns/openshift-insights...
2026-04-16T23:19:25.407603192Z Waiting for performance profile collector pods to become ready: 4
2026-04-16T23:19:26.523216721Z Waiting for performance profile collector pods to become ready: 5
2026-04-16T23:19:26.973597147Z Gathering data for ns/kserve...
2026-04-16T23:19:27.655344425Z Waiting for performance profile collector pods to become ready: 6
2026-04-16T23:19:28.714339935Z Gathering data for ns/openshift-config...
2026-04-16T23:19:28.788259123Z Waiting for performance profile collector pods to become ready: 7
2026-04-16T23:19:28.869847415Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-16T23:19:29.008232099Z Gathering data for ns/openshift-config-managed...
2026-04-16T23:19:29.449391979Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-16T23:19:29.739473413Z Gathering data for ns/openshift-kube-apiserver...
2026-04-16T23:19:29.898844044Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-16T23:19:30.034037382Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-16T23:19:30.169982353Z Collecting performance related data for node ip-10-0-130-16.ec2.internal
2026-04-16T23:19:30.183151630Z Collecting performance related data for node ip-10-0-136-39.ec2.internal
2026-04-16T23:19:30.187684874Z Collecting performance related data for node ip-10-0-133-183.ec2.internal
2026-04-16T23:19:30.379984382Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-16T23:19:30.698778464Z Gathering data for ns/openshift-kube-scheduler...
2026-04-16T23:19:31.030302564Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-16T23:19:31.377129720Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-16T23:19:31.762482974Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-16T23:19:32.069282183Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-16T23:19:32.564603944Z Gathering data for ns/openshift-multus...
2026-04-16T23:19:33.875411929Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-16T23:19:33.958241074Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-16T23:19:35.351392757Z Gathering data for ns/openshift-host-network...
2026-04-16T23:19:35.679468599Z Gathering data for ns/openshift-network-diagnostics...
2026-04-16T23:19:36.174043057Z Gathering data for ns/openshift-network-node-identity...
2026-04-16T23:19:36.538699735Z Gathering data for ns/openshift-network-operator...
2026-04-16T23:19:36.883486817Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-16T23:19:37.127452012Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-16T23:19:37.573001228Z Gathering data for ns/openshift-apiserver-operator...
2026-04-16T23:19:37.845264816Z Gathering data for ns/openshift-apiserver...
2026-04-16T23:19:38.265243235Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-16T23:19:38.553935722Z Collecting kubelet logs for node ip-10-0-130-16.ec2.internal
2026-04-16T23:19:38.556274434Z Collecting kubelet logs for node ip-10-0-136-39.ec2.internal
2026-04-16T23:19:38.558467338Z Collecting kubelet logs for node ip-10-0-133-183.ec2.internal
2026-04-16T23:19:38.583403984Z Gathering data for ns/openshift-controller-manager...
2026-04-16T23:19:38.881141422Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-16T23:19:38.900678308Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-16T23:19:38.970138826Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-16T23:19:38.974528394Z INFO: Node performance data collection complete.
2026-04-16T23:19:39.541040144Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-16T23:19:39.806359164Z Gathering data for ns/openshift-service-ca-operator...
2026-04-16T23:19:40.140209042Z Gathering data for ns/openshift-service-ca...
2026-04-16T23:19:40.548147992Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-16T23:19:41.487309822Z Wrote inspect data to must-gather.
2026-04-16T23:19:41.487352359Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T23:19:41.487352359Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-16T23:19:41.613886654Z error: the server doesn't have a resource type "clusters"
2026-04-16T23:19:41.753869799Z Caches written to disk