2026-05-21T18:56:22.836407182Z [disk usage checker] Started
2026-05-21T18:56:22.839699858Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-21T18:56:23.078136450Z Gathering data for ns/openshift-cluster-version...
2026-05-21T18:56:23.326771825Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-21T18:56:23.467047462Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-21T18:56:23.615125674Z Gathering data for ns/default...
2026-05-21T18:56:24.097175923Z Gathering data for ns/openshift...
2026-05-21T18:56:24.676949373Z Gathering data for ns/kube-system...
2026-05-21T18:56:25.210942140Z Waiting on subprocesses to finish execution.
2026-05-21T18:56:25.218970638Z INFO: Collecting host service logs for crio
2026-05-21T18:56:25.219280909Z INFO: Collecting host service logs for kubelet
2026-05-21T18:56:25.219641151Z INFO: Collecting host service logs for rpm-ostreed
2026-05-21T18:56:25.219965855Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-21T18:56:25.219982350Z INFO: Gathering HAProxy config files
2026-05-21T18:56:25.220314996Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-21T18:56:25.220648366Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-21T18:56:25.220977655Z INFO: Collecting host service logs for NetworkManager
2026-05-21T18:56:25.221296626Z INFO: Collecting host service logs for openvswitch
2026-05-21T18:56:25.221677375Z INFO: Collecting host service logs for ovs-configuration
2026-05-21T18:56:25.222018601Z INFO: Collecting host service logs for ovsdb-server
2026-05-21T18:56:25.222362672Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-21T18:56:25.222759666Z INFO: Waiting for worker host service log collection to complete ...
2026-05-21T18:56:25.265218846Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-21T18:56:25.309139977Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-21T18:56:25.419030877Z INFO: Waiting for node performance related collection to complete ...
2026-05-21T18:56:25.481826609Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-21T18:56:25.610458337Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-21T18:56:25.857869846Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-21T18:56:26.265964954Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-21T18:56:26.487529105Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-21T18:56:26.520559274Z INFO: "metallb-operator" not detected. Skipping.
2026-05-21T18:56:26.536444289Z Wrote inspect data to must-gather.
2026-05-21T18:56:26.557227068Z No resources found in openshift-etcd namespace.
2026-05-21T18:56:26.668376169Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-21T18:56:26.668474542Z INFO: on-disk MachineConfig config collection complete.
2026-05-21T18:56:26.698554113Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-d9jkq
2026-05-21T18:56:26.698607732Z insights-runtime-extractor-k4j68
2026-05-21T18:56:26.698628669Z insights-runtime-extractor-mntgl
2026-05-21T18:56:26.698649054Z insights-runtime-extractor-tcspb
2026-05-21T18:56:26.740600087Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-21T18:56:26.839698285Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-21T18:56:26.933952283Z INFO: Worker host service log collection to complete.
2026-05-21T18:56:26.968481228Z error: the server doesn't have a resource type "performanceprofile"
2026-05-21T18:56:27.027858080Z No resources found
2026-05-21T18:56:27.223992061Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-21T18:56:27.223992061Z See 'oc rsync -h' for help and examples
2026-05-21T18:56:27.428341041Z Gathering data for ns/openshift-etcd...
2026-05-21T18:56:27.486118043Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-21T18:56:27.516469727Z Wrote inspect data to must-gather.
2026-05-21T18:56:27.802238563Z Wrote inspect data to must-gather.
2026-05-21T18:56:27.850802451Z Wrote inspect data to must-gather.
2026-05-21T18:56:27.980762172Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-21T18:56:28.004906657Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-21T18:56:28.062424565Z tar: Removing leading `/' from member names
2026-05-21T18:56:28.065702803Z No resources found
2026-05-21T18:56:28.071082130Z INFO: HAProxy config collection complete.
2026-05-21T18:56:28.104449556Z INFO: 'previous-logs' folder not found on ip-10-0-129-93.ec2.internal, skipping...
2026-05-21T18:56:28.202547698Z ERROR: No running kube-apiserver pods found
2026-05-21T18:56:28.265547465Z Wrote inspect data to must-gather.
2026-05-21T18:56:28.314422496Z INFO: Getting rules from prometheus-k8s-0
2026-05-21T18:56:28.319214866Z Wrote inspect data to must-gather.
2026-05-21T18:56:28.439756640Z Wrote inspect data to must-gather.
2026-05-21T18:56:28.587972205Z INFO: 'previous-logs' folder not found on ip-10-0-138-250.ec2.internal, skipping...
2026-05-21T18:56:28.595591700Z Gathering data for ns/kuadrant-system...
2026-05-21T18:56:28.603594688Z error: the server doesn't have a resource type "machineconfigs"
2026-05-21T18:56:28.826218474Z Wrote inspect data to must-gather.
2026-05-21T18:56:28.826248546Z error: inspection completed with the errors occurred while gathering data:
2026-05-21T18:56:28.826248546Z     namespaces "assisted-installer" not found
2026-05-21T18:56:28.907486695Z Wrote inspect data to must-gather.
2026-05-21T18:56:28.950535188Z INFO: 'previous-logs' folder not found on ip-10-0-142-242.ec2.internal, skipping...
2026-05-21T18:56:28.950641287Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-21T18:56:28.950694268Z INFO: Machine Config Daemon termination log collection complete.
2026-05-21T18:56:28.984230895Z Wrote inspect data to must-gather.
2026-05-21T18:56:29.018406883Z INFO: Getting status/config from prometheus-k8s-0
2026-05-21T18:56:29.220018276Z Wrote inspect data to must-gather.
2026-05-21T18:56:29.272763096Z Wrote inspect data to must-gather.
2026-05-21T18:56:29.464988497Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-21T18:56:29.577667245Z INFO: INTERCONNECT MODE
2026-05-21T18:56:29.577703749Z INFO: Gathering ovn-kubernetes DBs
2026-05-21T18:56:29.632533152Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-21T18:56:29.779067977Z Wrote inspect data to must-gather.
2026-05-21T18:56:29.796413405Z INFO: Gathering OVN_Northbound from ovnkube-node-9qv8k...
2026-05-21T18:56:29.796670666Z INFO: Gathering OVN_Northbound from ovnkube-node-n4lmv...
2026-05-21T18:56:29.796937228Z INFO: Gathering OVN_Northbound from ovnkube-node-pfjxx...
2026-05-21T18:56:29.797208034Z INFO: Gathering OVN_Southbound from ovnkube-node-9qv8k...
2026-05-21T18:56:29.797465591Z INFO: Gathering OVN_Southbound from ovnkube-node-n4lmv...
2026-05-21T18:56:29.797700081Z INFO: Gathering OVN_Southbound from ovnkube-node-pfjxx...
2026-05-21T18:56:30.021297057Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-21T18:56:30.246203348Z tar: Removing leading `/' from member names
2026-05-21T18:56:30.253488035Z tar: Removing leading `/' from member names
2026-05-21T18:56:30.298843487Z tar: Removing leading `/' from member names
2026-05-21T18:56:30.310591264Z tar: Removing leading `/' from member names
2026-05-21T18:56:30.310854321Z tar: Removing leading `/' from member names
2026-05-21T18:56:30.350223391Z tar: Removing leading `/' from member names
2026-05-21T18:56:30.377876219Z Wrote inspect data to must-gather.
2026-05-21T18:56:30.503187887Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-21T18:56:30.532342383Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-21T18:56:30.704002496Z Wrote inspect data to must-gather.
2026-05-21T18:56:30.829137825Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-21T18:56:30.909386159Z INFO: Waiting for network log collection to complete ...
2026-05-21T18:56:30.909422380Z INFO: Waiting for ovnk database copies to complete ...
2026-05-21T18:56:30.909800507Z INFO: Copying ovnk databases complete.
2026-05-21T18:56:30.911572724Z 4.2M	must-gather/network_logs/ovnk_database_store
2026-05-21T18:56:30.917150355Z ovnk_database_store/
2026-05-21T18:56:30.917203968Z ovnk_database_store/ovnkube-node-9qv8k_nbdb
2026-05-21T18:56:30.931483712Z ovnk_database_store/ovnkube-node-n4lmv_nbdb
2026-05-21T18:56:30.943123164Z ovnk_database_store/ovnkube-node-n4lmv_sbdb
2026-05-21T18:56:30.969205410Z ovnk_database_store/ovnkube-node-pfjxx_sbdb
2026-05-21T18:56:31.000823281Z ovnk_database_store/ovnkube-node-pfjxx_nbdb
2026-05-21T18:56:31.013852068Z ovnk_database_store/ovnkube-node-9qv8k_sbdb
2026-05-21T18:56:31.015874094Z Wrote inspect data to must-gather.
2026-05-21T18:56:31.043428419Z INFO: Network log collection complete.
2026-05-21T18:56:31.113019883Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-21T18:56:31.349627153Z Gathering data for ns/openshift-monitoring...
2026-05-21T18:56:31.368989625Z INFO: Getting status from alertmanager-main-0
2026-05-21T18:56:31.390008611Z Wrote inspect data to must-gather.
2026-05-21T18:56:31.703915936Z Wrote inspect data to must-gather.
2026-05-21T18:56:31.979958065Z Wrote inspect data to must-gather.
2026-05-21T18:56:32.239608944Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-21T18:56:32.241846182Z INFO: Fallback to identify the container image from release info
2026-05-21T18:56:32.728387500Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda
2026-05-21T18:56:32.863124232Z daemonset.apps/perf-node-gather-daemonset created
2026-05-21T18:56:32.988231320Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-21T18:56:33.017908718Z Waiting for performance profile collector pods to become ready: 1
2026-05-21T18:56:33.869401198Z Gathering data for ns/openshift-network-console...
2026-05-21T18:56:34.186228741Z Waiting for performance profile collector pods to become ready: 2
2026-05-21T18:56:34.492481205Z Wrote inspect data to must-gather.
2026-05-21T18:56:34.588478161Z Gathering data for ns/openshift-console-operator...
2026-05-21T18:56:35.081963286Z Gathering data for ns/openshift-console...
2026-05-21T18:56:35.320243725Z Waiting for performance profile collector pods to become ready: 3
2026-05-21T18:56:35.658317745Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-21T18:56:36.153313654Z Gathering data for ns/openshift-dns-operator...
2026-05-21T18:56:36.435425900Z Waiting for performance profile collector pods to become ready: 4
2026-05-21T18:56:36.586033469Z Gathering data for ns/openshift-dns...
2026-05-21T18:56:37.446610420Z Gathering data for ns/openshift-image-registry...
2026-05-21T18:56:37.548179276Z Waiting for performance profile collector pods to become ready: 5
2026-05-21T18:56:37.995527068Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-21T18:56:38.275139406Z Gathering data for ns/openshift-ingress-operator...
2026-05-21T18:56:38.676254925Z Waiting for performance profile collector pods to become ready: 6
2026-05-21T18:56:38.742400624Z Gathering data for ns/openshift-ingress...
2026-05-21T18:56:39.773240418Z Gathering data for ns/openshift-ingress-canary...
2026-05-21T18:56:39.796657157Z Waiting for performance profile collector pods to become ready: 7
2026-05-21T18:56:40.630670695Z Gathering data for ns/openshift-insights...
2026-05-21T18:56:40.906877842Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-21T18:56:41.125595623Z Collecting performance related data for node ip-10-0-142-242.ec2.internal
2026-05-21T18:56:41.131981271Z Collecting performance related data for node ip-10-0-129-93.ec2.internal
2026-05-21T18:56:41.139195122Z Collecting performance related data for node ip-10-0-138-250.ec2.internal
2026-05-21T18:56:43.000023178Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-21T18:56:43.714165000Z Gathering data for ns/opendatahub...
2026-05-21T18:56:45.690591129Z Gathering data for ns/openshift-lws-operator...
2026-05-21T18:56:47.148048708Z Gathering data for ns/openshift-config...
2026-05-21T18:56:47.763631314Z Gathering data for ns/openshift-config-managed...
2026-05-21T18:56:48.005233799Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-21T18:56:48.426252613Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-21T18:56:48.837997520Z Gathering data for ns/openshift-kube-apiserver...
2026-05-21T18:56:49.260003929Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-21T18:56:49.672772255Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-21T18:56:50.163199509Z Gathering data for ns/openshift-kube-scheduler...
2026-05-21T18:56:50.535963679Z Collecting kubelet logs for node ip-10-0-142-242.ec2.internal
2026-05-21T18:56:50.537724514Z Collecting kubelet logs for node ip-10-0-129-93.ec2.internal
2026-05-21T18:56:50.539510532Z Collecting kubelet logs for node ip-10-0-138-250.ec2.internal
2026-05-21T18:56:50.595429788Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-21T18:56:50.908774025Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-21T18:56:50.917579392Z INFO: Node performance data collection complete.
2026-05-21T18:56:51.066419937Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-21T18:56:51.552807206Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-21T18:56:51.964910098Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-21T18:56:52.586297713Z Gathering data for ns/openshift-multus...
2026-05-21T18:56:53.009766174Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-21T18:56:54.821445969Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-21T18:56:56.935729107Z Gathering data for ns/openshift-host-network...
2026-05-21T18:56:57.369484643Z Gathering data for ns/openshift-network-diagnostics...
2026-05-21T18:56:58.014322425Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-21T18:56:58.177204822Z Gathering data for ns/openshift-network-node-identity...
2026-05-21T18:56:58.796239767Z Gathering data for ns/openshift-network-operator...
2026-05-21T18:56:59.472926887Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-21T18:56:59.912783228Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-21T18:57:00.504050826Z Gathering data for ns/openshift-apiserver-operator...
2026-05-21T18:57:00.902885835Z Gathering data for ns/openshift-apiserver...
2026-05-21T18:57:01.516445692Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-21T18:57:01.937190392Z Gathering data for ns/openshift-controller-manager...
2026-05-21T18:57:02.299034243Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-21T18:57:03.018956137Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-21T18:57:03.131597888Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-21T18:57:03.545655515Z Gathering data for ns/openshift-service-ca-operator...
2026-05-21T18:57:04.109416302Z Gathering data for ns/openshift-service-ca...
2026-05-21T18:57:04.681878139Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-21T18:57:06.226647339Z Wrote inspect data to must-gather.
2026-05-21T18:57:06.226694215Z error: inspection completed with the errors occurred while gathering data:
2026-05-21T18:57:06.226694215Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-21T18:57:06.363739104Z error: the server doesn't have a resource type "clusters"
2026-05-21T18:57:06.517265089Z Caches written to disk