2026-05-20T20:57:52.592127783Z [disk usage checker] Started
2026-05-20T20:57:52.595495817Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-20T20:57:52.839679099Z Gathering data for ns/openshift-cluster-version...
2026-05-20T20:57:53.081742285Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-20T20:57:53.202983248Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-20T20:57:53.306469168Z Gathering data for ns/default...
2026-05-20T20:57:53.757712462Z Gathering data for ns/openshift...
2026-05-20T20:57:54.260023972Z Gathering data for ns/kube-system...
2026-05-20T20:57:54.958622493Z Waiting on subprocesses to finish execution.
2026-05-20T20:57:54.966841038Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-20T20:57:54.974924486Z INFO: Collecting host service logs for crio
2026-05-20T20:57:54.975386470Z INFO: Collecting host service logs for kubelet
2026-05-20T20:57:54.975751721Z INFO: Collecting host service logs for rpm-ostreed
2026-05-20T20:57:54.976151278Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-20T20:57:54.976568243Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-20T20:57:54.976937737Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-20T20:57:54.977281143Z INFO: Collecting host service logs for NetworkManager
2026-05-20T20:57:54.977701547Z INFO: Collecting host service logs for openvswitch
2026-05-20T20:57:54.978038153Z INFO: Collecting host service logs for ovs-configuration
2026-05-20T20:57:54.978357444Z INFO: Collecting host service logs for ovsdb-server
2026-05-20T20:57:54.978703672Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-20T20:57:54.979020892Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-20T20:57:54.979105608Z INFO: Waiting for worker host service log collection to complete ...
2026-05-20T20:57:54.995818069Z INFO: Gathering HAProxy config files
2026-05-20T20:57:54.997509262Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-20T20:57:55.345673466Z INFO: Waiting for node performance related collection to complete ...
2026-05-20T20:57:55.562342063Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-wpr7n
2026-05-20T20:57:55.562402186Z insights-runtime-extractor-bgtlz
2026-05-20T20:57:55.562446903Z insights-runtime-extractor-cp7jf
2026-05-20T20:57:55.562471697Z insights-runtime-extractor-zwfzg
2026-05-20T20:57:55.762197852Z No resources found
2026-05-20T20:57:55.784016438Z Gathering data for ns/openshift-etcd...
2026-05-20T20:57:55.879601785Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-20T20:57:55.983261819Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-20T20:57:56.225805169Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-20T20:57:56.225892075Z INFO: on-disk MachineConfig config collection complete.
2026-05-20T20:57:56.335650934Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-20T20:57:56.335650934Z See 'oc rsync -h' for help and examples
2026-05-20T20:57:56.391803060Z Wrote inspect data to must-gather.
2026-05-20T20:57:56.453641508Z INFO: "metallb-operator" not detected. Skipping.
2026-05-20T20:57:56.461596099Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-20T20:57:56.498755741Z No resources found in openshift-etcd namespace.
2026-05-20T20:57:56.514956316Z INFO: Worker host service log collection to complete.
2026-05-20T20:57:56.612688999Z INFO: 'previous-logs' folder not found on ip-10-0-129-75.ec2.internal, skipping...
2026-05-20T20:57:56.709647460Z ERROR: No running kube-apiserver pods found
2026-05-20T20:57:56.720617427Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-20T20:57:56.838218657Z error: the server doesn't have a resource type "performanceprofile"
2026-05-20T20:57:57.067375406Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-20T20:57:57.119886673Z No resources found
2026-05-20T20:57:57.218077195Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-20T20:57:57.258599421Z Wrote inspect data to must-gather.
2026-05-20T20:57:57.573405794Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-20T20:57:57.660282748Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-20T20:57:57.691617492Z Wrote inspect data to must-gather.
2026-05-20T20:57:57.712992743Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-20T20:57:57.727566069Z INFO: 'previous-logs' folder not found on ip-10-0-138-220.ec2.internal, skipping...
2026-05-20T20:57:57.733133949Z Gathering data for ns/kuadrant-system...
2026-05-20T20:57:57.791717068Z Wrote inspect data to must-gather.
2026-05-20T20:57:57.854386236Z Wrote inspect data to must-gather.
2026-05-20T20:57:57.854419313Z error: inspection completed with the errors occurred while gathering data:
2026-05-20T20:57:57.854419313Z     namespaces "assisted-installer" not found
2026-05-20T20:57:57.945697702Z Wrote inspect data to must-gather.
2026-05-20T20:57:57.962003003Z Wrote inspect data to must-gather.
2026-05-20T20:57:58.143337904Z INFO: 'previous-logs' folder not found on ip-10-0-142-116.ec2.internal, skipping...
2026-05-20T20:57:58.145265238Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-20T20:57:58.145291550Z INFO: Machine Config Daemon termination log collection complete.
2026-05-20T20:57:58.165637901Z tar: Removing leading `/' from member names
2026-05-20T20:57:58.172848486Z INFO: HAProxy config collection complete.
2026-05-20T20:57:58.178039356Z error: the server doesn't have a resource type "machineconfigs"
2026-05-20T20:57:58.211998300Z Wrote inspect data to must-gather.
2026-05-20T20:57:58.316734079Z INFO: Getting rules from prometheus-k8s-0
2026-05-20T20:57:58.459400266Z Wrote inspect data to must-gather.
2026-05-20T20:57:58.556420386Z Wrote inspect data to must-gather.
2026-05-20T20:57:58.785232183Z Wrote inspect data to must-gather.
2026-05-20T20:57:58.868852122Z INFO: Getting status/config from prometheus-k8s-0
2026-05-20T20:57:58.882798714Z Wrote inspect data to must-gather.
2026-05-20T20:57:58.972162252Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-20T20:57:59.276725586Z INFO: INTERCONNECT MODE
2026-05-20T20:57:59.276808591Z INFO: Gathering ovn-kubernetes DBs
2026-05-20T20:57:59.291825419Z Wrote inspect data to must-gather.
2026-05-20T20:57:59.420864333Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-20T20:57:59.428017303Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-20T20:57:59.548323027Z INFO: Gathering OVN_Northbound from ovnkube-node-5z2t6...
2026-05-20T20:57:59.548599196Z INFO: Gathering OVN_Northbound from ovnkube-node-kdcdj...
2026-05-20T20:57:59.548825781Z INFO: Gathering OVN_Northbound from ovnkube-node-njrxw...
2026-05-20T20:57:59.549070545Z INFO: Gathering OVN_Southbound from ovnkube-node-5z2t6...
2026-05-20T20:57:59.549316507Z INFO: Gathering OVN_Southbound from ovnkube-node-kdcdj...
2026-05-20T20:57:59.549558353Z INFO: Gathering OVN_Southbound from ovnkube-node-njrxw...
2026-05-20T20:57:59.549731781Z Wrote inspect data to must-gather.
2026-05-20T20:57:59.744258530Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-20T20:57:59.930608103Z tar: Removing leading `/' from member names
2026-05-20T20:58:00.020100036Z tar: Removing leading `/' from member names
2026-05-20T20:58:00.064729100Z tar: Removing leading `/' from member names
2026-05-20T20:58:00.064874585Z tar: Removing leading `/' from member names
2026-05-20T20:58:00.086697255Z W0520 20:58:00.086663    1204 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-05-20T20:58:00.103862000Z tar: Removing leading `/' from member names
2026-05-20T20:58:00.109799237Z tar: Removing leading `/' from member names
2026-05-20T20:58:00.111531169Z Wrote inspect data to must-gather.
2026-05-20T20:58:00.116127363Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-20T20:58:00.223257601Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-20T20:58:00.403758494Z Gathering data for ns/openshift-monitoring...
2026-05-20T20:58:00.514790600Z Wrote inspect data to must-gather.
2026-05-20T20:58:00.594955871Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-20T20:58:00.695376469Z INFO: Waiting for network log collection to complete ...
2026-05-20T20:58:00.695410707Z INFO: Waiting for ovnk database copies to complete ...
2026-05-20T20:58:00.695797274Z INFO: Copying ovnk databases complete.
2026-05-20T20:58:00.697817248Z 4.3M	must-gather/network_logs/ovnk_database_store
2026-05-20T20:58:00.700248895Z ovnk_database_store/
2026-05-20T20:58:00.700301749Z ovnk_database_store/ovnkube-node-5z2t6_sbdb
2026-05-20T20:58:00.727036306Z ovnk_database_store/ovnkube-node-5z2t6_nbdb
2026-05-20T20:58:00.740290966Z ovnk_database_store/ovnkube-node-kdcdj_nbdb
2026-05-20T20:58:00.754387457Z ovnk_database_store/ovnkube-node-kdcdj_sbdb
2026-05-20T20:58:00.799238273Z ovnk_database_store/ovnkube-node-njrxw_nbdb
2026-05-20T20:58:00.813507090Z ovnk_database_store/ovnkube-node-njrxw_sbdb
2026-05-20T20:58:00.843238756Z INFO: Network log collection complete.
2026-05-20T20:58:00.864011956Z Wrote inspect data to must-gather.
2026-05-20T20:58:01.122167436Z INFO: Getting status from alertmanager-main-0
2026-05-20T20:58:01.189381875Z Wrote inspect data to must-gather.
2026-05-20T20:58:01.465837272Z Wrote inspect data to must-gather.
2026-05-20T20:58:01.717053902Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-20T20:58:01.719268732Z INFO: Fallback to identify the container image from release info
2026-05-20T20:58:02.285124806Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda
2026-05-20T20:58:02.431524108Z daemonset.apps/perf-node-gather-daemonset created
2026-05-20T20:58:02.584784848Z Waiting for performance profile collector pods to become ready: 1
2026-05-20T20:58:02.666205117Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-20T20:58:02.988940643Z Gathering data for ns/openshift-network-console...
2026-05-20T20:58:03.682474769Z Gathering data for ns/openshift-console-operator...
2026-05-20T20:58:03.732994230Z Waiting for performance profile collector pods to become ready: 2
2026-05-20T20:58:04.226244917Z Gathering data for ns/openshift-console...
2026-05-20T20:58:04.646149448Z Wrote inspect data to must-gather.
2026-05-20T20:58:04.756940515Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-20T20:58:04.865861922Z Waiting for performance profile collector pods to become ready: 3
2026-05-20T20:58:05.218613782Z Gathering data for ns/openshift-dns-operator...
2026-05-20T20:58:05.561521206Z Gathering data for ns/openshift-dns...
2026-05-20T20:58:05.973276427Z Waiting for performance profile collector pods to become ready: 4
2026-05-20T20:58:06.394315665Z Gathering data for ns/openshift-image-registry...
2026-05-20T20:58:06.936929599Z Gathering data for ns/openshift-ingress-operator...
2026-05-20T20:58:07.092511861Z Waiting for performance profile collector pods to become ready: 5
2026-05-20T20:58:07.337748804Z Gathering data for ns/openshift-ingress...
2026-05-20T20:58:07.671310014Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-20T20:58:08.212644040Z Waiting for performance profile collector pods to become ready: 6
2026-05-20T20:58:08.266524288Z Gathering data for ns/openshift-ingress-canary...
2026-05-20T20:58:08.847742422Z Gathering data for ns/openshift-insights...
2026-05-20T20:58:09.311461059Z Waiting for performance profile collector pods to become ready: 7
2026-05-20T20:58:10.417489531Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-20T20:58:10.647685795Z Collecting performance related data for node ip-10-0-138-220.ec2.internal
2026-05-20T20:58:10.654838593Z Collecting performance related data for node ip-10-0-142-116.ec2.internal
2026-05-20T20:58:10.662532790Z Collecting performance related data for node ip-10-0-129-75.ec2.internal
2026-05-20T20:58:11.362203614Z Gathering data for ns/opendatahub...
2026-05-20T20:58:12.676016426Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-20T20:58:13.175105465Z Gathering data for ns/openshift-lws-operator...
2026-05-20T20:58:14.513635415Z Gathering data for ns/openshift-config...
2026-05-20T20:58:14.939034431Z Gathering data for ns/openshift-config-managed...
2026-05-20T20:58:15.451542913Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-20T20:58:15.821395233Z Gathering data for ns/openshift-kube-apiserver...
2026-05-20T20:58:16.234614062Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-20T20:58:16.630257682Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-20T20:58:17.012934643Z Gathering data for ns/openshift-kube-scheduler...
2026-05-20T20:58:17.394556346Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-20T20:58:17.680863758Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-20T20:58:17.756284963Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-20T20:58:18.139530993Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-20T20:58:18.520715355Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-20T20:58:18.752253722Z Collecting kubelet logs for node ip-10-0-138-220.ec2.internal
2026-05-20T20:58:18.754135100Z Collecting kubelet logs for node ip-10-0-142-116.ec2.internal
2026-05-20T20:58:18.756578241Z Collecting kubelet logs for node ip-10-0-129-75.ec2.internal
2026-05-20T20:58:19.110593738Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-20T20:58:19.114909237Z INFO: Node performance data collection complete.
2026-05-20T20:58:19.277699181Z Gathering data for ns/openshift-multus...
2026-05-20T20:58:20.875663374Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-20T20:58:22.685724029Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-20T20:58:22.795364043Z Gathering data for ns/openshift-host-network...
2026-05-20T20:58:23.174130930Z Gathering data for ns/openshift-network-diagnostics...
2026-05-20T20:58:23.760756353Z Gathering data for ns/openshift-network-node-identity...
2026-05-20T20:58:24.225325381Z Gathering data for ns/openshift-network-operator...
2026-05-20T20:58:24.648226149Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-20T20:58:24.988079921Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-20T20:58:25.498445942Z Gathering data for ns/openshift-apiserver-operator...
2026-05-20T20:58:25.859809183Z Gathering data for ns/openshift-apiserver...
2026-05-20T20:58:26.436090290Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-20T20:58:26.760671530Z Gathering data for ns/openshift-controller-manager...
2026-05-20T20:58:27.079685889Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-20T20:58:27.690949694Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-20T20:58:27.793624555Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-20T20:58:28.136808249Z Gathering data for ns/openshift-service-ca-operator...
2026-05-20T20:58:28.502589969Z Gathering data for ns/openshift-service-ca...
2026-05-20T20:58:28.982531887Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-20T20:58:30.042114331Z Wrote inspect data to must-gather.
2026-05-20T20:58:30.042164294Z error: inspection completed with the errors occurred while gathering data:
2026-05-20T20:58:30.042164294Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-20T20:58:30.178303787Z error: the server doesn't have a resource type "clusters"
2026-05-20T20:58:30.363939898Z Caches written to disk