2026-05-27T21:51:56.339632278Z [disk usage checker] Started 2026-05-27T21:51:56.343452621Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-05-27T21:51:56.578490377Z Gathering data for ns/openshift-cluster-version... 2026-05-27T21:51:56.775511953Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-05-27T21:51:56.897045145Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-05-27T21:51:57.005514022Z Gathering data for ns/default... 2026-05-27T21:51:57.457664835Z Gathering data for ns/openshift... 2026-05-27T21:51:57.990610702Z Gathering data for ns/kube-system... 2026-05-27T21:51:58.791202114Z Waiting on subprocesses to finish execution. 2026-05-27T21:51:58.804057328Z INFO: Gathering HAProxy config files 2026-05-27T21:51:58.805252386Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-05-27T21:51:58.809423416Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-05-27T21:51:58.817958828Z INFO: Collecting host service logs for crio 2026-05-27T21:51:58.818275595Z INFO: Collecting host service logs for kubelet 2026-05-27T21:51:58.818615682Z INFO: Collecting host service logs for rpm-ostreed 2026-05-27T21:51:58.818984862Z INFO: Collecting host service logs for ostree-finalize-staged 2026-05-27T21:51:58.819314460Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-05-27T21:51:58.819631665Z INFO: Collecting host service logs for machine-config-daemon-host 2026-05-27T21:51:58.819968637Z INFO: Collecting host service logs for NetworkManager 2026-05-27T21:51:58.820274663Z INFO: Collecting host service logs for openvswitch 2026-05-27T21:51:58.820567530Z INFO: Collecting host service logs for ovs-configuration 2026-05-27T21:51:58.820905282Z INFO: Collecting host service logs for ovsdb-server 2026-05-27T21:51:58.821223709Z INFO: Collecting host service logs for ovs-vswitchd 2026-05-27T21:51:58.821591252Z INFO: Waiting for worker host service log collection to complete ... 2026-05-27T21:51:58.850493415Z INFO: Gathering machine config daemon's old logs from all nodes 2026-05-27T21:51:58.954258660Z Gathering data for ns/openshift-etcd... 2026-05-27T21:51:58.970998251Z INFO: Waiting for node performance related collection to complete ... 2026-05-27T21:51:59.375495056Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-05-27T21:51:59.700765151Z INFO: Found 1 replicas - prometheus-k8s-0 2026-05-27T21:51:59.819493424Z No resources found in openshift-etcd namespace. 2026-05-27T21:51:59.858355353Z Wrote inspect data to must-gather. 2026-05-27T21:52:00.028196765Z INFO: "sriov-network-operator" not detected. Skipping. 2026-05-27T21:52:00.122560358Z Wrote inspect data to must-gather. 2026-05-27T21:52:00.122600107Z error: inspection completed with the errors occurred while gathering data: 2026-05-27T21:52:00.122600107Z namespaces "assisted-installer" not found 2026-05-27T21:52:00.185097784Z No resources found 2026-05-27T21:52:00.322157151Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-8kjtr 2026-05-27T21:52:00.322157151Z insights-runtime-extractor-59nj7 2026-05-27T21:52:00.322157151Z insights-runtime-extractor-pdw2g 2026-05-27T21:52:00.322157151Z insights-runtime-extractor-zq4gf 2026-05-27T21:52:00.401826554Z ERROR: No running kube-apiserver pods found 2026-05-27T21:52:00.437238992Z INFO: "metallb-operator" not detected. Skipping. 2026-05-27T21:52:00.465751238Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-05-27T21:52:00.465842622Z INFO: on-disk MachineConfig config collection complete. 2026-05-27T21:52:00.465993580Z INFO: Worker host service log collection to complete. 2026-05-27T21:52:00.769929227Z error: the server doesn't have a resource type "performanceprofile" 2026-05-27T21:52:00.975587629Z INFO: Waiting for HAProxy config collection to complete ... 2026-05-27T21:52:00.992667541Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-05-27T21:52:01.105623973Z Wrote inspect data to must-gather. 2026-05-27T21:52:01.177535124Z No resources found 2026-05-27T21:52:01.198984585Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-05-27T21:52:01.198984585Z See 'oc rsync -h' for help and examples 2026-05-27T21:52:01.274870512Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-05-27T21:52:01.318274287Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-05-27T21:52:01.334438283Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-05-27T21:52:01.410667741Z Wrote inspect data to must-gather. 2026-05-27T21:52:01.448904868Z INFO: 'previous-logs' folder not found on ip-10-0-130-139.ec2.internal, skipping... 2026-05-27T21:52:01.462685625Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-05-27T21:52:01.566755431Z Wrote inspect data to must-gather. 2026-05-27T21:52:01.755410792Z Wrote inspect data to must-gather. 2026-05-27T21:52:01.757322237Z INFO: HAProxy config collection complete. 2026-05-27T21:52:01.759722289Z Gathering data for ns/kuadrant-system... 2026-05-27T21:52:01.794008397Z Wrote inspect data to must-gather. 2026-05-27T21:52:01.888056563Z INFO: 'previous-logs' folder not found on ip-10-0-131-147.ec2.internal, skipping... 2026-05-27T21:52:02.041827096Z error: the server doesn't have a resource type "machineconfigs" 2026-05-27T21:52:02.045792694Z INFO: Getting rules from prometheus-k8s-0 2026-05-27T21:52:02.109250496Z Wrote inspect data to must-gather. 2026-05-27T21:52:02.259934289Z INFO: 'previous-logs' folder not found on ip-10-0-140-70.ec2.internal, skipping... 2026-05-27T21:52:02.260032126Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-05-27T21:52:02.260083002Z INFO: Machine Config Daemon termination log collection complete. 2026-05-27T21:52:02.359613083Z Wrote inspect data to must-gather. 2026-05-27T21:52:02.467202042Z Wrote inspect data to must-gather. 2026-05-27T21:52:02.537145253Z INFO: Getting status/config from prometheus-k8s-0 2026-05-27T21:52:02.679002568Z Wrote inspect data to must-gather. 2026-05-27T21:52:02.849227874Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-05-27T21:52:02.853532676Z Wrote inspect data to must-gather. 2026-05-27T21:52:02.868746854Z error: the server doesn't have a resource type "kubeletconfigs" 2026-05-27T21:52:03.101476089Z INFO: Getting status/flags from prometheus-k8s-0 2026-05-27T21:52:03.202970422Z Wrote inspect data to must-gather. 2026-05-27T21:52:03.284941996Z INFO: INTERCONNECT MODE 2026-05-27T21:52:03.285062308Z INFO: Gathering ovn-kubernetes DBs 2026-05-27T21:52:03.374959948Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-05-27T21:52:03.473315572Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-05-27T21:52:03.504381697Z INFO: Gathering OVN_Northbound from ovnkube-node-hk5dd... 2026-05-27T21:52:03.504712641Z INFO: Gathering OVN_Northbound from ovnkube-node-j7fvn... 2026-05-27T21:52:03.505010834Z INFO: Gathering OVN_Northbound from ovnkube-node-szbhn... 2026-05-27T21:52:03.505321721Z INFO: Gathering OVN_Southbound from ovnkube-node-hk5dd... 2026-05-27T21:52:03.505587161Z INFO: Gathering OVN_Southbound from ovnkube-node-j7fvn... 2026-05-27T21:52:03.505836662Z INFO: Gathering OVN_Southbound from ovnkube-node-szbhn... 2026-05-27T21:52:03.671842790Z Wrote inspect data to must-gather. 2026-05-27T21:52:03.724925468Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-05-27T21:52:03.882523150Z Gathering data for ns/openshift-monitoring... 2026-05-27T21:52:03.910119789Z tar: Removing leading `/' from member names 2026-05-27T21:52:04.007926880Z tar: Removing leading `/' from member names 2026-05-27T21:52:04.010616078Z tar: Removing leading `/' from member names 2026-05-27T21:52:04.020807078Z tar: Removing leading `/' from member names 2026-05-27T21:52:04.043358730Z tar: Removing leading `/' from member names 2026-05-27T21:52:04.091086300Z tar: Removing leading `/' from member names 2026-05-27T21:52:04.177578804Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-05-27T21:52:04.252362003Z Wrote inspect data to must-gather. 2026-05-27T21:52:04.655047667Z Wrote inspect data to must-gather. 2026-05-27T21:52:04.660379078Z INFO: Waiting for network log collection to complete ... 2026-05-27T21:52:04.660425874Z INFO: Waiting for ovnk database copies to complete ... 2026-05-27T21:52:04.660923886Z INFO: Copying ovnk databases complete. 2026-05-27T21:52:04.662808786Z 4.0M must-gather/network_logs/ovnk_database_store 2026-05-27T21:52:04.665544914Z ovnk_database_store/ 2026-05-27T21:52:04.665544914Z ovnk_database_store/ovnkube-node-hk5dd_nbdb 2026-05-27T21:52:04.668158322Z INFO: Getting status from alertmanager-main-0 2026-05-27T21:52:04.676945281Z ovnk_database_store/ovnkube-node-szbhn_nbdb 2026-05-27T21:52:04.686763392Z ovnk_database_store/ovnkube-node-j7fvn_nbdb 2026-05-27T21:52:04.700380830Z ovnk_database_store/ovnkube-node-hk5dd_sbdb 2026-05-27T21:52:04.728674743Z ovnk_database_store/ovnkube-node-szbhn_sbdb 2026-05-27T21:52:04.749402005Z ovnk_database_store/ovnkube-node-j7fvn_sbdb 2026-05-27T21:52:04.779040882Z INFO: Network log collection complete. 2026-05-27T21:52:04.996880668Z Wrote inspect data to must-gather. 2026-05-27T21:52:05.275272543Z Wrote inspect data to must-gather. 2026-05-27T21:52:05.582197051Z Wrote inspect data to must-gather. 2026-05-27T21:52:05.873114840Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-05-27T21:52:05.876174518Z INFO: Fallback to identify the container image from release info 2026-05-27T21:52:06.467149379Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda 2026-05-27T21:52:06.468778459Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-05-27T21:52:06.603378233Z daemonset.apps/perf-node-gather-daemonset created 2026-05-27T21:52:06.729522146Z Gathering data for ns/openshift-network-console... 2026-05-27T21:52:06.769410339Z Waiting for performance profile collector pods to become ready: 1 2026-05-27T21:52:07.398623904Z Gathering data for ns/openshift-console-operator... 2026-05-27T21:52:07.759251092Z Wrote inspect data to must-gather. 2026-05-27T21:52:07.908969978Z Gathering data for ns/openshift-console... 2026-05-27T21:52:07.912066548Z Waiting for performance profile collector pods to become ready: 2 2026-05-27T21:52:08.445139590Z Gathering data for ns/openshift-cluster-storage-operator... 2026-05-27T21:52:08.985935313Z Gathering data for ns/openshift-dns-operator... 2026-05-27T21:52:09.050852366Z Waiting for performance profile collector pods to become ready: 3 2026-05-27T21:52:09.384937966Z Gathering data for ns/openshift-dns... 2026-05-27T21:52:10.143653865Z Gathering data for ns/openshift-image-registry... 2026-05-27T21:52:10.176311667Z Waiting for performance profile collector pods to become ready: 4 2026-05-27T21:52:10.732692283Z Gathering data for ns/openshift-ingress-operator... 2026-05-27T21:52:11.263792236Z Gathering data for ns/openshift-ingress... 2026-05-27T21:52:11.305993003Z Waiting for performance profile collector pods to become ready: 5 2026-05-27T21:52:11.474396397Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-05-27T21:52:12.082959042Z Gathering data for ns/openshift-ingress-canary... 2026-05-27T21:52:12.421056986Z Waiting for performance profile collector pods to become ready: 6 2026-05-27T21:52:12.630473914Z Gathering data for ns/openshift-insights... 2026-05-27T21:52:13.545350660Z Waiting for performance profile collector pods to become ready: 7 2026-05-27T21:52:14.661292627Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-05-27T21:52:14.929064036Z Gathering data for ns/opendatahub... 2026-05-27T21:52:14.939198803Z Collecting performance related data for node ip-10-0-140-70.ec2.internal 2026-05-27T21:52:14.946710602Z Collecting performance related data for node ip-10-0-131-147.ec2.internal 2026-05-27T21:52:14.955330978Z Collecting performance related data for node ip-10-0-130-139.ec2.internal 2026-05-27T21:52:16.481248827Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-05-27T21:52:16.580080445Z Gathering data for ns/openshift-lws-operator... 2026-05-27T21:52:17.961398031Z Gathering data for ns/openshift-config... 2026-05-27T21:52:18.370176979Z Gathering data for ns/openshift-config-managed... 2026-05-27T21:52:18.892605323Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-05-27T21:52:19.280928131Z Gathering data for ns/openshift-kube-apiserver... 2026-05-27T21:52:19.653953574Z Gathering data for ns/openshift-kube-controller-manager... 2026-05-27T21:52:20.101998622Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-05-27T21:52:20.506580265Z Gathering data for ns/openshift-kube-scheduler... 2026-05-27T21:52:20.884403615Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-05-27T21:52:21.301232679Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-05-27T21:52:21.489123363Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-05-27T21:52:21.723739164Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-05-27T21:52:22.189073981Z Gathering data for ns/openshift-user-workload-monitoring... 2026-05-27T21:52:22.962853675Z Gathering data for ns/openshift-multus... 2026-05-27T21:52:23.402627053Z Collecting kubelet logs for node ip-10-0-140-70.ec2.internal 2026-05-27T21:52:23.404558610Z Collecting kubelet logs for node ip-10-0-131-147.ec2.internal 2026-05-27T21:52:23.406316216Z Collecting kubelet logs for node ip-10-0-130-139.ec2.internal 2026-05-27T21:52:23.751035006Z daemonset.apps "perf-node-gather-daemonset" deleted 2026-05-27T21:52:23.756300264Z INFO: Node performance data collection complete. 2026-05-27T21:52:24.732142508Z Gathering data for ns/openshift-ovn-kubernetes... 2026-05-27T21:52:26.410385245Z Gathering data for ns/openshift-host-network... 2026-05-27T21:52:26.494529295Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-05-27T21:52:26.796591153Z Gathering data for ns/openshift-network-diagnostics... 2026-05-27T21:52:27.447058103Z Gathering data for ns/openshift-network-node-identity... 2026-05-27T21:52:27.982784022Z Gathering data for ns/openshift-network-operator... 2026-05-27T21:52:28.451652942Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-05-27T21:52:28.801136536Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-05-27T21:52:29.373180664Z Gathering data for ns/openshift-apiserver-operator... 2026-05-27T21:52:29.770417761Z Gathering data for ns/openshift-apiserver... 2026-05-27T21:52:30.317663802Z Gathering data for ns/openshift-controller-manager-operator... 2026-05-27T21:52:30.671473988Z Gathering data for ns/openshift-controller-manager... 2026-05-27T21:52:31.061034009Z Gathering data for ns/openshift-cluster-samples-operator... 2026-05-27T21:52:31.500334527Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-05-27T21:52:31.822337241Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-05-27T21:52:32.226592182Z Gathering data for ns/openshift-service-ca-operator... 2026-05-27T21:52:32.641200568Z Gathering data for ns/openshift-service-ca... 2026-05-27T21:52:33.117960902Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-05-27T21:52:34.223576483Z Wrote inspect data to must-gather. 2026-05-27T21:52:34.223616324Z error: inspection completed with the errors occurred while gathering data: 2026-05-27T21:52:34.223616324Z skipping gathering secrets/support due to error: secrets "support" not found 2026-05-27T21:52:34.358593055Z error: the server doesn't have a resource type "clusters" 2026-05-27T21:52:34.549099123Z Caches written to disk