2026-04-16T19:00:01.071456205Z [disk usage checker] Started
2026-04-16T19:00:01.075172798Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-04-16T19:00:01.437021117Z Gathering data for ns/openshift-cluster-version...
2026-04-16T19:00:01.597027026Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T19:00:01.701546688Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T19:00:01.798422598Z Gathering data for ns/default...
2026-04-16T19:00:02.157934211Z Gathering data for ns/openshift...
2026-04-16T19:00:02.631425191Z Gathering data for ns/kube-system...
2026-04-16T19:00:03.356402683Z Gathering data for ns/openshift-etcd...
2026-04-16T19:00:03.716386532Z Wrote inspect data to must-gather.
2026-04-16T19:00:03.716419981Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T19:00:03.716419981Z     namespaces "assisted-installer" not found
2026-04-16T19:00:03.931315423Z Waiting on subprocesses to finish execution.
2026-04-16T19:00:03.937696148Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-16T19:00:03.943237030Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-16T19:00:03.948548836Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-16T19:00:03.954436451Z INFO: Collecting host service logs for crio
2026-04-16T19:00:03.954693708Z INFO: Collecting host service logs for kubelet
2026-04-16T19:00:03.955062550Z INFO: Collecting host service logs for rpm-ostreed
2026-04-16T19:00:03.955346423Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-16T19:00:03.955655920Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-16T19:00:03.955998778Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-16T19:00:03.956306223Z INFO: Collecting host service logs for NetworkManager
2026-04-16T19:00:03.956620778Z INFO: Collecting host service logs for openvswitch
2026-04-16T19:00:03.956941660Z INFO: Collecting host service logs for ovs-configuration
2026-04-16T19:00:03.957287692Z INFO: Collecting host service logs for ovsdb-server
2026-04-16T19:00:03.957559224Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-16T19:00:03.957942628Z INFO: Waiting for worker host service log collection to complete ...
2026-04-16T19:00:03.971901764Z INFO: Waiting for node performance related collection to complete ...
2026-04-16T19:00:03.977854045Z INFO: Gathering HAProxy config files
2026-04-16T19:00:04.608355702Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-16T19:00:04.612258215Z INFO: Collecting Insights Archives from insights-operator-5785d4fcdd-g7zhh
2026-04-16T19:00:04.612307816Z insights-runtime-extractor-6d74h
2026-04-16T19:00:04.612328134Z insights-runtime-extractor-fphbb
2026-04-16T19:00:04.612363030Z insights-runtime-extractor-pjfc2
2026-04-16T19:00:04.686852582Z No resources found in openshift-etcd namespace.
2026-04-16T19:00:04.697476053Z No resources found
2026-04-16T19:00:04.883289180Z error: the server doesn't have a resource type "performanceprofile"
2026-04-16T19:00:04.916238693Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-16T19:00:04.916282087Z INFO: on-disk MachineConfig config collection complete.
2026-04-16T19:00:04.917900341Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-16T19:00:05.064495673Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-16T19:00:05.088121358Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-16T19:00:05.185185438Z INFO: "metallb-operator" not detected. Skipping.
2026-04-16T19:00:05.310151259Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-16T19:00:05.310151259Z See 'oc rsync -h' for help and examples
2026-04-16T19:00:05.333898395Z INFO: Worker host service log collection to complete.
2026-04-16T19:00:05.367496747Z Wrote inspect data to must-gather.
2026-04-16T19:00:05.525247044Z ERROR: No running kube-apiserver pods found
2026-04-16T19:00:05.793025562Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-16T19:00:06.200193346Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-04-16T19:00:06.211854520Z Wrote inspect data to must-gather.
2026-04-16T19:00:06.329027181Z No resources found
2026-04-16T19:00:06.388639074Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-16T19:00:06.564722398Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T19:00:06.593938446Z Wrote inspect data to must-gather.
2026-04-16T19:00:06.730841830Z Wrote inspect data to must-gather.
2026-04-16T19:00:06.762523728Z error: the server doesn't have a resource type "machineconfigs"
2026-04-16T19:00:06.855353279Z Wrote inspect data to must-gather.
2026-04-16T19:00:06.867436250Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-16T19:00:06.882364554Z INFO: 'previous-logs' folder not found on ip-10-0-135-146.ec2.internal, skipping...
2026-04-16T19:00:06.949957150Z Wrote inspect data to must-gather.
2026-04-16T19:00:07.009818354Z Gathering data for ns/kuadrant-system...
2026-04-16T19:00:07.109495547Z INFO: Getting rules from prometheus-k8s-0
2026-04-16T19:00:07.170960250Z tar: Removing leading `/' from member names
2026-04-16T19:00:07.179686184Z INFO: HAProxy config collection complete.
2026-04-16T19:00:07.195450808Z Wrote inspect data to must-gather.
2026-04-16T19:00:07.212459108Z Wrote inspect data to must-gather.
2026-04-16T19:00:07.341457328Z INFO: 'previous-logs' folder not found on ip-10-0-137-47.ec2.internal, skipping...
2026-04-16T19:00:07.569333139Z Wrote inspect data to must-gather.
2026-04-16T19:00:07.585306675Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T19:00:07.642122872Z Wrote inspect data to must-gather.
2026-04-16T19:00:07.681456685Z INFO: Getting status/config from prometheus-k8s-0
2026-04-16T19:00:07.767590030Z INFO: 'previous-logs' folder not found on ip-10-0-140-154.ec2.internal, skipping...
2026-04-16T19:00:07.767728299Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-16T19:00:07.767800783Z INFO: Machine Config Daemon termination log collection complete.
2026-04-16T19:00:07.851991990Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-16T19:00:07.955378030Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T19:00:08.107942983Z Wrote inspect data to must-gather.
2026-04-16T19:00:08.180156844Z Gathering data for ns/openshift-monitoring...
2026-04-16T19:00:08.253965011Z Wrote inspect data to must-gather.
2026-04-16T19:00:08.307370994Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-16T19:00:08.529225668Z INFO: INTERCONNECT MODE
2026-04-16T19:00:08.529256165Z INFO: Gathering ovn-kubernetes DBs
2026-04-16T19:00:08.561182738Z Wrote inspect data to must-gather.
2026-04-16T19:00:08.591079411Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-16T19:00:08.731068033Z INFO: Gathering OVN_Northbound from ovnkube-node-6nzqh...
2026-04-16T19:00:08.731335507Z INFO: Gathering OVN_Northbound from ovnkube-node-78v92...
2026-04-16T19:00:08.731628409Z INFO: Gathering OVN_Northbound from ovnkube-node-tchmw...
2026-04-16T19:00:08.732046151Z INFO: Gathering OVN_Southbound from ovnkube-node-6nzqh...
2026-04-16T19:00:08.732409227Z INFO: Gathering OVN_Southbound from ovnkube-node-78v92...
2026-04-16T19:00:08.732716424Z INFO: Gathering OVN_Southbound from ovnkube-node-tchmw...
2026-04-16T19:00:08.948274404Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-16T19:00:09.060293225Z tar: Removing leading `/' from member names
2026-04-16T19:00:09.061049363Z tar: Removing leading `/' from member names
2026-04-16T19:00:09.077444484Z Wrote inspect data to must-gather.
2026-04-16T19:00:09.111027811Z tar: Removing leading `/' from member names
2026-04-16T19:00:09.169597355Z tar: Removing leading `/' from member names
2026-04-16T19:00:09.201781275Z tar: Removing leading `/' from member names
2026-04-16T19:00:09.241832431Z tar: Removing leading `/' from member names
2026-04-16T19:00:09.387161213Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-16T19:00:09.536478106Z Wrote inspect data to must-gather.
2026-04-16T19:00:09.889079686Z Wrote inspect data to must-gather.
2026-04-16T19:00:09.920194521Z INFO: Getting status from alertmanager-main-0
2026-04-16T19:00:09.954045513Z INFO: Waiting for network log collection to complete ...
2026-04-16T19:00:09.954180951Z INFO: Waiting for ovnk database copies to complete ...
2026-04-16T19:00:09.954710085Z INFO: Copying ovnk databases complete.
2026-04-16T19:00:09.957066739Z 4.8M	must-gather/network_logs/ovnk_database_store
2026-04-16T19:00:09.962204421Z ovnk_database_store/
2026-04-16T19:00:09.962273137Z ovnk_database_store/ovnkube-node-tchmw_nbdb
2026-04-16T19:00:09.977162992Z ovnk_database_store/ovnkube-node-6nzqh_sbdb
2026-04-16T19:00:10.005941329Z ovnk_database_store/ovnkube-node-78v92_sbdb
2026-04-16T19:00:10.039474987Z ovnk_database_store/ovnkube-node-tchmw_sbdb
2026-04-16T19:00:10.076055106Z ovnk_database_store/ovnkube-node-6nzqh_nbdb
2026-04-16T19:00:10.088920635Z ovnk_database_store/ovnkube-node-78v92_nbdb
2026-04-16T19:00:10.103883492Z INFO: Network log collection complete.
2026-04-16T19:00:10.197440829Z Wrote inspect data to must-gather.
2026-04-16T19:00:10.548469939Z Wrote inspect data to must-gather.
2026-04-16T19:00:10.629619185Z Gathering data for ns/openshift-network-console...
2026-04-16T19:00:10.863061767Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-16T19:00:10.867591649Z INFO: Fallback to identify the container image from release info
2026-04-16T19:00:11.146654361Z Gathering data for ns/openshift-console-operator...
2026-04-16T19:00:11.209381794Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-04-16T19:00:11.386393400Z Wrote inspect data to must-gather.
2026-04-16T19:00:11.575610452Z Gathering data for ns/openshift-console...
2026-04-16T19:00:11.577009667Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc77fc5f1546189d6b57967ccc684e794bd846ea36c3e3c1e61d83a684f0f9b9
2026-04-16T19:00:11.715733460Z daemonset.apps/perf-node-gather-daemonset created
2026-04-16T19:00:11.844131664Z Waiting for performance profile collector pods to become ready: 1
2026-04-16T19:00:12.068956746Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-16T19:00:12.487408131Z Gathering data for ns/openshift-dns-operator...
2026-04-16T19:00:12.819293990Z Gathering data for ns/openshift-dns...
2026-04-16T19:00:12.962549912Z Waiting for performance profile collector pods to become ready: 2
2026-04-16T19:00:13.478143930Z Gathering data for ns/openshift-image-registry...
2026-04-16T19:00:13.991887604Z Gathering data for ns/openshift-ingress-operator...
2026-04-16T19:00:14.113784974Z Waiting for performance profile collector pods to become ready: 3
2026-04-16T19:00:14.382931635Z Gathering data for ns/openshift-ingress...
2026-04-16T19:00:14.929112795Z Gathering data for ns/openshift-ingress-canary...
2026-04-16T19:00:15.256496614Z Waiting for performance profile collector pods to become ready: 4
2026-04-16T19:00:15.408439986Z Gathering data for ns/openshift-insights...
2026-04-16T19:00:16.214324302Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-04-16T19:00:16.380471354Z Waiting for performance profile collector pods to become ready: 5
2026-04-16T19:00:17.515001772Z Waiting for performance profile collector pods to become ready: 6
2026-04-16T19:00:18.221697779Z Gathering data for ns/openshift-lws-operator...
2026-04-16T19:00:18.667728013Z Waiting for performance profile collector pods to become ready: 7
2026-04-16T19:00:18.791642102Z Gathering data for ns/kserve...
2026-04-16T19:00:19.821796862Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-16T19:00:20.113598744Z Collecting performance related data for node ip-10-0-135-146.ec2.internal
2026-04-16T19:00:20.117925579Z Collecting performance related data for node ip-10-0-137-47.ec2.internal
2026-04-16T19:00:20.126068661Z Collecting performance related data for node ip-10-0-140-154.ec2.internal
2026-04-16T19:00:20.937860463Z Gathering data for ns/openshift-config...
2026-04-16T19:00:21.221614169Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-04-16T19:00:21.360529057Z Gathering data for ns/openshift-config-managed...
2026-04-16T19:00:21.946714840Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-16T19:00:22.251253804Z Gathering data for ns/openshift-kube-apiserver...
2026-04-16T19:00:22.565110713Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-16T19:00:22.945326802Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-16T19:00:23.330131211Z Gathering data for ns/openshift-kube-scheduler...
2026-04-16T19:00:23.652983201Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-16T19:00:24.014894296Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-16T19:00:24.376994836Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-16T19:00:24.743057406Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-16T19:00:25.326164671Z Gathering data for ns/openshift-multus...
2026-04-16T19:00:26.227129477Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-04-16T19:00:26.861638397Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-16T19:00:28.600103932Z Gathering data for ns/openshift-host-network...
2026-04-16T19:00:28.958693147Z Gathering data for ns/openshift-network-diagnostics...
2026-04-16T19:00:29.016588375Z Collecting kubelet logs for node ip-10-0-135-146.ec2.internal
2026-04-16T19:00:29.018378372Z Collecting kubelet logs for node ip-10-0-137-47.ec2.internal
2026-04-16T19:00:29.020495405Z Collecting kubelet logs for node ip-10-0-140-154.ec2.internal
2026-04-16T19:00:29.492965867Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-16T19:00:29.499124864Z INFO: Node performance data collection complete.
2026-04-16T19:00:29.630530662Z Gathering data for ns/openshift-network-node-identity...
2026-04-16T19:00:30.109138811Z Gathering data for ns/openshift-network-operator...
2026-04-16T19:00:30.512231235Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-16T19:00:30.819108348Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-16T19:00:31.231539877Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-04-16T19:00:31.294261764Z Gathering data for ns/openshift-apiserver-operator...
2026-04-16T19:00:31.654969049Z Gathering data for ns/openshift-apiserver...
2026-04-16T19:00:32.113530197Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-16T19:00:32.419422583Z Gathering data for ns/openshift-controller-manager...
2026-04-16T19:00:32.742935228Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-16T19:00:33.405661933Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-16T19:00:33.747705285Z Gathering data for ns/openshift-service-ca-operator...
2026-04-16T19:00:34.153516604Z Gathering data for ns/openshift-service-ca...
2026-04-16T19:00:34.561892187Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-16T19:00:35.630468772Z Wrote inspect data to must-gather.
2026-04-16T19:00:35.630513249Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T19:00:35.630513249Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-16T19:00:35.775753024Z error: the server doesn't have a resource type "clusters"
2026-04-16T19:00:35.950223109Z Caches written to disk