2026-05-20T22:25:53.262598318Z [disk usage checker] Started
2026-05-20T22:25:53.265945775Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-20T22:25:53.476896146Z Gathering data for ns/openshift-cluster-version...
2026-05-20T22:25:53.645757738Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-20T22:25:53.761591048Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-20T22:25:53.866377394Z Gathering data for ns/default...
2026-05-20T22:25:54.258958179Z Gathering data for ns/openshift...
2026-05-20T22:25:54.740783317Z Gathering data for ns/kube-system...
2026-05-20T22:25:55.539644813Z Gathering data for ns/openshift-etcd...
2026-05-20T22:25:55.705741739Z Waiting on subprocesses to finish execution.
2026-05-20T22:25:55.715980070Z INFO: Gathering HAProxy config files
2026-05-20T22:25:55.716882037Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-20T22:25:55.717939105Z INFO: Collecting host service logs for crio
2026-05-20T22:25:55.718232649Z INFO: Collecting host service logs for kubelet
2026-05-20T22:25:55.718539842Z INFO: Collecting host service logs for rpm-ostreed
2026-05-20T22:25:55.718824552Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-20T22:25:55.719309636Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-20T22:25:55.719482956Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-20T22:25:55.719791019Z INFO: Collecting host service logs for NetworkManager
2026-05-20T22:25:55.720059441Z INFO: Collecting host service logs for openvswitch
2026-05-20T22:25:55.720348769Z INFO: Collecting host service logs for ovs-configuration
2026-05-20T22:25:55.720630228Z INFO: Collecting host service logs for ovsdb-server
2026-05-20T22:25:55.720918368Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-20T22:25:55.721909371Z INFO: Waiting for worker host service log collection to complete ...
2026-05-20T22:25:55.723553849Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-20T22:25:55.731285663Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-20T22:25:55.999627131Z INFO: Waiting for node performance related collection to complete ...
2026-05-20T22:25:56.297113412Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-20T22:25:56.667600903Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-20T22:25:56.725412390Z No resources found
2026-05-20T22:25:56.775550218Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-20T22:25:56.798222793Z No resources found in openshift-etcd namespace.
2026-05-20T22:25:56.892512409Z INFO: "metallb-operator" not detected. Skipping.
2026-05-20T22:25:56.923350650Z Wrote inspect data to must-gather.
2026-05-20T22:25:56.923478667Z error: inspection completed with the errors occurred while gathering data:
2026-05-20T22:25:56.923478667Z     namespaces "assisted-installer" not found
2026-05-20T22:25:57.033598157Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-2fql2
2026-05-20T22:25:57.033645158Z insights-runtime-extractor-7rwgv
2026-05-20T22:25:57.033665131Z insights-runtime-extractor-g65q5
2026-05-20T22:25:57.033686202Z insights-runtime-extractor-wrb8d
2026-05-20T22:25:57.092840925Z INFO: Worker host service log collection to complete.
2026-05-20T22:25:57.121269545Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-20T22:25:57.121315015Z INFO: on-disk MachineConfig config collection complete.
2026-05-20T22:25:57.133409774Z Wrote inspect data to must-gather.
2026-05-20T22:25:57.244036219Z error: the server doesn't have a resource type "performanceprofile"
2026-05-20T22:25:57.287136376Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-20T22:25:57.500579247Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-20T22:25:57.514547961Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-20T22:25:57.514547961Z See 'oc rsync -h' for help and examples
2026-05-20T22:25:57.525174312Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-20T22:25:57.889869904Z ERROR: No running kube-apiserver pods found
2026-05-20T22:25:58.106848789Z Wrote inspect data to must-gather.
2026-05-20T22:25:58.198518210Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-20T22:25:58.337415429Z Wrote inspect data to must-gather.
2026-05-20T22:25:58.344899742Z No resources found
2026-05-20T22:25:58.356907201Z tar: Removing leading `/' from member names
2026-05-20T22:25:58.363487858Z INFO: HAProxy config collection complete.
2026-05-20T22:25:58.364199842Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-20T22:25:58.493685573Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-20T22:25:58.509202853Z Wrote inspect data to must-gather.
2026-05-20T22:25:58.525305855Z INFO: 'previous-logs' folder not found on ip-10-0-129-137.ec2.internal, skipping...
2026-05-20T22:25:58.567267027Z Wrote inspect data to must-gather.
2026-05-20T22:25:58.649270281Z Wrote inspect data to must-gather.
2026-05-20T22:25:58.723367272Z INFO: Getting rules from prometheus-k8s-0
2026-05-20T22:25:58.809621689Z Wrote inspect data to must-gather.
2026-05-20T22:25:58.854050065Z error: the server doesn't have a resource type "machineconfigs"
2026-05-20T22:25:58.889828676Z Gathering data for ns/kuadrant-system...
2026-05-20T22:25:58.903893113Z INFO: 'previous-logs' folder not found on ip-10-0-131-201.ec2.internal, skipping...
2026-05-20T22:25:59.195020507Z Wrote inspect data to must-gather.
2026-05-20T22:25:59.197898555Z Wrote inspect data to must-gather.
2026-05-20T22:25:59.239449163Z INFO: 'previous-logs' folder not found on ip-10-0-135-24.ec2.internal, skipping...
2026-05-20T22:25:59.239555195Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-20T22:25:59.239605743Z INFO: Machine Config Daemon termination log collection complete.
2026-05-20T22:25:59.251722540Z INFO: Getting status/config from prometheus-k8s-0
2026-05-20T22:25:59.534376832Z Wrote inspect data to must-gather.
2026-05-20T22:25:59.550821215Z Wrote inspect data to must-gather.
2026-05-20T22:25:59.750467450Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-20T22:25:59.773285940Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-20T22:25:59.938466209Z INFO: INTERCONNECT MODE
2026-05-20T22:25:59.938494315Z INFO: Gathering ovn-kubernetes DBs
2026-05-20T22:26:00.053058279Z Wrote inspect data to must-gather.
2026-05-20T22:26:00.054005862Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-20T22:26:00.143894040Z INFO: Gathering OVN_Northbound from ovnkube-node-dsx9x...
2026-05-20T22:26:00.144183340Z INFO: Gathering OVN_Northbound from ovnkube-node-jgks5...
2026-05-20T22:26:00.144465579Z INFO: Gathering OVN_Northbound from ovnkube-node-shgjs...
2026-05-20T22:26:00.144773098Z INFO: Gathering OVN_Southbound from ovnkube-node-dsx9x...
2026-05-20T22:26:00.145031303Z INFO: Gathering OVN_Southbound from ovnkube-node-jgks5...
2026-05-20T22:26:00.145322665Z INFO: Gathering OVN_Southbound from ovnkube-node-shgjs...
2026-05-20T22:26:00.428267312Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-20T22:26:00.489489092Z Wrote inspect data to must-gather.
2026-05-20T22:26:00.555025250Z tar: Removing leading `/' from member names
2026-05-20T22:26:00.611588840Z tar: Removing leading `/' from member names
2026-05-20T22:26:00.674215459Z tar: Removing leading `/' from member names
2026-05-20T22:26:00.680217879Z tar: Removing leading `/' from member names
2026-05-20T22:26:00.681823187Z tar: Removing leading `/' from member names
2026-05-20T22:26:00.683523049Z tar: Removing leading `/' from member names
2026-05-20T22:26:00.685000537Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-20T22:26:00.891721243Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-20T22:26:00.892233212Z Wrote inspect data to must-gather.
2026-05-20T22:26:01.211200050Z W0520 22:26:01.211158    1321 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-05-20T22:26:01.232650327Z Wrote inspect data to must-gather.
2026-05-20T22:26:01.317880376Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-20T22:26:01.432607479Z INFO: Waiting for network log collection to complete ...
2026-05-20T22:26:01.432607479Z INFO: Waiting for ovnk database copies to complete ...
2026-05-20T22:26:01.432767611Z INFO: Copying ovnk databases complete.
2026-05-20T22:26:01.438944421Z 4.5M	must-gather/network_logs/ovnk_database_store
2026-05-20T22:26:01.441934656Z ovnk_database_store/
2026-05-20T22:26:01.441981554Z ovnk_database_store/ovnkube-node-dsx9x_nbdb
2026-05-20T22:26:01.453860488Z INFO: Getting status from alertmanager-main-0
2026-05-20T22:26:01.460738508Z ovnk_database_store/ovnkube-node-shgjs_sbdb
2026-05-20T22:26:01.497300377Z ovnk_database_store/ovnkube-node-dsx9x_sbdb
2026-05-20T22:26:01.526519562Z ovnk_database_store/ovnkube-node-jgks5_nbdb
2026-05-20T22:26:01.539414321Z ovnk_database_store/ovnkube-node-jgks5_sbdb
2026-05-20T22:26:01.564510696Z ovnk_database_store/ovnkube-node-shgjs_nbdb
2026-05-20T22:26:01.582172151Z INFO: Network log collection complete.
2026-05-20T22:26:01.593529149Z Gathering data for ns/openshift-monitoring...
2026-05-20T22:26:01.665460500Z Wrote inspect data to must-gather.
2026-05-20T22:26:01.995256595Z Wrote inspect data to must-gather.
2026-05-20T22:26:02.266164000Z Wrote inspect data to must-gather.
2026-05-20T22:26:02.516573720Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-20T22:26:02.519465997Z INFO: Fallback to identify the container image from release info
2026-05-20T22:26:03.048333609Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda
2026-05-20T22:26:03.192536950Z daemonset.apps/perf-node-gather-daemonset created
2026-05-20T22:26:03.328505540Z Waiting for performance profile collector pods to become ready: 1
2026-05-20T22:26:03.371198407Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-20T22:26:03.844711293Z Gathering data for ns/openshift-network-console...
2026-05-20T22:26:04.396932873Z Gathering data for ns/openshift-console-operator...
2026-05-20T22:26:04.469069772Z Waiting for performance profile collector pods to become ready: 2
2026-05-20T22:26:04.887606337Z Gathering data for ns/openshift-console...
2026-05-20T22:26:05.284783405Z Wrote inspect data to must-gather.
2026-05-20T22:26:05.433394519Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-20T22:26:05.609808851Z Waiting for performance profile collector pods to become ready: 3
2026-05-20T22:26:05.919269128Z Gathering data for ns/openshift-dns-operator...
2026-05-20T22:26:06.292364149Z Gathering data for ns/openshift-dns...
2026-05-20T22:26:06.742245196Z Waiting for performance profile collector pods to become ready: 4
2026-05-20T22:26:07.137349399Z Gathering data for ns/openshift-image-registry...
2026-05-20T22:26:07.872298615Z Waiting for performance profile collector pods to become ready: 5
2026-05-20T22:26:07.897234564Z Gathering data for ns/openshift-ingress-operator...
2026-05-20T22:26:08.377680913Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-20T22:26:08.391909848Z Gathering data for ns/openshift-ingress...
2026-05-20T22:26:08.997024642Z Waiting for performance profile collector pods to become ready: 6
2026-05-20T22:26:09.329096198Z Gathering data for ns/openshift-ingress-canary...
2026-05-20T22:26:09.810032805Z Gathering data for ns/openshift-insights...
2026-05-20T22:26:10.103836317Z Waiting for performance profile collector pods to become ready: 7
2026-05-20T22:26:11.219277154Z Waiting for performance profile collector pods to become ready: 8
2026-05-20T22:26:12.038409642Z Gathering data for ns/opendatahub...
2026-05-20T22:26:12.329892037Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-20T22:26:12.589653450Z Collecting performance related data for node ip-10-0-135-24.ec2.internal
2026-05-20T22:26:12.600011956Z Collecting performance related data for node ip-10-0-129-137.ec2.internal
2026-05-20T22:26:12.607428278Z Collecting performance related data for node ip-10-0-131-201.ec2.internal
2026-05-20T22:26:13.383224011Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-20T22:26:14.008576898Z Gathering data for ns/openshift-lws-operator...
2026-05-20T22:26:15.418926748Z Gathering data for ns/openshift-config...
2026-05-20T22:26:15.832762776Z Gathering data for ns/openshift-config-managed...
2026-05-20T22:26:16.327542898Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-20T22:26:16.735487049Z Gathering data for ns/openshift-kube-apiserver...
2026-05-20T22:26:17.110521521Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-20T22:26:17.491970328Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-20T22:26:17.907028683Z Gathering data for ns/openshift-kube-scheduler...
2026-05-20T22:26:18.320697106Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-20T22:26:18.389068632Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-20T22:26:18.712304637Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-20T22:26:19.139933009Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-20T22:26:19.579385728Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-20T22:26:20.301177438Z Gathering data for ns/openshift-multus...
2026-05-20T22:26:21.056524226Z Collecting kubelet logs for node ip-10-0-135-24.ec2.internal
2026-05-20T22:26:21.058232302Z Collecting kubelet logs for node ip-10-0-129-137.ec2.internal
2026-05-20T22:26:21.059962063Z Collecting kubelet logs for node ip-10-0-131-201.ec2.internal
2026-05-20T22:26:21.402700822Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-20T22:26:21.408565828Z INFO: Node performance data collection complete.
2026-05-20T22:26:22.351398273Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-20T22:26:23.393860603Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-20T22:26:24.319220943Z Gathering data for ns/openshift-host-network...
2026-05-20T22:26:24.672914993Z Gathering data for ns/openshift-network-diagnostics...
2026-05-20T22:26:25.269821393Z Gathering data for ns/openshift-network-node-identity...
2026-05-20T22:26:25.728863706Z Gathering data for ns/openshift-network-operator...
2026-05-20T22:26:26.170011019Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-20T22:26:26.501912127Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-20T22:26:27.037484188Z Gathering data for ns/openshift-apiserver-operator...
2026-05-20T22:26:27.440561115Z Gathering data for ns/openshift-apiserver...
2026-05-20T22:26:27.932523286Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-20T22:26:28.273520548Z Gathering data for ns/openshift-controller-manager...
2026-05-20T22:26:28.398400101Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-20T22:26:28.588660056Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-20T22:26:29.321699395Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-20T22:26:29.795454573Z Gathering data for ns/openshift-service-ca-operator...
2026-05-20T22:26:30.343661552Z Gathering data for ns/openshift-service-ca...
2026-05-20T22:26:30.788483545Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-20T22:26:31.895329636Z Wrote inspect data to must-gather.
2026-05-20T22:26:31.895374071Z error: inspection completed with the errors occurred while gathering data:
2026-05-20T22:26:31.895374071Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-20T22:26:32.032424218Z error: the server doesn't have a resource type "clusters"
2026-05-20T22:26:32.234868813Z Caches written to disk