2026-05-28T00:02:23.954577111Z [disk usage checker] Started
2026-05-28T00:02:23.958534545Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-05-28T00:02:24.148144304Z Gathering data for ns/openshift-cluster-version...
2026-05-28T00:02:24.275822702Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-28T00:02:24.365054430Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-28T00:02:24.443722863Z Gathering data for ns/default...
2026-05-28T00:02:24.728834467Z Gathering data for ns/openshift...
2026-05-28T00:02:25.133352212Z Gathering data for ns/kube-system...
2026-05-28T00:02:25.861315028Z Gathering data for ns/openshift-etcd...
2026-05-28T00:02:26.155591403Z Waiting on subprocesses to finish execution.
2026-05-28T00:02:26.159317227Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-28T00:02:26.162938762Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-28T00:02:26.163318408Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-28T00:02:26.180736181Z INFO: Collecting host service logs for crio
2026-05-28T00:02:26.181085389Z INFO: Collecting host service logs for kubelet
2026-05-28T00:02:26.181456087Z INFO: Collecting host service logs for rpm-ostreed
2026-05-28T00:02:26.181798068Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-28T00:02:26.182177472Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-28T00:02:26.182634635Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-28T00:02:26.182981863Z INFO: Collecting host service logs for NetworkManager
2026-05-28T00:02:26.183336399Z INFO: Collecting host service logs for openvswitch
2026-05-28T00:02:26.192622321Z INFO: Collecting host service logs for ovs-configuration
2026-05-28T00:02:26.193010390Z INFO: Collecting host service logs for ovsdb-server
2026-05-28T00:02:26.193398231Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-28T00:02:26.195640848Z INFO: Waiting for worker host service log collection to complete ...
2026-05-28T00:02:26.203112602Z INFO: Waiting for node performance related collection to complete ...
2026-05-28T00:02:26.217819362Z INFO: Gathering HAProxy config files
2026-05-28T00:02:26.273675864Z Wrote inspect data to must-gather.
2026-05-28T00:02:26.273725876Z error: inspection completed with the errors occurred while gathering data:
2026-05-28T00:02:26.273725876Z     namespaces "assisted-installer" not found
2026-05-28T00:02:26.509435971Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-28T00:02:26.605311240Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-28T00:02:26.810840193Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-28T00:02:26.985742520Z error: the server doesn't have a resource type "performanceprofile"
2026-05-28T00:02:27.034882799Z No resources found
2026-05-28T00:02:27.036159888Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-28T00:02:27.036235218Z INFO: on-disk MachineConfig config collection complete.
2026-05-28T00:02:27.106246342Z No resources found in openshift-etcd namespace.
2026-05-28T00:02:27.149599367Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-gtr8n
2026-05-28T00:02:27.149648254Z insights-runtime-extractor-pp5wm
2026-05-28T00:02:27.149669282Z insights-runtime-extractor-wbpzb
2026-05-28T00:02:27.149703761Z insights-runtime-extractor-wvmmg
2026-05-28T00:02:27.339404789Z ERROR: No running kube-apiserver pods found
2026-05-28T00:02:27.343933377Z Wrote inspect data to must-gather.
2026-05-28T00:02:27.367947677Z Wrote inspect data to must-gather.
2026-05-28T00:02:27.384560719Z INFO: "metallb-operator" not detected. Skipping.
2026-05-28T00:02:27.435989422Z INFO: Worker host service log collection to complete.
2026-05-28T00:02:27.486210472Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-28T00:02:27.486210472Z See 'oc rsync -h' for help and examples
2026-05-28T00:02:27.520295062Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-28T00:02:27.607932628Z Wrote inspect data to must-gather.
2026-05-28T00:02:27.737311743Z INFO: 'previous-logs' folder not found on ip-10-0-130-189.ec2.internal, skipping...
2026-05-28T00:02:28.048892842Z No resources found
2026-05-28T00:02:28.180641700Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-28T00:02:28.206749311Z Wrote inspect data to must-gather.
2026-05-28T00:02:28.214308066Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-28T00:02:28.256876087Z error: the server doesn't have a resource type "machineconfigs"
2026-05-28T00:02:28.261111566Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-28T00:02:28.291239132Z Wrote inspect data to must-gather.
2026-05-28T00:02:28.298818848Z Wrote inspect data to must-gather.
2026-05-28T00:02:28.302225704Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-28T00:02:28.381650501Z Wrote inspect data to must-gather.
2026-05-28T00:02:28.422828860Z Gathering data for ns/openshift-monitoring...
2026-05-28T00:02:28.500264653Z INFO: 'previous-logs' folder not found on ip-10-0-132-191.ec2.internal, skipping...
2026-05-28T00:02:28.556576302Z Wrote inspect data to must-gather.
2026-05-28T00:02:28.558230003Z Wrote inspect data to must-gather.
2026-05-28T00:02:28.616268250Z tar: Removing leading `/' from member names
2026-05-28T00:02:28.622984776Z INFO: HAProxy config collection complete.
2026-05-28T00:02:28.734866625Z INFO: Getting rules from prometheus-k8s-0
2026-05-28T00:02:28.759085964Z INFO: 'previous-logs' folder not found on ip-10-0-143-62.ec2.internal, skipping...
2026-05-28T00:02:28.759142513Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-28T00:02:28.759234126Z INFO: Machine Config Daemon termination log collection complete.
2026-05-28T00:02:28.768063671Z Wrote inspect data to must-gather.
2026-05-28T00:02:28.806506683Z Wrote inspect data to must-gather.
2026-05-28T00:02:28.907604649Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-28T00:02:28.963784562Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-05-28T00:02:29.031158433Z Wrote inspect data to must-gather.
2026-05-28T00:02:29.101876675Z Wrote inspect data to must-gather.
2026-05-28T00:02:29.103668926Z INFO: Getting status/config from prometheus-k8s-0
2026-05-28T00:02:29.280924672Z Wrote inspect data to must-gather.
2026-05-28T00:02:29.295450581Z INFO: INTERCONNECT MODE
2026-05-28T00:02:29.295481246Z INFO: Gathering ovn-kubernetes DBs
2026-05-28T00:02:29.450771641Z INFO: Gathering OVN_Northbound from ovnkube-node-dv6tg...
2026-05-28T00:02:29.450974430Z INFO: Gathering OVN_Northbound from ovnkube-node-khn4g...
2026-05-28T00:02:29.451186077Z INFO: Gathering OVN_Northbound from ovnkube-node-s7vdp...
2026-05-28T00:02:29.451507838Z INFO: Gathering OVN_Southbound from ovnkube-node-dv6tg...
2026-05-28T00:02:29.451757301Z INFO: Gathering OVN_Southbound from ovnkube-node-khn4g...
2026-05-28T00:02:29.452011873Z INFO: Gathering OVN_Southbound from ovnkube-node-s7vdp...
2026-05-28T00:02:29.480888902Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-28T00:02:29.481829542Z Wrote inspect data to must-gather.
2026-05-28T00:02:29.723374654Z tar: Removing leading `/' from member names
2026-05-28T00:02:29.723812271Z tar: Removing leading `/' from member names
2026-05-28T00:02:29.729109406Z tar: Removing leading `/' from member names
2026-05-28T00:02:29.738387036Z tar: Removing leading `/' from member names
2026-05-28T00:02:29.751788080Z tar: Removing leading `/' from member names
2026-05-28T00:02:29.760443738Z Wrote inspect data to must-gather.
2026-05-28T00:02:29.792076999Z tar: Removing leading `/' from member names
2026-05-28T00:02:29.840204803Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-28T00:02:29.996680000Z Wrote inspect data to must-gather.
2026-05-28T00:02:30.060802482Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-28T00:02:30.138977941Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-28T00:02:30.209180598Z INFO: Waiting for network log collection to complete ...
2026-05-28T00:02:30.209214435Z INFO: Waiting for ovnk database copies to complete ...
2026-05-28T00:02:30.209728448Z INFO: Copying ovnk databases complete.
2026-05-28T00:02:30.211338568Z 3.6M	must-gather/network_logs/ovnk_database_store
2026-05-28T00:02:30.213744394Z ovnk_database_store/
2026-05-28T00:02:30.213744394Z ovnk_database_store/ovnkube-node-dv6tg_nbdb
2026-05-28T00:02:30.215004492Z Wrote inspect data to must-gather.
2026-05-28T00:02:30.222994234Z ovnk_database_store/ovnkube-node-dv6tg_sbdb
2026-05-28T00:02:30.244779909Z ovnk_database_store/ovnkube-node-khn4g_nbdb
2026-05-28T00:02:30.256336938Z ovnk_database_store/ovnkube-node-s7vdp_nbdb
2026-05-28T00:02:30.265499204Z ovnk_database_store/ovnkube-node-khn4g_sbdb
2026-05-28T00:02:30.285244912Z ovnk_database_store/ovnkube-node-s7vdp_sbdb
2026-05-28T00:02:30.308800431Z INFO: Network log collection complete.
2026-05-28T00:02:30.370698924Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-28T00:02:30.419521117Z Wrote inspect data to must-gather.
2026-05-28T00:02:30.615253400Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-28T00:02:30.656083667Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-28T00:02:30.659021549Z INFO: Fallback to identify the container image from release info
2026-05-28T00:02:30.728362389Z INFO: Getting status from alertmanager-main-0
2026-05-28T00:02:30.791342699Z Gathering data for ns/openshift-network-console...
2026-05-28T00:02:31.183740102Z Gathering data for ns/openshift-console-operator...
2026-05-28T00:02:31.254317092Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda
2026-05-28T00:02:31.373145003Z daemonset.apps/perf-node-gather-daemonset created
2026-05-28T00:02:31.491218414Z Waiting for performance profile collector pods to become ready: 1
2026-05-28T00:02:31.579889571Z Gathering data for ns/openshift-console...
2026-05-28T00:02:32.009168039Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-28T00:02:32.361435261Z Gathering data for ns/openshift-dns-operator...
2026-05-28T00:02:32.611381507Z Waiting for performance profile collector pods to become ready: 2
2026-05-28T00:02:32.651330941Z Gathering data for ns/openshift-dns...
2026-05-28T00:02:33.236832289Z Gathering data for ns/openshift-image-registry...
2026-05-28T00:02:33.728476024Z Waiting for performance profile collector pods to become ready: 3
2026-05-28T00:02:33.762469286Z Gathering data for ns/openshift-ingress-operator...
2026-05-28T00:02:33.969112798Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-05-28T00:02:34.083969853Z Gathering data for ns/openshift-ingress...
2026-05-28T00:02:34.391905712Z Gathering data for ns/openshift-ingress-canary...
2026-05-28T00:02:34.785686616Z Gathering data for ns/openshift-insights...
2026-05-28T00:02:34.849408214Z Waiting for performance profile collector pods to become ready: 4
2026-05-28T00:02:35.963004128Z Waiting for performance profile collector pods to become ready: 5
2026-05-28T00:02:36.708264569Z Gathering data for ns/openshift-config...
2026-05-28T00:02:36.966279831Z Gathering data for ns/openshift-config-managed...
2026-05-28T00:02:37.080635416Z Waiting for performance profile collector pods to become ready: 6
2026-05-28T00:02:37.390229922Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-28T00:02:37.662349539Z Gathering data for ns/openshift-kube-apiserver...
2026-05-28T00:02:37.947843302Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-28T00:02:38.199426819Z Waiting for performance profile collector pods to become ready: 7
2026-05-28T00:02:38.254552931Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-28T00:02:38.605478736Z Gathering data for ns/openshift-kube-scheduler...
2026-05-28T00:02:38.907728371Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-28T00:02:38.973557129Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-05-28T00:02:39.197650533Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-28T00:02:39.320668934Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-28T00:02:39.546632324Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-28T00:02:39.561544127Z Collecting performance related data for node ip-10-0-132-191.ec2.internal
2026-05-28T00:02:39.569204305Z Collecting performance related data for node ip-10-0-130-189.ec2.internal
2026-05-28T00:02:39.576532738Z Collecting performance related data for node ip-10-0-143-62.ec2.internal
2026-05-28T00:02:39.899208849Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-28T00:02:40.510823197Z Gathering data for ns/openshift-multus...
2026-05-28T00:02:42.054322453Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-28T00:02:43.587838329Z Gathering data for ns/openshift-host-network...
2026-05-28T00:02:43.894335155Z Gathering data for ns/openshift-network-diagnostics...
2026-05-28T00:02:43.978024487Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-05-28T00:02:44.425264675Z Gathering data for ns/openshift-network-node-identity...
2026-05-28T00:02:44.837530121Z Gathering data for ns/openshift-network-operator...
2026-05-28T00:02:45.210742957Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-28T00:02:45.517360750Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-28T00:02:45.974603182Z Gathering data for ns/openshift-apiserver-operator...
2026-05-28T00:02:46.262084409Z Gathering data for ns/openshift-apiserver...
2026-05-28T00:02:46.698542683Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-28T00:02:46.961631068Z Gathering data for ns/openshift-controller-manager...
2026-05-28T00:02:47.256581702Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-28T00:02:47.341478802Z Collecting kubelet logs for node ip-10-0-132-191.ec2.internal
2026-05-28T00:02:47.343676812Z Collecting kubelet logs for node ip-10-0-130-189.ec2.internal
2026-05-28T00:02:47.346064860Z Collecting kubelet logs for node ip-10-0-143-62.ec2.internal
2026-05-28T00:02:47.688862494Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-28T00:02:47.693991878Z INFO: Node performance data collection complete.
2026-05-28T00:02:47.952443926Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-28T00:02:48.263344683Z Gathering data for ns/openshift-service-ca-operator...
2026-05-28T00:02:48.570713355Z Gathering data for ns/openshift-service-ca...
2026-05-28T00:02:48.937338073Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-28T00:02:48.982507414Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-05-28T00:02:49.793095909Z Wrote inspect data to must-gather.
2026-05-28T00:02:49.793148721Z error: inspection completed with the errors occurred while gathering data:
2026-05-28T00:02:49.793148721Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-28T00:02:49.919053716Z error: the server doesn't have a resource type "clusters"
2026-05-28T00:02:50.047526011Z Caches written to disk