2026-05-25T01:10:16.149222824Z [disk usage checker] Started
2026-05-25T01:10:16.152576079Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-25T01:10:16.373201010Z Gathering data for ns/openshift-cluster-version...
2026-05-25T01:10:16.499625112Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-25T01:10:16.594103686Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-25T01:10:16.684658735Z Gathering data for ns/default...
2026-05-25T01:10:17.011411016Z Gathering data for ns/openshift...
2026-05-25T01:10:17.494959538Z Gathering data for ns/kube-system...
2026-05-25T01:10:18.154346405Z Gathering data for ns/openshift-etcd...
2026-05-25T01:10:18.499490905Z Wrote inspect data to must-gather.
2026-05-25T01:10:18.499579520Z error: inspection completed with the errors occurred while gathering data:
2026-05-25T01:10:18.499579520Z     namespaces "assisted-installer" not found
2026-05-25T01:10:18.577495276Z Waiting on subprocesses to finish execution.
2026-05-25T01:10:18.586865388Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-25T01:10:18.589306911Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-25T01:10:18.589806158Z INFO: Gathering HAProxy config files
2026-05-25T01:10:18.594822570Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-25T01:10:18.597324507Z INFO: Collecting host service logs for crio
2026-05-25T01:10:18.597741319Z INFO: Collecting host service logs for kubelet
2026-05-25T01:10:18.598093001Z INFO: Collecting host service logs for rpm-ostreed
2026-05-25T01:10:18.598411343Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-25T01:10:18.598761539Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-25T01:10:18.599263238Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-25T01:10:18.599621198Z INFO: Collecting host service logs for NetworkManager
2026-05-25T01:10:18.599945755Z INFO: Collecting host service logs for openvswitch
2026-05-25T01:10:18.600264100Z INFO: Collecting host service logs for ovs-configuration
2026-05-25T01:10:18.600799034Z INFO: Collecting host service logs for ovsdb-server
2026-05-25T01:10:18.601138150Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-25T01:10:18.601576634Z INFO: Waiting for worker host service log collection to complete ...
2026-05-25T01:10:18.729994290Z INFO: Waiting for node performance related collection to complete ...
2026-05-25T01:10:19.033086486Z No resources found in openshift-etcd namespace.
2026-05-25T01:10:19.216668359Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-25T01:10:19.216761143Z INFO: on-disk MachineConfig config collection complete.
2026-05-25T01:10:19.418069507Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-25T01:10:19.421480575Z No resources found
2026-05-25T01:10:19.619696951Z ERROR: No running kube-apiserver pods found
2026-05-25T01:10:19.706334820Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-25T01:10:19.898752789Z INFO: "metallb-operator" not detected. Skipping.
2026-05-25T01:10:20.015335246Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-25T01:10:20.018978358Z Wrote inspect data to must-gather.
2026-05-25T01:10:20.095569260Z INFO: Worker host service log collection to complete.
2026-05-25T01:10:20.138053421Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-25T01:10:20.142923903Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-25T01:10:20.157326964Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-n44jg
2026-05-25T01:10:20.157326964Z insights-runtime-extractor-9mvh7
2026-05-25T01:10:20.157326964Z insights-runtime-extractor-ddk6t
2026-05-25T01:10:20.157326964Z insights-runtime-extractor-w2xfc
2026-05-25T01:10:20.256958092Z error: the server doesn't have a resource type "performanceprofile"
2026-05-25T01:10:20.644068405Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-25T01:10:20.948020711Z Wrote inspect data to must-gather.
2026-05-25T01:10:20.955108538Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-25T01:10:20.987961591Z Gathering data for ns/openshift-monitoring...
2026-05-25T01:10:21.062271905Z No resources found
2026-05-25T01:10:21.081772091Z Wrote inspect data to must-gather.
2026-05-25T01:10:21.165651690Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-25T01:10:21.166163503Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-25T01:10:21.222682698Z INFO: 'previous-logs' folder not found on ip-10-0-128-11.ec2.internal, skipping...
2026-05-25T01:10:21.246100341Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-25T01:10:21.246100341Z See 'oc rsync -h' for help and examples
2026-05-25T01:10:21.263139354Z Wrote inspect data to must-gather.
2026-05-25T01:10:21.328275143Z Wrote inspect data to must-gather.
2026-05-25T01:10:21.335706280Z tar: Removing leading `/' from member names
2026-05-25T01:10:21.341245596Z INFO: HAProxy config collection complete.
2026-05-25T01:10:21.372782398Z Wrote inspect data to must-gather.
2026-05-25T01:10:21.498614435Z error: the server doesn't have a resource type "machineconfigs"
2026-05-25T01:10:21.635177274Z INFO: 'previous-logs' folder not found on ip-10-0-141-154.ec2.internal, skipping...
2026-05-25T01:10:21.653209189Z INFO: Getting rules from prometheus-k8s-0
2026-05-25T01:10:21.759791983Z Wrote inspect data to must-gather.
2026-05-25T01:10:21.854856119Z Wrote inspect data to must-gather.
2026-05-25T01:10:21.950564707Z INFO: 'previous-logs' folder not found on ip-10-0-143-117.ec2.internal, skipping...
2026-05-25T01:10:21.950611022Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-25T01:10:21.950611022Z INFO: Machine Config Daemon termination log collection complete.
2026-05-25T01:10:22.107201034Z Wrote inspect data to must-gather.
2026-05-25T01:10:22.120900707Z INFO: Getting status/config from prometheus-k8s-0
2026-05-25T01:10:22.153622105Z Wrote inspect data to must-gather.
2026-05-25T01:10:22.330605767Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-25T01:10:22.398979255Z Wrote inspect data to must-gather.
2026-05-25T01:10:22.625250359Z Wrote inspect data to must-gather.
2026-05-25T01:10:22.648146098Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-25T01:10:22.772936656Z INFO: INTERCONNECT MODE
2026-05-25T01:10:22.772987779Z INFO: Gathering ovn-kubernetes DBs
2026-05-25T01:10:22.793381366Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-25T01:10:22.929848077Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-25T01:10:22.964202869Z Wrote inspect data to must-gather.
2026-05-25T01:10:22.990178086Z INFO: Gathering OVN_Northbound from ovnkube-node-hqqzs...
2026-05-25T01:10:22.990470899Z INFO: Gathering OVN_Northbound from ovnkube-node-t4qz7...
2026-05-25T01:10:22.990735853Z INFO: Gathering OVN_Northbound from ovnkube-node-t5l7k...
2026-05-25T01:10:22.991010201Z INFO: Gathering OVN_Southbound from ovnkube-node-hqqzs...
2026-05-25T01:10:22.991266863Z INFO: Gathering OVN_Southbound from ovnkube-node-t4qz7...
2026-05-25T01:10:22.991527976Z INFO: Gathering OVN_Southbound from ovnkube-node-t5l7k...
2026-05-25T01:10:23.292964631Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-25T01:10:23.368815818Z tar: Removing leading `/' from member names
2026-05-25T01:10:23.396396507Z tar: Removing leading `/' from member names
2026-05-25T01:10:23.405692122Z tar: Removing leading `/' from member names
2026-05-25T01:10:23.424142407Z tar: Removing leading `/' from member names
2026-05-25T01:10:23.431694738Z tar: Removing leading `/' from member names
2026-05-25T01:10:23.433305040Z Wrote inspect data to must-gather.
2026-05-25T01:10:23.436781416Z tar: Removing leading `/' from member names
2026-05-25T01:10:23.609886595Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-25T01:10:23.673689846Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-25T01:10:23.848792968Z Wrote inspect data to must-gather.
2026-05-25T01:10:23.969090417Z Gathering data for ns/openshift-network-console...
2026-05-25T01:10:24.017450601Z INFO: Waiting for network log collection to complete ...
2026-05-25T01:10:24.017504256Z INFO: Waiting for ovnk database copies to complete ...
2026-05-25T01:10:24.017987053Z INFO: Copying ovnk databases complete.
2026-05-25T01:10:24.020161262Z 3.1M	must-gather/network_logs/ovnk_database_store
2026-05-25T01:10:24.023182117Z ovnk_database_store/
2026-05-25T01:10:24.023182117Z ovnk_database_store/ovnkube-node-hqqzs_nbdb
2026-05-25T01:10:24.043558859Z ovnk_database_store/ovnkube-node-t5l7k_nbdb
2026-05-25T01:10:24.053137245Z ovnk_database_store/ovnkube-node-hqqzs_sbdb
2026-05-25T01:10:24.079302967Z ovnk_database_store/ovnkube-node-t4qz7_sbdb
2026-05-25T01:10:24.103498454Z ovnk_database_store/ovnkube-node-t5l7k_sbdb
2026-05-25T01:10:24.109915284Z INFO: Getting status from alertmanager-main-0
2026-05-25T01:10:24.125984940Z ovnk_database_store/ovnkube-node-t4qz7_nbdb
2026-05-25T01:10:24.139710506Z INFO: Network log collection complete.
2026-05-25T01:10:24.226687635Z Wrote inspect data to must-gather.
2026-05-25T01:10:24.560874988Z Wrote inspect data to must-gather.
2026-05-25T01:10:24.584874116Z Gathering data for ns/openshift-console-operator...
2026-05-25T01:10:24.863588174Z Wrote inspect data to must-gather.
2026-05-25T01:10:25.107210934Z Gathering data for ns/openshift-console...
2026-05-25T01:10:25.153739265Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-25T01:10:25.156260604Z INFO: Fallback to identify the container image from release info
2026-05-25T01:10:25.668330662Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-25T01:10:25.710427775Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda
2026-05-25T01:10:25.852573450Z daemonset.apps/perf-node-gather-daemonset created
2026-05-25T01:10:25.994020206Z Waiting for performance profile collector pods to become ready: 1
2026-05-25T01:10:26.011269745Z Wrote inspect data to must-gather.
2026-05-25T01:10:26.112642484Z Gathering data for ns/openshift-dns-operator...
2026-05-25T01:10:26.176200089Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-25T01:10:26.445636681Z Gathering data for ns/openshift-dns...
2026-05-25T01:10:27.059191133Z Gathering data for ns/openshift-image-registry...
2026-05-25T01:10:27.116355992Z Waiting for performance profile collector pods to become ready: 2
2026-05-25T01:10:27.581080181Z Gathering data for ns/openshift-ingress-operator...
2026-05-25T01:10:27.959062653Z Gathering data for ns/openshift-ingress...
2026-05-25T01:10:28.239817803Z Waiting for performance profile collector pods to become ready: 3
2026-05-25T01:10:28.543219791Z Gathering data for ns/openshift-ingress-canary...
2026-05-25T01:10:28.993057951Z Gathering data for ns/openshift-insights...
2026-05-25T01:10:29.368598003Z Waiting for performance profile collector pods to become ready: 4
2026-05-25T01:10:30.479656581Z Waiting for performance profile collector pods to become ready: 5
2026-05-25T01:10:30.877844592Z Gathering data for ns/opendatahub...
2026-05-25T01:10:31.181525669Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-25T01:10:31.625547336Z Waiting for performance profile collector pods to become ready: 6
2026-05-25T01:10:32.036412051Z Gathering data for ns/openshift-lws-operator...
2026-05-25T01:10:32.756848398Z Waiting for performance profile collector pods to become ready: 7
2026-05-25T01:10:33.186259738Z Gathering data for ns/openshift-config...
2026-05-25T01:10:33.509067434Z Gathering data for ns/openshift-config-managed...
2026-05-25T01:10:33.885533862Z Waiting for performance profile collector pods to become ready: 8
2026-05-25T01:10:33.953934277Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-25T01:10:34.235241568Z Gathering data for ns/openshift-kube-apiserver...
2026-05-25T01:10:34.557232970Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-25T01:10:34.866400365Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-25T01:10:35.010821698Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-25T01:10:35.213756689Z Gathering data for ns/openshift-kube-scheduler...
2026-05-25T01:10:35.272001835Z Collecting performance related data for node ip-10-0-128-11.ec2.internal
2026-05-25T01:10:35.281989864Z Collecting performance related data for node ip-10-0-143-117.ec2.internal
2026-05-25T01:10:35.295406278Z Collecting performance related data for node ip-10-0-141-154.ec2.internal
2026-05-25T01:10:35.565506632Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-25T01:10:35.961899707Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-25T01:10:36.186238327Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-25T01:10:36.407411899Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-25T01:10:36.828134055Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-25T01:10:37.454570844Z Gathering data for ns/openshift-multus...
2026-05-25T01:10:39.200961801Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-25T01:10:40.703351560Z Gathering data for ns/openshift-host-network...
2026-05-25T01:10:41.053287583Z Gathering data for ns/openshift-network-diagnostics...
2026-05-25T01:10:41.193535493Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-25T01:10:41.597195606Z Gathering data for ns/openshift-network-node-identity...
2026-05-25T01:10:42.055866421Z Gathering data for ns/openshift-network-operator...
2026-05-25T01:10:42.466616943Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-25T01:10:42.791000571Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-25T01:10:43.270908991Z Gathering data for ns/openshift-apiserver-operator...
2026-05-25T01:10:43.587730810Z Gathering data for ns/openshift-apiserver...
2026-05-25T01:10:43.723380528Z Collecting kubelet logs for node ip-10-0-128-11.ec2.internal
2026-05-25T01:10:43.725133663Z Collecting kubelet logs for node ip-10-0-143-117.ec2.internal
2026-05-25T01:10:43.726971083Z Collecting kubelet logs for node ip-10-0-141-154.ec2.internal
2026-05-25T01:10:44.081844830Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-25T01:10:44.086618171Z INFO: Node performance data collection complete.
2026-05-25T01:10:44.130599567Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-25T01:10:44.420513018Z Gathering data for ns/openshift-controller-manager...
2026-05-25T01:10:44.738623932Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-25T01:10:45.380275301Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-25T01:10:45.722589612Z Gathering data for ns/openshift-service-ca-operator...
2026-05-25T01:10:46.036927276Z Gathering data for ns/openshift-service-ca...
2026-05-25T01:10:46.198499745Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-25T01:10:46.412028003Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-25T01:10:47.311407984Z Wrote inspect data to must-gather.
2026-05-25T01:10:47.311486779Z error: inspection completed with the errors occurred while gathering data:
2026-05-25T01:10:47.311486779Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-25T01:10:47.443362416Z error: the server doesn't have a resource type "clusters"
2026-05-25T01:10:47.601270537Z Caches written to disk