2026-04-24T21:38:01.458136754Z [disk usage checker] Started 2026-04-24T21:38:01.461489383Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-04-24T21:38:01.660705250Z Gathering data for ns/openshift-cluster-version... 2026-04-24T21:38:01.803818908Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-04-24T21:38:01.886194145Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-24T21:38:01.963042989Z Gathering data for ns/default... 2026-04-24T21:38:02.242876030Z Gathering data for ns/openshift... 2026-04-24T21:38:02.667985541Z Gathering data for ns/kube-system... 2026-04-24T21:38:03.441797997Z Gathering data for ns/openshift-etcd... 2026-04-24T21:38:03.777954382Z Wrote inspect data to must-gather. 2026-04-24T21:38:03.777998690Z error: inspection completed with the errors occurred while gathering data: 2026-04-24T21:38:03.777998690Z namespaces "assisted-installer" not found 2026-04-24T21:38:03.855955315Z Waiting on subprocesses to finish execution. 2026-04-24T21:38:03.859602710Z INFO: Gathering HAProxy config files 2026-04-24T21:38:03.859788438Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-04-24T21:38:03.869444010Z INFO: Gathering machine config daemon's old logs from all nodes 2026-04-24T21:38:03.878210177Z INFO: Waiting for node performance related collection to complete ... 2026-04-24T21:38:03.889802879Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-04-24T21:38:03.908641676Z INFO: Collecting host service logs for crio 2026-04-24T21:38:03.909028021Z INFO: Collecting host service logs for kubelet 2026-04-24T21:38:03.909338817Z INFO: Collecting host service logs for rpm-ostreed 2026-04-24T21:38:03.909629031Z INFO: Collecting host service logs for ostree-finalize-staged 2026-04-24T21:38:03.909988962Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-04-24T21:38:03.914143659Z INFO: Collecting host service logs for machine-config-daemon-host 2026-04-24T21:38:03.914562472Z INFO: Collecting host service logs for NetworkManager 2026-04-24T21:38:03.914976327Z INFO: Collecting host service logs for openvswitch 2026-04-24T21:38:03.915344186Z INFO: Collecting host service logs for ovs-configuration 2026-04-24T21:38:03.915731467Z INFO: Collecting host service logs for ovsdb-server 2026-04-24T21:38:03.916126452Z INFO: Collecting host service logs for ovs-vswitchd 2026-04-24T21:38:03.916575117Z INFO: Waiting for worker host service log collection to complete ... 2026-04-24T21:38:04.905986220Z Wrote inspect data to must-gather. 2026-04-24T21:38:04.912922342Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-04-24T21:38:04.926136249Z INFO: Found 1 replicas - prometheus-k8s-0 2026-04-24T21:38:04.995982311Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-bm7ns 2026-04-24T21:38:04.995982311Z insights-runtime-extractor-6n8rw 2026-04-24T21:38:04.995982311Z insights-runtime-extractor-8928p 2026-04-24T21:38:04.995982311Z insights-runtime-extractor-l2wft 2026-04-24T21:38:05.077382412Z error: the server doesn't have a resource type "performanceprofile" 2026-04-24T21:38:05.114879422Z No resources found in openshift-etcd namespace. 2026-04-24T21:38:05.127824368Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-04-24T21:38:05.127824368Z INFO: on-disk MachineConfig config collection complete. 2026-04-24T21:38:05.223401697Z No resources found 2026-04-24T21:38:05.229215242Z ERROR: No running kube-apiserver pods found 2026-04-24T21:38:05.241728550Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-04-24T21:38:05.274445551Z INFO: "sriov-network-operator" not detected. Skipping. 2026-04-24T21:38:05.333280474Z INFO: Waiting for HAProxy config collection to complete ... 2026-04-24T21:38:05.443294212Z INFO: "metallb-operator" not detected. Skipping. 2026-04-24T21:38:05.544951703Z INFO: Worker host service log collection to complete. 2026-04-24T21:38:05.606980855Z Wrote inspect data to must-gather. 2026-04-24T21:38:05.902709472Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-04-24T21:38:05.920642133Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-04-24T21:38:05.967267288Z No resources found 2026-04-24T21:38:05.988128268Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-24T21:38:06.023018945Z Wrote inspect data to must-gather. 2026-04-24T21:38:06.033191818Z Wrote inspect data to must-gather. 2026-04-24T21:38:06.088213652Z INFO: 'previous-logs' folder not found on ip-10-0-132-118.ec2.internal, skipping... 2026-04-24T21:38:06.114096393Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-04-24T21:38:06.114096393Z See 'oc rsync -h' for help and examples 2026-04-24T21:38:06.173225558Z Wrote inspect data to must-gather. 2026-04-24T21:38:06.195431853Z Wrote inspect data to must-gather. 2026-04-24T21:38:06.196866861Z Wrote inspect data to must-gather. 2026-04-24T21:38:06.198800463Z tar: Removing leading `/' from member names 2026-04-24T21:38:06.206538528Z INFO: HAProxy config collection complete. 2026-04-24T21:38:06.259955628Z Gathering data for ns/openshift-monitoring... 2026-04-24T21:38:06.367030382Z error: the server doesn't have a resource type "machineconfigs" 2026-04-24T21:38:06.402205160Z INFO: 'previous-logs' folder not found on ip-10-0-132-219.ec2.internal, skipping... 2026-04-24T21:38:06.471009443Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-04-24T21:38:06.479626249Z Wrote inspect data to must-gather. 2026-04-24T21:38:06.528888316Z INFO: Getting rules from prometheus-k8s-0 2026-04-24T21:38:06.613462416Z Wrote inspect data to must-gather. 2026-04-24T21:38:06.681300551Z INFO: 'previous-logs' folder not found on ip-10-0-141-46.ec2.internal, skipping... 2026-04-24T21:38:06.681403469Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-04-24T21:38:06.681451949Z INFO: Machine Config Daemon termination log collection complete. 2026-04-24T21:38:06.746551221Z Wrote inspect data to must-gather. 2026-04-24T21:38:06.833239127Z Wrote inspect data to must-gather. 2026-04-24T21:38:06.945381346Z INFO: Getting status/config from prometheus-k8s-0 2026-04-24T21:38:06.988388133Z Wrote inspect data to must-gather. 2026-04-24T21:38:06.990696193Z error: the server doesn't have a resource type "kubeletconfigs" 2026-04-24T21:38:07.200035508Z Wrote inspect data to must-gather. 2026-04-24T21:38:07.277360099Z INFO: INTERCONNECT MODE 2026-04-24T21:38:07.277390306Z INFO: Gathering ovn-kubernetes DBs 2026-04-24T21:38:07.365004832Z INFO: Getting status/flags from prometheus-k8s-0 2026-04-24T21:38:07.417930717Z Wrote inspect data to must-gather. 2026-04-24T21:38:07.443993238Z INFO: Gathering OVN_Northbound from ovnkube-node-fsxch... 2026-04-24T21:38:07.444199245Z INFO: Gathering OVN_Northbound from ovnkube-node-g9z5w... 2026-04-24T21:38:07.444410997Z INFO: Gathering OVN_Northbound from ovnkube-node-pqvlx... 2026-04-24T21:38:07.444688198Z INFO: Gathering OVN_Southbound from ovnkube-node-fsxch... 2026-04-24T21:38:07.444940424Z INFO: Gathering OVN_Southbound from ovnkube-node-g9z5w... 2026-04-24T21:38:07.445211302Z INFO: Gathering OVN_Southbound from ovnkube-node-pqvlx... 2026-04-24T21:38:07.699522142Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-04-24T21:38:07.721994222Z Wrote inspect data to must-gather. 2026-04-24T21:38:07.784032944Z tar: Removing leading `/' from member names 2026-04-24T21:38:07.803647460Z tar: Removing leading `/' from member names 2026-04-24T21:38:07.820622268Z tar: Removing leading `/' from member names 2026-04-24T21:38:07.851006513Z tar: Removing leading `/' from member names 2026-04-24T21:38:07.853442727Z tar: Removing leading `/' from member names 2026-04-24T21:38:07.885675270Z tar: Removing leading `/' from member names 2026-04-24T21:38:07.960074550Z Wrote inspect data to must-gather. 2026-04-24T21:38:07.985309403Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-04-24T21:38:08.202659151Z Wrote inspect data to must-gather. 2026-04-24T21:38:08.263156488Z INFO: Waiting for network log collection to complete ... 2026-04-24T21:38:08.263202245Z INFO: Waiting for ovnk database copies to complete ... 2026-04-24T21:38:08.263669916Z INFO: Copying ovnk databases complete. 2026-04-24T21:38:08.265213029Z 3.3M must-gather/network_logs/ovnk_database_store 2026-04-24T21:38:08.267344239Z ovnk_database_store/ 2026-04-24T21:38:08.267344239Z ovnk_database_store/ovnkube-node-pqvlx_nbdb 2026-04-24T21:38:08.275816920Z ovnk_database_store/ovnkube-node-g9z5w_sbdb 2026-04-24T21:38:08.287723002Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-04-24T21:38:08.295536574Z ovnk_database_store/ovnkube-node-g9z5w_nbdb 2026-04-24T21:38:08.304846410Z ovnk_database_store/ovnkube-node-fsxch_sbdb 2026-04-24T21:38:08.312913425Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-04-24T21:38:08.327493113Z ovnk_database_store/ovnkube-node-fsxch_nbdb 2026-04-24T21:38:08.334792009Z ovnk_database_store/ovnkube-node-pqvlx_sbdb 2026-04-24T21:38:08.356076753Z INFO: Network log collection complete. 2026-04-24T21:38:08.417589914Z Wrote inspect data to must-gather. 2026-04-24T21:38:08.640238683Z Wrote inspect data to must-gather. 2026-04-24T21:38:08.687064754Z INFO: Getting status from alertmanager-main-0 2026-04-24T21:38:08.910978215Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-24T21:38:08.917094739Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-04-24T21:38:08.919322845Z INFO: Fallback to identify the container image from release info 2026-04-24T21:38:09.084347254Z Gathering data for ns/openshift-network-console... 2026-04-24T21:38:09.492070216Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d 2026-04-24T21:38:09.532241871Z Gathering data for ns/openshift-console-operator... 2026-04-24T21:38:09.618045979Z daemonset.apps/perf-node-gather-daemonset created 2026-04-24T21:38:09.748850820Z Waiting for performance profile collector pods to become ready: 1 2026-04-24T21:38:09.985654405Z Gathering data for ns/openshift-console... 2026-04-24T21:38:10.494420396Z Gathering data for ns/openshift-cluster-storage-operator... 2026-04-24T21:38:10.875352509Z Waiting for performance profile collector pods to become ready: 2 2026-04-24T21:38:10.980567751Z Gathering data for ns/openshift-dns-operator... 2026-04-24T21:38:11.285591499Z Gathering data for ns/openshift-dns... 2026-04-24T21:38:11.475336398Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-04-24T21:38:11.992509568Z Gathering data for ns/openshift-image-registry... 2026-04-24T21:38:12.008347719Z Waiting for performance profile collector pods to become ready: 3 2026-04-24T21:38:12.497117695Z Gathering data for ns/openshift-ingress-operator... 2026-04-24T21:38:12.868680871Z Gathering data for ns/openshift-ingress... 2026-04-24T21:38:13.127134587Z Waiting for performance profile collector pods to become ready: 4 2026-04-24T21:38:13.231546968Z Gathering data for ns/openshift-ingress-canary... 2026-04-24T21:38:13.761662546Z Gathering data for ns/openshift-insights... 2026-04-24T21:38:14.239933118Z Waiting for performance profile collector pods to become ready: 5 2026-04-24T21:38:15.367259626Z Waiting for performance profile collector pods to become ready: 6 2026-04-24T21:38:15.994172359Z Gathering data for ns/kserve... 2026-04-24T21:38:16.479775844Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-04-24T21:38:16.500504235Z Waiting for performance profile collector pods to become ready: 7 2026-04-24T21:38:17.641793367Z Gathering data for ns/openshift-config... 2026-04-24T21:38:17.647172556Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-04-24T21:38:17.912849540Z Collecting performance related data for node ip-10-0-132-118.ec2.internal 2026-04-24T21:38:17.921570480Z Collecting performance related data for node ip-10-0-132-219.ec2.internal 2026-04-24T21:38:17.931472273Z Gathering data for ns/openshift-config-managed... 2026-04-24T21:38:17.932096179Z Collecting performance related data for node ip-10-0-141-46.ec2.internal 2026-04-24T21:38:18.460026555Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-04-24T21:38:18.826614799Z Gathering data for ns/openshift-kube-apiserver... 2026-04-24T21:38:19.227334112Z Gathering data for ns/openshift-kube-controller-manager... 2026-04-24T21:38:19.596166424Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-04-24T21:38:19.922324380Z Gathering data for ns/openshift-kube-scheduler... 2026-04-24T21:38:20.213130535Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-04-24T21:38:20.533350843Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-04-24T21:38:20.878636738Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-04-24T21:38:21.240005527Z Gathering data for ns/openshift-user-workload-monitoring... 2026-04-24T21:38:21.486023609Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-04-24T21:38:21.732327125Z Gathering data for ns/openshift-multus... 2026-04-24T21:38:23.230668602Z Gathering data for ns/openshift-ovn-kubernetes... 2026-04-24T21:38:24.954247913Z Gathering data for ns/openshift-host-network... 2026-04-24T21:38:25.251513561Z Gathering data for ns/openshift-network-diagnostics... 2026-04-24T21:38:25.833041793Z Gathering data for ns/openshift-network-node-identity... 2026-04-24T21:38:26.236344307Z Gathering data for ns/openshift-network-operator... 2026-04-24T21:38:26.242649191Z Collecting kubelet logs for node ip-10-0-132-118.ec2.internal 2026-04-24T21:38:26.244625124Z Collecting kubelet logs for node ip-10-0-132-219.ec2.internal 2026-04-24T21:38:26.246430456Z Collecting kubelet logs for node ip-10-0-141-46.ec2.internal 2026-04-24T21:38:26.490544580Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-04-24T21:38:26.619727122Z daemonset.apps "perf-node-gather-daemonset" deleted 2026-04-24T21:38:26.625395785Z INFO: Node performance data collection complete. 2026-04-24T21:38:26.695547739Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-04-24T21:38:26.991114137Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-04-24T21:38:27.449903973Z Gathering data for ns/openshift-apiserver-operator... 2026-04-24T21:38:27.769618348Z Gathering data for ns/openshift-apiserver... 2026-04-24T21:38:28.180980878Z Gathering data for ns/openshift-controller-manager-operator... 2026-04-24T21:38:28.434163385Z Gathering data for ns/openshift-controller-manager... 2026-04-24T21:38:28.746248753Z Gathering data for ns/openshift-cluster-samples-operator... 2026-04-24T21:38:29.375495960Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-04-24T21:38:29.645842200Z Gathering data for ns/openshift-service-ca-operator... 2026-04-24T21:38:29.951205893Z Gathering data for ns/openshift-service-ca... 2026-04-24T21:38:30.321787289Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-04-24T21:38:31.193149734Z Wrote inspect data to must-gather. 2026-04-24T21:38:31.193188158Z error: inspection completed with the errors occurred while gathering data: 2026-04-24T21:38:31.193188158Z skipping gathering secrets/support due to error: secrets "support" not found 2026-04-24T21:38:31.319786873Z error: the server doesn't have a resource type "clusters" 2026-04-24T21:38:31.462033503Z Caches written to disk