2026-04-21T15:25:42.351059553Z [disk usage checker] Started 2026-04-21T15:25:42.354138753Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-04-21T15:25:42.551066413Z Gathering data for ns/openshift-cluster-version... 2026-04-21T15:25:42.683317012Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-04-21T15:25:42.786706235Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-21T15:25:42.868355553Z Gathering data for ns/default... 2026-04-21T15:25:43.234436408Z Gathering data for ns/openshift... 2026-04-21T15:25:43.660731521Z Gathering data for ns/kube-system... 2026-04-21T15:25:44.356396950Z Gathering data for ns/openshift-etcd... 2026-04-21T15:25:44.561837857Z Waiting on subprocesses to finish execution. 2026-04-21T15:25:44.570251821Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-04-21T15:25:44.571809955Z INFO: Collecting host service logs for crio 2026-04-21T15:25:44.572072113Z INFO: Collecting host service logs for kubelet 2026-04-21T15:25:44.572369462Z INFO: Collecting host service logs for rpm-ostreed 2026-04-21T15:25:44.572641566Z INFO: Collecting host service logs for ostree-finalize-staged 2026-04-21T15:25:44.572988572Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-04-21T15:25:44.573334728Z INFO: Collecting host service logs for machine-config-daemon-host 2026-04-21T15:25:44.573665247Z INFO: Collecting host service logs for NetworkManager 2026-04-21T15:25:44.574074949Z INFO: Collecting host service logs for openvswitch 2026-04-21T15:25:44.574409376Z INFO: Collecting host service logs for ovs-configuration 2026-04-21T15:25:44.574767067Z INFO: Collecting host service logs for ovsdb-server 2026-04-21T15:25:44.575225311Z INFO: Collecting host service logs for ovs-vswitchd 2026-04-21T15:25:44.575656726Z INFO: Waiting for worker host service log collection to complete ... 2026-04-21T15:25:44.578315023Z INFO: Gathering HAProxy config files 2026-04-21T15:25:44.593859217Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-04-21T15:25:44.602347017Z INFO: Gathering machine config daemon's old logs from all nodes 2026-04-21T15:25:44.906785445Z Wrote inspect data to must-gather. 2026-04-21T15:25:44.907145367Z error: inspection completed with the errors occurred while gathering data: 2026-04-21T15:25:44.907145367Z namespaces "assisted-installer" not found 2026-04-21T15:25:45.080997104Z INFO: Waiting for node performance related collection to complete ... 2026-04-21T15:25:45.176750325Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-04-21T15:25:45.176750325Z INFO: on-disk MachineConfig config collection complete. 2026-04-21T15:25:45.186070301Z No resources found 2026-04-21T15:25:45.241337494Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-04-21T15:25:45.243562919Z INFO: Found 1 replicas - prometheus-k8s-0 2026-04-21T15:25:45.504999650Z INFO: "sriov-network-operator" not detected. Skipping. 2026-04-21T15:25:45.524066348Z No resources found in openshift-etcd namespace. 2026-04-21T15:25:45.537096619Z INFO: "metallb-operator" not detected. Skipping. 2026-04-21T15:25:45.549127685Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-n8zqd 2026-04-21T15:25:45.549127685Z insights-runtime-extractor-46kkk 2026-04-21T15:25:45.549127685Z insights-runtime-extractor-8jpdv 2026-04-21T15:25:45.549127685Z insights-runtime-extractor-98544 2026-04-21T15:25:45.551544673Z Wrote inspect data to must-gather. 2026-04-21T15:25:45.621261717Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-04-21T15:25:45.788657914Z error: the server doesn't have a resource type "performanceprofile" 2026-04-21T15:25:45.925938689Z INFO: Worker host service log collection to complete. 2026-04-21T15:25:45.930102321Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-04-21T15:25:46.018968131Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-04-21T15:25:46.018968131Z See 'oc rsync -h' for help and examples 2026-04-21T15:25:46.340413701Z Wrote inspect data to must-gather. 2026-04-21T15:25:46.401985166Z ERROR: No running kube-apiserver pods found 2026-04-21T15:25:46.601541585Z Wrote inspect data to must-gather. 2026-04-21T15:25:46.602055737Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-04-21T15:25:46.666937190Z INFO: Waiting for HAProxy config collection to complete ... 2026-04-21T15:25:46.686240285Z No resources found 2026-04-21T15:25:46.802144176Z Wrote inspect data to must-gather. 2026-04-21T15:25:46.809056847Z Gathering data for ns/openshift-monitoring... 2026-04-21T15:25:46.823117682Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-21T15:25:46.846527385Z INFO: 'previous-logs' folder not found on ip-10-0-131-11.ec2.internal, skipping... 2026-04-21T15:25:46.852422299Z Wrote inspect data to must-gather. 2026-04-21T15:25:46.855630744Z Wrote inspect data to must-gather. 2026-04-21T15:25:46.889457405Z Wrote inspect data to must-gather. 2026-04-21T15:25:46.980912092Z INFO: Getting rules from prometheus-k8s-0 2026-04-21T15:25:47.009372370Z tar: Removing leading `/' from member names 2026-04-21T15:25:47.015649675Z INFO: HAProxy config collection complete. 2026-04-21T15:25:47.045019468Z error: the server doesn't have a resource type "machineconfigs" 2026-04-21T15:25:47.123208137Z Wrote inspect data to must-gather. 2026-04-21T15:25:47.123935926Z INFO: 'previous-logs' folder not found on ip-10-0-137-168.ec2.internal, skipping... 2026-04-21T15:25:47.259634527Z Wrote inspect data to must-gather. 2026-04-21T15:25:47.353553461Z Wrote inspect data to must-gather. 2026-04-21T15:25:47.359788077Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-04-21T15:25:47.388808751Z INFO: 'previous-logs' folder not found on ip-10-0-143-120.ec2.internal, skipping... 2026-04-21T15:25:47.388849872Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-04-21T15:25:47.388886054Z INFO: Machine Config Daemon termination log collection complete. 2026-04-21T15:25:47.413506607Z INFO: Getting status/config from prometheus-k8s-0 2026-04-21T15:25:47.468149530Z Wrote inspect data to must-gather. 2026-04-21T15:25:47.597628798Z Wrote inspect data to must-gather. 2026-04-21T15:25:47.612492020Z error: the server doesn't have a resource type "kubeletconfigs" 2026-04-21T15:25:47.787041996Z INFO: Getting status/flags from prometheus-k8s-0 2026-04-21T15:25:47.814742093Z Wrote inspect data to must-gather. 2026-04-21T15:25:47.861231272Z INFO: INTERCONNECT MODE 2026-04-21T15:25:47.861274134Z INFO: Gathering ovn-kubernetes DBs 2026-04-21T15:25:48.002087174Z Wrote inspect data to must-gather. 2026-04-21T15:25:48.008376499Z INFO: Gathering OVN_Northbound from ovnkube-node-bf2p7... 2026-04-21T15:25:48.008562969Z INFO: Gathering OVN_Northbound from ovnkube-node-hj57q... 2026-04-21T15:25:48.008791236Z INFO: Gathering OVN_Northbound from ovnkube-node-mbnjg... 2026-04-21T15:25:48.009084199Z INFO: Gathering OVN_Southbound from ovnkube-node-bf2p7... 2026-04-21T15:25:48.009306267Z INFO: Gathering OVN_Southbound from ovnkube-node-hj57q... 2026-04-21T15:25:48.009656353Z INFO: Gathering OVN_Southbound from ovnkube-node-mbnjg... 2026-04-21T15:25:48.013881087Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-04-21T15:25:48.312186123Z tar: Removing leading `/' from member names 2026-04-21T15:25:48.312820278Z Wrote inspect data to must-gather. 2026-04-21T15:25:48.319379490Z tar: Removing leading `/' from member names 2026-04-21T15:25:48.327680134Z tar: Removing leading `/' from member names 2026-04-21T15:25:48.328501522Z tar: Removing leading `/' from member names 2026-04-21T15:25:48.345909935Z tar: Removing leading `/' from member names 2026-04-21T15:25:48.358142468Z tar: Removing leading `/' from member names 2026-04-21T15:25:48.387073955Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-04-21T15:25:48.547345981Z Wrote inspect data to must-gather. 2026-04-21T15:25:48.634482598Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-04-21T15:25:48.699185316Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-04-21T15:25:48.717493635Z INFO: Waiting for network log collection to complete ... 2026-04-21T15:25:48.717542369Z INFO: Waiting for ovnk database copies to complete ... 2026-04-21T15:25:48.718040215Z INFO: Copying ovnk databases complete. 2026-04-21T15:25:48.719497104Z 3.0M must-gather/network_logs/ovnk_database_store 2026-04-21T15:25:48.721629504Z ovnk_database_store/ 2026-04-21T15:25:48.721629504Z ovnk_database_store/ovnkube-node-bf2p7_sbdb 2026-04-21T15:25:48.740301921Z ovnk_database_store/ovnkube-node-bf2p7_nbdb 2026-04-21T15:25:48.740607265Z Wrote inspect data to must-gather. 2026-04-21T15:25:48.748731351Z ovnk_database_store/ovnkube-node-hj57q_nbdb 2026-04-21T15:25:48.758308014Z ovnk_database_store/ovnkube-node-mbnjg_sbdb 2026-04-21T15:25:48.782609477Z ovnk_database_store/ovnkube-node-hj57q_sbdb 2026-04-21T15:25:48.800977188Z ovnk_database_store/ovnkube-node-mbnjg_nbdb 2026-04-21T15:25:48.812407936Z INFO: Network log collection complete. 2026-04-21T15:25:48.940609916Z Wrote inspect data to must-gather. 2026-04-21T15:25:49.066695504Z INFO: Getting status from alertmanager-main-0 2026-04-21T15:25:49.122457772Z Wrote inspect data to must-gather. 2026-04-21T15:25:49.283024222Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-21T15:25:49.347577186Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-04-21T15:25:49.349628923Z INFO: Fallback to identify the container image from release info 2026-04-21T15:25:49.428046903Z Gathering data for ns/openshift-network-console... 2026-04-21T15:25:49.794334790Z Gathering data for ns/openshift-console-operator... 2026-04-21T15:25:49.927923767Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d 2026-04-21T15:25:50.051278395Z daemonset.apps/perf-node-gather-daemonset created 2026-04-21T15:25:50.169084744Z Waiting for performance profile collector pods to become ready: 1 2026-04-21T15:25:50.202820795Z Gathering data for ns/openshift-console... 2026-04-21T15:25:50.689619628Z Gathering data for ns/openshift-cluster-storage-operator... 2026-04-21T15:25:51.038271649Z Gathering data for ns/openshift-dns-operator... 2026-04-21T15:25:51.291126875Z Waiting for performance profile collector pods to become ready: 2 2026-04-21T15:25:51.308841608Z Gathering data for ns/openshift-dns... 2026-04-21T15:25:51.869587654Z Gathering data for ns/openshift-image-registry... 2026-04-21T15:25:52.289941705Z Gathering data for ns/openshift-ingress-operator... 2026-04-21T15:25:52.364250219Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-04-21T15:25:52.407638597Z Waiting for performance profile collector pods to become ready: 3 2026-04-21T15:25:52.593047812Z Gathering data for ns/openshift-ingress... 2026-04-21T15:25:52.934547697Z Gathering data for ns/openshift-ingress-canary... 2026-04-21T15:25:53.339343341Z Gathering data for ns/openshift-insights... 2026-04-21T15:25:53.512129973Z Waiting for performance profile collector pods to become ready: 4 2026-04-21T15:25:54.619802378Z Waiting for performance profile collector pods to become ready: 5 2026-04-21T15:25:55.238943081Z Gathering data for ns/openshift-jobset-operator... 2026-04-21T15:25:55.743010295Z Waiting for performance profile collector pods to become ready: 6 2026-04-21T15:25:56.075911069Z Gathering data for ns/openshift-config... 2026-04-21T15:25:56.309748734Z Gathering data for ns/openshift-config-managed... 2026-04-21T15:25:56.797135024Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-04-21T15:25:56.881951496Z Waiting for performance profile collector pods to become ready: 7 2026-04-21T15:25:57.107179421Z Gathering data for ns/openshift-kube-apiserver... 2026-04-21T15:25:57.368530085Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-04-21T15:25:57.385284148Z Gathering data for ns/openshift-kube-controller-manager... 2026-04-21T15:25:57.653786324Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-04-21T15:25:57.923803017Z Gathering data for ns/openshift-kube-scheduler... 2026-04-21T15:25:58.008317615Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-04-21T15:25:58.179594302Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-04-21T15:25:58.254496257Z Collecting performance related data for node ip-10-0-131-11.ec2.internal 2026-04-21T15:25:58.261321703Z Collecting performance related data for node ip-10-0-143-120.ec2.internal 2026-04-21T15:25:58.268299455Z Collecting performance related data for node ip-10-0-137-168.ec2.internal 2026-04-21T15:25:58.486571752Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-04-21T15:25:58.890974925Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-04-21T15:25:59.255058519Z Gathering data for ns/openshift-user-workload-monitoring... 2026-04-21T15:25:59.868353417Z Gathering data for ns/openshift-multus... 2026-04-21T15:26:01.571365537Z Gathering data for ns/openshift-ovn-kubernetes... 2026-04-21T15:26:02.373446236Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-04-21T15:26:03.203952712Z Gathering data for ns/openshift-host-network... 2026-04-21T15:26:03.572997918Z Gathering data for ns/openshift-network-diagnostics... 2026-04-21T15:26:04.148205467Z Gathering data for ns/openshift-network-node-identity... 2026-04-21T15:26:04.522125496Z Gathering data for ns/openshift-network-operator... 2026-04-21T15:26:04.973804935Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-04-21T15:26:05.346060161Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-04-21T15:26:05.817220595Z Gathering data for ns/openshift-apiserver-operator... 2026-04-21T15:26:05.848359375Z Collecting kubelet logs for node ip-10-0-131-11.ec2.internal 2026-04-21T15:26:05.850619146Z Collecting kubelet logs for node ip-10-0-143-120.ec2.internal 2026-04-21T15:26:05.852850777Z Collecting kubelet logs for node ip-10-0-137-168.ec2.internal 2026-04-21T15:26:06.160135174Z Gathering data for ns/openshift-apiserver... 2026-04-21T15:26:06.195824827Z daemonset.apps "perf-node-gather-daemonset" deleted 2026-04-21T15:26:06.202351781Z INFO: Node performance data collection complete. 2026-04-21T15:26:06.611696495Z Gathering data for ns/openshift-controller-manager-operator... 2026-04-21T15:26:06.890759256Z Gathering data for ns/openshift-controller-manager... 2026-04-21T15:26:07.201257406Z Gathering data for ns/openshift-cluster-samples-operator... 2026-04-21T15:26:07.378063188Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-04-21T15:26:07.909053470Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-04-21T15:26:08.250675765Z Gathering data for ns/openshift-service-ca-operator... 2026-04-21T15:26:08.562804201Z Gathering data for ns/openshift-service-ca... 2026-04-21T15:26:08.943332145Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-04-21T15:26:09.906201726Z Wrote inspect data to must-gather. 2026-04-21T15:26:09.906242367Z error: inspection completed with the errors occurred while gathering data: 2026-04-21T15:26:09.906242367Z skipping gathering secrets/support due to error: secrets "support" not found 2026-04-21T15:26:10.034427544Z error: the server doesn't have a resource type "clusters" 2026-04-21T15:26:10.133792958Z Caches written to disk