2026-06-02T03:05:07.871345996Z [disk usage checker] Started 2026-06-02T03:05:07.875101139Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-06-02T03:05:08.108869792Z Gathering data for ns/openshift-cluster-version... 2026-06-02T03:05:08.293975682Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-06-02T03:05:08.416170863Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-06-02T03:05:08.528335404Z Gathering data for ns/default... 2026-06-02T03:05:09.041533241Z Gathering data for ns/openshift... 2026-06-02T03:05:09.612177042Z Gathering data for ns/kube-system... 2026-06-02T03:05:10.356328840Z Waiting on subprocesses to finish execution. 2026-06-02T03:05:10.361122460Z INFO: Gathering HAProxy config files 2026-06-02T03:05:10.362215119Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-06-02T03:05:10.371170013Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-06-02T03:05:10.387623123Z INFO: Collecting host service logs for crio 2026-06-02T03:05:10.388035154Z INFO: Collecting host service logs for kubelet 2026-06-02T03:05:10.388429267Z INFO: Collecting host service logs for rpm-ostreed 2026-06-02T03:05:10.388796867Z INFO: Collecting host service logs for ostree-finalize-staged 2026-06-02T03:05:10.389138668Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-06-02T03:05:10.389484683Z INFO: Collecting host service logs for machine-config-daemon-host 2026-06-02T03:05:10.389819558Z INFO: Collecting host service logs for NetworkManager 2026-06-02T03:05:10.390167785Z INFO: Collecting host service logs for openvswitch 2026-06-02T03:05:10.390511976Z INFO: Collecting host service logs for ovs-configuration 2026-06-02T03:05:10.390853680Z INFO: Collecting host service logs for ovsdb-server 2026-06-02T03:05:10.391212274Z INFO: Collecting host service logs for ovs-vswitchd 2026-06-02T03:05:10.391744957Z INFO: Waiting for worker host service log collection to complete ... 2026-06-02T03:05:10.417462165Z INFO: Gathering machine config daemon's old logs from all nodes 2026-06-02T03:05:10.457897431Z INFO: Waiting for node performance related collection to complete ... 2026-06-02T03:05:10.831044180Z No resources found in openshift-etcd namespace. 2026-06-02T03:05:10.901324520Z No resources found 2026-06-02T03:05:10.949096429Z INFO: "sriov-network-operator" not detected. Skipping. 2026-06-02T03:05:11.068584245Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-06-02T03:05:11.137715630Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-06-02T03:05:11.178802689Z error: the server doesn't have a resource type "performanceprofile" 2026-06-02T03:05:11.241127865Z INFO: "metallb-operator" not detected. Skipping. 2026-06-02T03:05:11.371789891Z INFO: Found 1 replicas - prometheus-k8s-0 2026-06-02T03:05:11.743248294Z INFO: Collecting Insights Archives from insights-operator-788bf58df5-dt6tw 2026-06-02T03:05:11.743358593Z insights-runtime-extractor-knmcm 2026-06-02T03:05:11.743382468Z insights-runtime-extractor-lrjsv 2026-06-02T03:05:11.743403220Z insights-runtime-extractor-zqk28 2026-06-02T03:05:11.879595277Z INFO: Waiting for HAProxy config collection to complete ... 2026-06-02T03:05:12.372312267Z Wrote inspect data to must-gather. 2026-06-02T03:05:12.464051321Z No resources found 2026-06-02T03:05:12.475286675Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-06-02T03:05:12.503527322Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-06-02T03:05:12.503599300Z INFO: on-disk MachineConfig config collection complete. 2026-06-02T03:05:12.526776209Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-06-02T03:05:12.550855628Z INFO: Worker host service log collection to complete. 2026-06-02T03:05:12.559223981Z Wrote inspect data to must-gather. 2026-06-02T03:05:12.751389614Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-06-02T03:05:12.751389614Z See 'oc rsync -h' for help and examples 2026-06-02T03:05:12.801443923Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-06-02T03:05:12.837087477Z Gathering data for ns/openshift-etcd... 2026-06-02T03:05:12.900810312Z tar: Removing leading `/' from member names 2026-06-02T03:05:12.905220905Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-06-02T03:05:12.906026640Z INFO: HAProxy config collection complete. 2026-06-02T03:05:12.991497064Z Wrote inspect data to must-gather. 2026-06-02T03:05:13.006183396Z Wrote inspect data to must-gather. 2026-06-02T03:05:13.074958994Z Wrote inspect data to must-gather. 2026-06-02T03:05:13.111710747Z ERROR: No running kube-apiserver pods found 2026-06-02T03:05:13.181783872Z INFO: 'previous-logs' folder not found on ip-10-0-129-101.ec2.internal, skipping... 2026-06-02T03:05:13.203227670Z Wrote inspect data to must-gather. 2026-06-02T03:05:13.307505234Z Gathering data for ns/kuadrant-system... 2026-06-02T03:05:13.500369334Z error: the server doesn't have a resource type "machineconfigs" 2026-06-02T03:05:13.609992637Z INFO: Getting rules from prometheus-k8s-0 2026-06-02T03:05:13.644382312Z INFO: 'previous-logs' folder not found on ip-10-0-138-19.ec2.internal, skipping... 2026-06-02T03:05:13.653137740Z Wrote inspect data to must-gather. 2026-06-02T03:05:13.881752165Z Wrote inspect data to must-gather. 2026-06-02T03:05:13.881848617Z error: inspection completed with the errors occurred while gathering data: 2026-06-02T03:05:13.881848617Z namespaces "assisted-installer" not found 2026-06-02T03:05:13.955542009Z Wrote inspect data to must-gather. 2026-06-02T03:05:14.056451018Z INFO: 'previous-logs' folder not found on ip-10-0-139-87.ec2.internal, skipping... 2026-06-02T03:05:14.056557796Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-06-02T03:05:14.056619206Z INFO: Machine Config Daemon termination log collection complete. 2026-06-02T03:05:14.175001482Z Wrote inspect data to must-gather. 2026-06-02T03:05:14.222701795Z INFO: Getting status/config from prometheus-k8s-0 2026-06-02T03:05:14.398689090Z Wrote inspect data to must-gather. 2026-06-02T03:05:14.557793115Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-06-02T03:05:14.606953503Z error: the server doesn't have a resource type "kubeletconfigs" 2026-06-02T03:05:14.652535757Z Wrote inspect data to must-gather. 2026-06-02T03:05:14.788643914Z INFO: Getting status/flags from prometheus-k8s-0 2026-06-02T03:05:15.000308788Z Wrote inspect data to must-gather. 2026-06-02T03:05:15.041342966Z INFO: INTERCONNECT MODE 2026-06-02T03:05:15.041376623Z INFO: Gathering ovn-kubernetes DBs 2026-06-02T03:05:15.106763795Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-06-02T03:05:15.173138837Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-06-02T03:05:15.258245054Z INFO: Gathering OVN_Northbound from ovnkube-node-hd4s2... 2026-06-02T03:05:15.258585843Z INFO: Gathering OVN_Northbound from ovnkube-node-pw6lq... 2026-06-02T03:05:15.258879820Z INFO: Gathering OVN_Northbound from ovnkube-node-ws7xp... 2026-06-02T03:05:15.259222686Z INFO: Gathering OVN_Southbound from ovnkube-node-hd4s2... 2026-06-02T03:05:15.259507054Z INFO: Gathering OVN_Southbound from ovnkube-node-pw6lq... 2026-06-02T03:05:15.259775063Z INFO: Gathering OVN_Southbound from ovnkube-node-ws7xp... 2026-06-02T03:05:15.571706133Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-06-02T03:05:15.573374906Z Wrote inspect data to must-gather. 2026-06-02T03:05:15.647794017Z Gathering data for ns/openshift-monitoring... 2026-06-02T03:05:15.684178993Z tar: Removing leading `/' from member names 2026-06-02T03:05:15.690400635Z tar: Removing leading `/' from member names 2026-06-02T03:05:15.731319440Z tar: Removing leading `/' from member names 2026-06-02T03:05:15.754776557Z tar: Removing leading `/' from member names 2026-06-02T03:05:15.758883613Z tar: Removing leading `/' from member names 2026-06-02T03:05:15.791953015Z tar: Removing leading `/' from member names 2026-06-02T03:05:16.003794417Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-06-02T03:05:16.076428275Z Wrote inspect data to must-gather. 2026-06-02T03:05:16.413238534Z INFO: Waiting for network log collection to complete ... 2026-06-02T03:05:16.413281353Z INFO: Waiting for ovnk database copies to complete ... 2026-06-02T03:05:16.413773247Z INFO: Copying ovnk databases complete. 2026-06-02T03:05:16.415855803Z 4.3M must-gather/network_logs/ovnk_database_store 2026-06-02T03:05:16.418270106Z ovnk_database_store/ 2026-06-02T03:05:16.418270106Z ovnk_database_store/ovnkube-node-ws7xp_nbdb 2026-06-02T03:05:16.431383548Z ovnk_database_store/ovnkube-node-pw6lq_sbdb 2026-06-02T03:05:16.459662946Z ovnk_database_store/ovnkube-node-pw6lq_nbdb 2026-06-02T03:05:16.470518608Z ovnk_database_store/ovnkube-node-hd4s2_nbdb 2026-06-02T03:05:16.483808123Z ovnk_database_store/ovnkube-node-hd4s2_sbdb 2026-06-02T03:05:16.498345693Z Wrote inspect data to must-gather. 2026-06-02T03:05:16.512562068Z ovnk_database_store/ovnkube-node-ws7xp_sbdb 2026-06-02T03:05:16.531804641Z INFO: Getting status from alertmanager-main-0 2026-06-02T03:05:16.542920170Z INFO: Network log collection complete. 2026-06-02T03:05:16.851530514Z Wrote inspect data to must-gather. 2026-06-02T03:05:17.180941039Z Wrote inspect data to must-gather. 2026-06-02T03:05:17.558128907Z Wrote inspect data to must-gather. 2026-06-02T03:05:17.878758489Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-06-02T03:05:17.881261267Z INFO: Fallback to identify the container image from release info 2026-06-02T03:05:17.911049883Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-06-02T03:05:18.598717363Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d8381cb85e479760bd6b601e111b39e6c56a632bfa298867f4197ec4421e9f5 2026-06-02T03:05:18.669440526Z Gathering data for ns/openshift-network-console... 2026-06-02T03:05:18.748807299Z daemonset.apps/perf-node-gather-daemonset created 2026-06-02T03:05:18.897868401Z Waiting for performance profile collector pods to become ready: 1 2026-06-02T03:05:19.350332494Z Gathering data for ns/openshift-console-operator... 2026-06-02T03:05:19.936460927Z Gathering data for ns/openshift-console... 2026-06-02T03:05:20.032727557Z Waiting for performance profile collector pods to become ready: 2 2026-06-02T03:05:20.470442021Z Wrote inspect data to must-gather. 2026-06-02T03:05:20.651759616Z Gathering data for ns/openshift-cluster-storage-operator... 2026-06-02T03:05:21.072397590Z Gathering data for ns/openshift-dns-operator... 2026-06-02T03:05:21.178933813Z Waiting for performance profile collector pods to become ready: 3 2026-06-02T03:05:21.459268646Z Gathering data for ns/openshift-dns... 2026-06-02T03:05:22.194934079Z Gathering data for ns/openshift-image-registry... 2026-06-02T03:05:22.292169419Z Waiting for performance profile collector pods to become ready: 4 2026-06-02T03:05:22.707675125Z Gathering data for ns/openshift-ingress-operator... 2026-06-02T03:05:22.915696515Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-06-02T03:05:23.133285416Z Gathering data for ns/openshift-ingress... 2026-06-02T03:05:23.415481274Z Waiting for performance profile collector pods to become ready: 5 2026-06-02T03:05:23.904023102Z Gathering data for ns/openshift-ingress-canary... 2026-06-02T03:05:24.410825822Z Gathering data for ns/openshift-insights... 2026-06-02T03:05:24.532929835Z Waiting for performance profile collector pods to become ready: 6 2026-06-02T03:05:25.645305864Z Waiting for performance profile collector pods to become ready: 7 2026-06-02T03:05:26.675117924Z Gathering data for ns/opendatahub... 2026-06-02T03:05:26.761449031Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-06-02T03:05:26.999339712Z Collecting performance related data for node ip-10-0-139-87.ec2.internal 2026-06-02T03:05:27.006206985Z Collecting performance related data for node ip-10-0-138-19.ec2.internal 2026-06-02T03:05:27.013212107Z Collecting performance related data for node ip-10-0-129-101.ec2.internal 2026-06-02T03:05:27.921658901Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-06-02T03:05:29.089679428Z Gathering data for ns/openshift-lws-operator... 2026-06-02T03:05:31.005945590Z Gathering data for ns/openshift-config... 2026-06-02T03:05:31.375833564Z Gathering data for ns/openshift-config-managed... 2026-06-02T03:05:31.921733161Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-06-02T03:05:32.297802778Z Gathering data for ns/openshift-kube-apiserver... 2026-06-02T03:05:32.668276956Z Gathering data for ns/openshift-kube-controller-manager... 2026-06-02T03:05:32.929427366Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-06-02T03:05:33.056242793Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-06-02T03:05:33.459393977Z Gathering data for ns/openshift-kube-scheduler... 2026-06-02T03:05:33.800438882Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-06-02T03:05:34.156825001Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-06-02T03:05:34.570936711Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-06-02T03:05:34.952881690Z Gathering data for ns/openshift-user-workload-monitoring... 2026-06-02T03:05:35.639950989Z Gathering data for ns/openshift-multus... 2026-06-02T03:05:35.843838370Z Collecting kubelet logs for node ip-10-0-139-87.ec2.internal 2026-06-02T03:05:35.845608522Z Collecting kubelet logs for node ip-10-0-138-19.ec2.internal 2026-06-02T03:05:35.847312366Z Collecting kubelet logs for node ip-10-0-129-101.ec2.internal 2026-06-02T03:05:36.181175108Z daemonset.apps "perf-node-gather-daemonset" deleted 2026-06-02T03:05:36.186970766Z INFO: Node performance data collection complete. 2026-06-02T03:05:37.306299637Z Gathering data for ns/openshift-ovn-kubernetes... 2026-06-02T03:05:37.935572340Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-06-02T03:05:39.199998855Z Gathering data for ns/openshift-host-network... 2026-06-02T03:05:39.578510684Z Gathering data for ns/openshift-network-diagnostics... 2026-06-02T03:05:40.253174043Z Gathering data for ns/openshift-network-node-identity... 2026-06-02T03:05:40.972289031Z Gathering data for ns/openshift-network-operator... 2026-06-02T03:05:41.456180880Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-06-02T03:05:41.828190076Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-06-02T03:05:42.375695631Z Gathering data for ns/openshift-apiserver-operator... 2026-06-02T03:05:42.724107874Z Gathering data for ns/openshift-apiserver... 2026-06-02T03:05:42.940461374Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-06-02T03:05:43.243049624Z Gathering data for ns/openshift-controller-manager-operator... 2026-06-02T03:05:43.749130088Z Gathering data for ns/openshift-controller-manager... 2026-06-02T03:05:44.135760853Z Gathering data for ns/openshift-cluster-samples-operator... 2026-06-02T03:05:44.844989579Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-06-02T03:05:45.180307945Z Gathering data for ns/openshift-service-ca-operator... 2026-06-02T03:05:45.542926767Z Gathering data for ns/openshift-service-ca... 2026-06-02T03:05:45.964369777Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-06-02T03:05:47.072017631Z Wrote inspect data to must-gather. 2026-06-02T03:05:47.072050468Z error: inspection completed with the errors occurred while gathering data: 2026-06-02T03:05:47.072050468Z skipping gathering secrets/support due to error: secrets "support" not found 2026-06-02T03:05:47.210918168Z error: the server doesn't have a resource type "clusters" 2026-06-02T03:05:47.400949170Z Caches written to disk