2026-04-16T04:54:57.781111238Z [disk usage checker] Started 2026-04-16T04:54:57.820747516Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-04-16T04:54:58.007052848Z Gathering data for ns/openshift-cluster-version... 2026-04-16T04:54:58.157553733Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-04-16T04:54:58.249156320Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-16T04:54:58.348936501Z Gathering data for ns/default... 2026-04-16T04:54:58.693569402Z Gathering data for ns/openshift... 2026-04-16T04:54:59.160995152Z Gathering data for ns/kube-system... 2026-04-16T04:54:59.866685004Z Gathering data for ns/openshift-etcd... 2026-04-16T04:55:00.223874656Z Wrote inspect data to must-gather. 2026-04-16T04:55:00.223992714Z error: inspection completed with the errors occurred while gathering data: 2026-04-16T04:55:00.223992714Z namespaces "assisted-installer" not found 2026-04-16T04:55:00.282701589Z Waiting on subprocesses to finish execution. 2026-04-16T04:55:00.289315444Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-04-16T04:55:00.297178178Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-04-16T04:55:00.298440392Z INFO: Gathering HAProxy config files 2026-04-16T04:55:00.309780301Z INFO: Gathering machine config daemon's old logs from all nodes 2026-04-16T04:55:00.318814023Z INFO: Collecting host service logs for crio 2026-04-16T04:55:00.319184447Z INFO: Collecting host service logs for kubelet 2026-04-16T04:55:00.319562926Z INFO: Collecting host service logs for rpm-ostreed 2026-04-16T04:55:00.319908806Z INFO: Collecting host service logs for ostree-finalize-staged 2026-04-16T04:55:00.320253478Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-04-16T04:55:00.320616970Z INFO: Collecting host service logs for machine-config-daemon-host 2026-04-16T04:55:00.320953531Z INFO: Collecting host service logs for NetworkManager 2026-04-16T04:55:00.321316601Z INFO: Collecting host service logs for openvswitch 2026-04-16T04:55:00.321659408Z INFO: Collecting host service logs for ovs-configuration 2026-04-16T04:55:00.322008739Z INFO: Collecting host service logs for ovsdb-server 2026-04-16T04:55:00.322372972Z INFO: Collecting host service logs for ovs-vswitchd 2026-04-16T04:55:00.322784915Z INFO: Waiting for worker host service log collection to complete ... 2026-04-16T04:55:00.354410321Z INFO: Waiting for node performance related collection to complete ... 2026-04-16T04:55:01.025232743Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-04-16T04:55:01.053130214Z No resources found in openshift-etcd namespace. 2026-04-16T04:55:01.321090812Z INFO: Collecting Insights Archives from insights-operator-5785d4fcdd-zwp94 2026-04-16T04:55:01.321146947Z insights-runtime-extractor-8kmvb 2026-04-16T04:55:01.321168074Z insights-runtime-extractor-9728b 2026-04-16T04:55:01.321188251Z insights-runtime-extractor-kmwkb 2026-04-16T04:55:01.447788702Z INFO: Waiting for HAProxy config collection to complete ... 2026-04-16T04:55:01.650286647Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-04-16T04:55:01.650382655Z INFO: on-disk MachineConfig config collection complete. 2026-04-16T04:55:01.699865706Z error: the server doesn't have a resource type "performanceprofile" 2026-04-16T04:55:01.723236499Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-04-16T04:55:02.014907565Z INFO: Found 1 replicas - prometheus-k8s-0 2026-04-16T04:55:02.199956073Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-04-16T04:55:02.199956073Z See 'oc rsync -h' for help and examples 2026-04-16T04:55:02.201672691Z No resources found 2026-04-16T04:55:02.234687711Z INFO: "metallb-operator" not detected. Skipping. 2026-04-16T04:55:02.296621897Z INFO: "sriov-network-operator" not detected. Skipping. 2026-04-16T04:55:02.323380054Z INFO: Worker host service log collection to complete. 2026-04-16T04:55:02.424193287Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-04-16T04:55:02.641027025Z tar: Removing leading `/' from member names 2026-04-16T04:55:02.649425276Z INFO: HAProxy config collection complete. 2026-04-16T04:55:02.659764592Z Wrote inspect data to must-gather. 2026-04-16T04:55:02.726395120Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-16T04:55:02.731180297Z Wrote inspect data to must-gather. 2026-04-16T04:55:02.733186710Z No resources found 2026-04-16T04:55:02.813680421Z Wrote inspect data to must-gather. 2026-04-16T04:55:02.815413192Z Wrote inspect data to must-gather. 2026-04-16T04:55:02.922344203Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-04-16T04:55:02.988717236Z ERROR: No running kube-apiserver pods found 2026-04-16T04:55:03.060776585Z Wrote inspect data to must-gather. 2026-04-16T04:55:03.111284329Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-04-16T04:55:03.177709822Z Wrote inspect data to must-gather. 2026-04-16T04:55:03.216618940Z error: the server doesn't have a resource type "machineconfigs" 2026-04-16T04:55:03.351413520Z INFO: 'previous-logs' folder not found on ip-10-0-133-103.ec2.internal, skipping... 2026-04-16T04:55:03.395052506Z Wrote inspect data to must-gather. 2026-04-16T04:55:03.494894557Z Gathering data for ns/kuadrant-system... 2026-04-16T04:55:03.570114565Z Wrote inspect data to must-gather. 2026-04-16T04:55:03.627846110Z INFO: Getting rules from prometheus-k8s-0 2026-04-16T04:55:03.761175601Z Wrote inspect data to must-gather. 2026-04-16T04:55:03.777974303Z INFO: 'previous-logs' folder not found on ip-10-0-133-81.ec2.internal, skipping... 2026-04-16T04:55:03.932458078Z Wrote inspect data to must-gather. 2026-04-16T04:55:04.184203986Z error: the server doesn't have a resource type "kubeletconfigs" 2026-04-16T04:55:04.194533525Z INFO: 'previous-logs' folder not found on ip-10-0-140-211.ec2.internal, skipping... 2026-04-16T04:55:04.194574274Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-04-16T04:55:04.194574274Z INFO: Machine Config Daemon termination log collection complete. 2026-04-16T04:55:04.197145083Z INFO: Getting status/config from prometheus-k8s-0 2026-04-16T04:55:04.199930543Z Wrote inspect data to must-gather. 2026-04-16T04:55:04.542255549Z Wrote inspect data to must-gather. 2026-04-16T04:55:04.620128671Z INFO: INTERCONNECT MODE 2026-04-16T04:55:04.620189859Z INFO: Gathering ovn-kubernetes DBs 2026-04-16T04:55:04.688159147Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-04-16T04:55:04.742263006Z INFO: Getting status/flags from prometheus-k8s-0 2026-04-16T04:55:04.852692036Z Wrote inspect data to must-gather. 2026-04-16T04:55:04.894521199Z INFO: Gathering OVN_Northbound from ovnkube-node-4bdrp... 2026-04-16T04:55:04.894929486Z INFO: Gathering OVN_Northbound from ovnkube-node-7vmzk... 2026-04-16T04:55:04.895237105Z INFO: Gathering OVN_Northbound from ovnkube-node-s79zz... 2026-04-16T04:55:04.895577449Z INFO: Gathering OVN_Southbound from ovnkube-node-4bdrp... 2026-04-16T04:55:04.895878446Z INFO: Gathering OVN_Southbound from ovnkube-node-7vmzk... 2026-04-16T04:55:04.896181868Z INFO: Gathering OVN_Southbound from ovnkube-node-s79zz... 2026-04-16T04:55:05.094399172Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-04-16T04:55:05.251202252Z tar: Removing leading `/' from member names 2026-04-16T04:55:05.280002156Z tar: Removing leading `/' from member names 2026-04-16T04:55:05.317851908Z Wrote inspect data to must-gather. 2026-04-16T04:55:05.322078797Z tar: Removing leading `/' from member names 2026-04-16T04:55:05.352643700Z tar: Removing leading `/' from member names 2026-04-16T04:55:05.369094358Z tar: Removing leading `/' from member names 2026-04-16T04:55:05.391359187Z tar: Removing leading `/' from member names 2026-04-16T04:55:05.487975947Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-16T04:55:05.499166902Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-04-16T04:55:05.742532391Z Wrote inspect data to must-gather. 2026-04-16T04:55:05.759374562Z Gathering data for ns/openshift-monitoring... 2026-04-16T04:55:05.800024040Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-04-16T04:55:06.112764542Z INFO: Waiting for network log collection to complete ... 2026-04-16T04:55:06.112811643Z INFO: Waiting for ovnk database copies to complete ... 2026-04-16T04:55:06.113352212Z INFO: Copying ovnk databases complete. 2026-04-16T04:55:06.115001574Z 4.2M must-gather/network_logs/ovnk_database_store 2026-04-16T04:55:06.117337830Z ovnk_database_store/ 2026-04-16T04:55:06.117506338Z ovnk_database_store/ovnkube-node-4bdrp_sbdb 2026-04-16T04:55:06.146998344Z ovnk_database_store/ovnkube-node-4bdrp_nbdb 2026-04-16T04:55:06.154090542Z Wrote inspect data to must-gather. 2026-04-16T04:55:06.160539862Z ovnk_database_store/ovnkube-node-7vmzk_nbdb 2026-04-16T04:55:06.172862427Z ovnk_database_store/ovnkube-node-s79zz_sbdb 2026-04-16T04:55:06.198619399Z ovnk_database_store/ovnkube-node-7vmzk_sbdb 2026-04-16T04:55:06.220668312Z ovnk_database_store/ovnkube-node-s79zz_nbdb 2026-04-16T04:55:06.234916032Z INFO: Network log collection complete. 2026-04-16T04:55:06.293173387Z INFO: Getting status from alertmanager-main-0 2026-04-16T04:55:06.488016970Z Wrote inspect data to must-gather. 2026-04-16T04:55:06.748336312Z Wrote inspect data to must-gather. 2026-04-16T04:55:06.999026248Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-04-16T04:55:07.001227081Z INFO: Fallback to identify the container image from release info 2026-04-16T04:55:07.544735489Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc77fc5f1546189d6b57967ccc684e794bd846ea36c3e3c1e61d83a684f0f9b9 2026-04-16T04:55:07.665284194Z daemonset.apps/perf-node-gather-daemonset created 2026-04-16T04:55:07.831905775Z Waiting for performance profile collector pods to become ready: 1 2026-04-16T04:55:07.938106862Z Gathering data for ns/openshift-network-console... 2026-04-16T04:55:07.996904143Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-04-16T04:55:08.426206523Z Gathering data for ns/openshift-console-operator... 2026-04-16T04:55:08.855900205Z Gathering data for ns/openshift-console... 2026-04-16T04:55:08.968054410Z Waiting for performance profile collector pods to become ready: 2 2026-04-16T04:55:09.374444346Z Gathering data for ns/openshift-cluster-storage-operator... 2026-04-16T04:55:09.690905282Z Wrote inspect data to must-gather. 2026-04-16T04:55:09.761541060Z Gathering data for ns/openshift-dns-operator... 2026-04-16T04:55:10.097250056Z Gathering data for ns/openshift-dns... 2026-04-16T04:55:10.099714624Z Waiting for performance profile collector pods to become ready: 3 2026-04-16T04:55:10.744318075Z Gathering data for ns/openshift-image-registry... 2026-04-16T04:55:11.234549070Z Waiting for performance profile collector pods to become ready: 4 2026-04-16T04:55:11.250599479Z Gathering data for ns/openshift-ingress-operator... 2026-04-16T04:55:11.605505580Z Gathering data for ns/openshift-ingress... 2026-04-16T04:55:12.328023577Z Gathering data for ns/openshift-ingress-canary... 2026-04-16T04:55:12.357918772Z Waiting for performance profile collector pods to become ready: 5 2026-04-16T04:55:12.780138504Z Gathering data for ns/openshift-insights... 2026-04-16T04:55:13.002185437Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-04-16T04:55:13.488781984Z Waiting for performance profile collector pods to become ready: 6 2026-04-16T04:55:14.609596483Z Waiting for performance profile collector pods to become ready: 7 2026-04-16T04:55:14.806947898Z Gathering data for ns/opendatahub... 2026-04-16T04:55:15.726756906Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-04-16T04:55:15.968182722Z Collecting performance related data for node ip-10-0-133-81.ec2.internal 2026-04-16T04:55:15.974852810Z Collecting performance related data for node ip-10-0-140-211.ec2.internal 2026-04-16T04:55:15.982403264Z Collecting performance related data for node ip-10-0-133-103.ec2.internal 2026-04-16T04:55:16.382451620Z Gathering data for ns/openshift-lws-operator... 2026-04-16T04:55:17.847866602Z Gathering data for ns/openshift-config... 2026-04-16T04:55:18.006815850Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-04-16T04:55:18.161235639Z Gathering data for ns/openshift-config-managed... 2026-04-16T04:55:18.728434872Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-04-16T04:55:19.048885762Z Gathering data for ns/openshift-kube-apiserver... 2026-04-16T04:55:19.380327214Z Gathering data for ns/openshift-kube-controller-manager... 2026-04-16T04:55:19.711545958Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-04-16T04:55:20.059696880Z Gathering data for ns/openshift-kube-scheduler... 2026-04-16T04:55:20.375600914Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-04-16T04:55:20.710995678Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-04-16T04:55:21.081350723Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-04-16T04:55:21.448709406Z Gathering data for ns/openshift-user-workload-monitoring... 2026-04-16T04:55:22.036889609Z Gathering data for ns/openshift-multus... 2026-04-16T04:55:23.011898825Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-04-16T04:55:23.517924494Z Gathering data for ns/openshift-ovn-kubernetes... 2026-04-16T04:55:24.446840958Z Collecting kubelet logs for node ip-10-0-133-81.ec2.internal 2026-04-16T04:55:24.448603262Z Collecting kubelet logs for node ip-10-0-140-211.ec2.internal 2026-04-16T04:55:24.450765131Z Collecting kubelet logs for node ip-10-0-133-103.ec2.internal 2026-04-16T04:55:24.821369048Z daemonset.apps "perf-node-gather-daemonset" deleted 2026-04-16T04:55:24.827620659Z INFO: Node performance data collection complete. 2026-04-16T04:55:25.155612215Z Gathering data for ns/openshift-host-network... 2026-04-16T04:55:25.511544046Z Gathering data for ns/openshift-network-diagnostics... 2026-04-16T04:55:26.069398379Z Gathering data for ns/openshift-network-node-identity... 2026-04-16T04:55:26.497577780Z Gathering data for ns/openshift-network-operator... 2026-04-16T04:55:26.901330273Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-04-16T04:55:27.178915720Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-04-16T04:55:27.649143739Z Gathering data for ns/openshift-apiserver-operator... 2026-04-16T04:55:27.969361425Z Gathering data for ns/openshift-apiserver... 2026-04-16T04:55:28.019819563Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-04-16T04:55:28.422585751Z Gathering data for ns/openshift-controller-manager-operator... 2026-04-16T04:55:28.708679419Z Gathering data for ns/openshift-controller-manager... 2026-04-16T04:55:29.009089394Z Gathering data for ns/openshift-cluster-samples-operator... 2026-04-16T04:55:29.667921942Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-04-16T04:55:30.013208738Z Gathering data for ns/openshift-service-ca-operator... 2026-04-16T04:55:30.355406083Z Gathering data for ns/openshift-service-ca... 2026-04-16T04:55:30.742613334Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-04-16T04:55:31.771637388Z Wrote inspect data to must-gather. 2026-04-16T04:55:31.771680526Z error: inspection completed with the errors occurred while gathering data: 2026-04-16T04:55:31.771680526Z skipping gathering secrets/support due to error: secrets "support" not found 2026-04-16T04:55:31.907127909Z error: the server doesn't have a resource type "clusters" 2026-04-16T04:55:32.111664103Z Caches written to disk