2026-04-16T20:59:56.162046674Z [disk usage checker] Started 2026-04-16T20:59:56.165374981Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-04-16T20:59:56.361827436Z Gathering data for ns/openshift-cluster-version... 2026-04-16T20:59:56.502143580Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-04-16T20:59:56.581475666Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-16T20:59:56.657836249Z Gathering data for ns/default... 2026-04-16T20:59:56.934232453Z Gathering data for ns/openshift... 2026-04-16T20:59:57.355538143Z Gathering data for ns/kube-system... 2026-04-16T20:59:58.035333858Z Gathering data for ns/openshift-etcd... 2026-04-16T20:59:58.353899479Z Wrote inspect data to must-gather. 2026-04-16T20:59:58.353931912Z error: inspection completed with the errors occurred while gathering data: 2026-04-16T20:59:58.353931912Z namespaces "assisted-installer" not found 2026-04-16T20:59:58.560878626Z Waiting on subprocesses to finish execution. 2026-04-16T20:59:58.563907077Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-04-16T20:59:58.572220110Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-04-16T20:59:58.577848608Z INFO: Gathering HAProxy config files 2026-04-16T20:59:58.580441130Z INFO: Gathering machine config daemon's old logs from all nodes 2026-04-16T20:59:58.583719146Z INFO: Collecting host service logs for crio 2026-04-16T20:59:58.584036851Z INFO: Collecting host service logs for kubelet 2026-04-16T20:59:58.584396322Z INFO: Collecting host service logs for rpm-ostreed 2026-04-16T20:59:58.584707273Z INFO: Collecting host service logs for ostree-finalize-staged 2026-04-16T20:59:58.586390380Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-04-16T20:59:58.586700091Z INFO: Collecting host service logs for machine-config-daemon-host 2026-04-16T20:59:58.587013152Z INFO: Collecting host service logs for NetworkManager 2026-04-16T20:59:58.587328318Z INFO: Collecting host service logs for openvswitch 2026-04-16T20:59:58.587682187Z INFO: Collecting host service logs for ovs-configuration 2026-04-16T20:59:58.588043483Z INFO: Collecting host service logs for ovsdb-server 2026-04-16T20:59:58.588375488Z INFO: Collecting host service logs for ovs-vswitchd 2026-04-16T20:59:58.588734362Z INFO: Waiting for worker host service log collection to complete ... 2026-04-16T20:59:58.596690939Z INFO: Waiting for node performance related collection to complete ... 2026-04-16T20:59:59.439736163Z No resources found 2026-04-16T20:59:59.476640074Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-04-16T20:59:59.476672018Z INFO: on-disk MachineConfig config collection complete. 2026-04-16T20:59:59.752354370Z Wrote inspect data to must-gather. 2026-04-16T20:59:59.764437677Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-04-16T20:59:59.840795093Z INFO: "metallb-operator" not detected. Skipping. 2026-04-16T20:59:59.897417152Z Wrote inspect data to must-gather. 2026-04-16T21:00:00.085036925Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-qlvhv 2026-04-16T21:00:00.085036925Z insights-runtime-extractor-bd6r6 2026-04-16T21:00:00.085036925Z insights-runtime-extractor-nbhmg 2026-04-16T21:00:00.085036925Z insights-runtime-extractor-nzqv4 2026-04-16T21:00:00.103615374Z No resources found in openshift-etcd namespace. 2026-04-16T21:00:00.124840285Z error: the server doesn't have a resource type "performanceprofile" 2026-04-16T21:00:00.288373713Z INFO: Found 1 replicas - prometheus-k8s-0 2026-04-16T21:00:00.293768639Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-04-16T21:00:00.328695731Z INFO: "sriov-network-operator" not detected. Skipping. 2026-04-16T21:00:00.400132670Z Wrote inspect data to must-gather. 2026-04-16T21:00:00.444294746Z ERROR: No running kube-apiserver pods found 2026-04-16T21:00:00.467401891Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-04-16T21:00:00.467401891Z See 'oc rsync -h' for help and examples 2026-04-16T21:00:00.475322124Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-16T21:00:00.516434513Z Wrote inspect data to must-gather. 2026-04-16T21:00:00.579989609Z Wrote inspect data to must-gather. 2026-04-16T21:00:00.607953205Z INFO: Worker host service log collection to complete. 2026-04-16T21:00:00.705443641Z INFO: Waiting for HAProxy config collection to complete ... 2026-04-16T21:00:00.713602068Z No resources found 2026-04-16T21:00:00.786044343Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-04-16T21:00:00.835375997Z INFO: 'previous-logs' folder not found on ip-10-0-129-41.ec2.internal, skipping... 2026-04-16T21:00:00.841279287Z Wrote inspect data to must-gather. 2026-04-16T21:00:00.844403294Z Wrote inspect data to must-gather. 2026-04-16T21:00:00.875458283Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-04-16T21:00:00.995565020Z tar: Removing leading `/' from member names 2026-04-16T21:00:01.003849956Z INFO: HAProxy config collection complete. 2026-04-16T21:00:01.027864570Z error: the server doesn't have a resource type "machineconfigs" 2026-04-16T21:00:01.123486510Z Wrote inspect data to must-gather. 2026-04-16T21:00:01.126083441Z INFO: 'previous-logs' folder not found on ip-10-0-134-158.ec2.internal, skipping... 2026-04-16T21:00:01.176702571Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-04-16T21:00:01.241604510Z Gathering data for ns/openshift-monitoring... 2026-04-16T21:00:01.265496242Z Wrote inspect data to must-gather. 2026-04-16T21:00:01.307729477Z INFO: Getting rules from prometheus-k8s-0 2026-04-16T21:00:01.385270019Z Wrote inspect data to must-gather. 2026-04-16T21:00:01.407803644Z INFO: 'previous-logs' folder not found on ip-10-0-135-182.ec2.internal, skipping... 2026-04-16T21:00:01.407838824Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-04-16T21:00:01.407851091Z INFO: Machine Config Daemon termination log collection complete. 2026-04-16T21:00:01.505425054Z Wrote inspect data to must-gather. 2026-04-16T21:00:01.619961214Z Wrote inspect data to must-gather. 2026-04-16T21:00:01.660625784Z error: the server doesn't have a resource type "kubeletconfigs" 2026-04-16T21:00:01.717967850Z INFO: Getting status/config from prometheus-k8s-0 2026-04-16T21:00:01.874158091Z Wrote inspect data to must-gather. 2026-04-16T21:00:01.905707908Z INFO: INTERCONNECT MODE 2026-04-16T21:00:01.905748021Z INFO: Gathering ovn-kubernetes DBs 2026-04-16T21:00:02.075653564Z INFO: Gathering OVN_Northbound from ovnkube-node-blr8v... 2026-04-16T21:00:02.076016152Z INFO: Gathering OVN_Northbound from ovnkube-node-kmsm8... 2026-04-16T21:00:02.076278564Z INFO: Gathering OVN_Northbound from ovnkube-node-wh5vc... 2026-04-16T21:00:02.076563676Z INFO: Gathering OVN_Southbound from ovnkube-node-blr8v... 2026-04-16T21:00:02.076808029Z INFO: Gathering OVN_Southbound from ovnkube-node-kmsm8... 2026-04-16T21:00:02.077206316Z INFO: Gathering OVN_Southbound from ovnkube-node-wh5vc... 2026-04-16T21:00:02.109526123Z Wrote inspect data to must-gather. 2026-04-16T21:00:02.195062661Z INFO: Getting status/flags from prometheus-k8s-0 2026-04-16T21:00:02.361921686Z tar: Removing leading `/' from member names 2026-04-16T21:00:02.365246065Z tar: Removing leading `/' from member names 2026-04-16T21:00:02.401645447Z tar: Removing leading `/' from member names 2026-04-16T21:00:02.401890325Z tar: Removing leading `/' from member names 2026-04-16T21:00:02.406406687Z tar: Removing leading `/' from member names 2026-04-16T21:00:02.435616176Z tar: Removing leading `/' from member names 2026-04-16T21:00:02.442595429Z Wrote inspect data to must-gather. 2026-04-16T21:00:02.504091172Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-04-16T21:00:02.689947473Z Wrote inspect data to must-gather. 2026-04-16T21:00:02.706683712Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-04-16T21:00:02.773402866Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-04-16T21:00:02.860151267Z INFO: Waiting for network log collection to complete ... 2026-04-16T21:00:02.860187499Z INFO: Waiting for ovnk database copies to complete ... 2026-04-16T21:00:02.860661890Z INFO: Copying ovnk databases complete. 2026-04-16T21:00:02.862270025Z 3.9M must-gather/network_logs/ovnk_database_store 2026-04-16T21:00:02.865233009Z ovnk_database_store/ 2026-04-16T21:00:02.865283806Z ovnk_database_store/ovnkube-node-blr8v_sbdb 2026-04-16T21:00:02.888969443Z ovnk_database_store/ovnkube-node-blr8v_nbdb 2026-04-16T21:00:02.898676814Z ovnk_database_store/ovnkube-node-kmsm8_sbdb 2026-04-16T21:00:02.913771521Z Wrote inspect data to must-gather. 2026-04-16T21:00:02.922655512Z ovnk_database_store/ovnkube-node-kmsm8_nbdb 2026-04-16T21:00:02.933894670Z ovnk_database_store/ovnkube-node-wh5vc_sbdb 2026-04-16T21:00:02.955132494Z ovnk_database_store/ovnkube-node-wh5vc_nbdb 2026-04-16T21:00:02.965883409Z INFO: Network log collection complete. 2026-04-16T21:00:03.031245816Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-04-16T21:00:03.141972327Z Wrote inspect data to must-gather. 2026-04-16T21:00:03.277346015Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-16T21:00:03.424363583Z Wrote inspect data to must-gather. 2026-04-16T21:00:03.439607951Z INFO: Getting status from alertmanager-main-0 2026-04-16T21:00:03.439943802Z Gathering data for ns/openshift-network-console... 2026-04-16T21:00:03.695372092Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-04-16T21:00:03.697870107Z INFO: Fallback to identify the container image from release info 2026-04-16T21:00:03.837285064Z Gathering data for ns/openshift-console-operator... 2026-04-16T21:00:04.152872275Z Gathering data for ns/openshift-console... 2026-04-16T21:00:04.535126794Z Gathering data for ns/openshift-cluster-storage-operator... 2026-04-16T21:00:04.854399497Z Gathering data for ns/openshift-dns-operator... 2026-04-16T21:00:05.093904324Z Gathering data for ns/openshift-dns... 2026-04-16T21:00:05.566717980Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d 2026-04-16T21:00:05.649254888Z Gathering data for ns/openshift-image-registry... 2026-04-16T21:00:05.694442050Z daemonset.apps/perf-node-gather-daemonset created 2026-04-16T21:00:05.818829915Z Waiting for performance profile collector pods to become ready: 1 2026-04-16T21:00:06.121839380Z Gathering data for ns/openshift-ingress-operator... 2026-04-16T21:00:06.182764327Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-04-16T21:00:06.462960625Z Gathering data for ns/openshift-ingress... 2026-04-16T21:00:06.763126298Z Gathering data for ns/openshift-ingress-canary... 2026-04-16T21:00:06.951157065Z Waiting for performance profile collector pods to become ready: 2 2026-04-16T21:00:07.131381365Z Gathering data for ns/openshift-insights... 2026-04-16T21:00:08.066563963Z Waiting for performance profile collector pods to become ready: 3 2026-04-16T21:00:09.192057104Z Waiting for performance profile collector pods to become ready: 4 2026-04-16T21:00:09.212083986Z Gathering data for ns/kserve... 2026-04-16T21:00:10.311439366Z Waiting for performance profile collector pods to become ready: 5 2026-04-16T21:00:10.993820917Z Gathering data for ns/openshift-config... 2026-04-16T21:00:11.187467305Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-04-16T21:00:11.276307781Z Gathering data for ns/openshift-config-managed... 2026-04-16T21:00:11.433441351Z Waiting for performance profile collector pods to become ready: 6 2026-04-16T21:00:11.677149477Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-04-16T21:00:11.955160150Z Gathering data for ns/openshift-kube-apiserver... 2026-04-16T21:00:12.193181002Z Gathering data for ns/openshift-kube-controller-manager... 2026-04-16T21:00:12.448983316Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-04-16T21:00:12.567390053Z Waiting for performance profile collector pods to become ready: 7 2026-04-16T21:00:12.729351537Z Gathering data for ns/openshift-kube-scheduler... 2026-04-16T21:00:12.966711045Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-04-16T21:00:13.238915567Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-04-16T21:00:13.543518326Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-04-16T21:00:13.703574357Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-04-16T21:00:13.855536391Z Gathering data for ns/openshift-user-workload-monitoring... 2026-04-16T21:00:13.981488170Z Collecting performance related data for node ip-10-0-134-158.ec2.internal 2026-04-16T21:00:13.988058751Z Collecting performance related data for node ip-10-0-135-182.ec2.internal 2026-04-16T21:00:14.000063345Z Collecting performance related data for node ip-10-0-129-41.ec2.internal 2026-04-16T21:00:14.436389379Z Gathering data for ns/openshift-multus... 2026-04-16T21:00:16.193525326Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-04-16T21:00:16.349277583Z Gathering data for ns/openshift-ovn-kubernetes... 2026-04-16T21:00:17.918235473Z Gathering data for ns/openshift-host-network... 2026-04-16T21:00:18.208858768Z Gathering data for ns/openshift-network-diagnostics... 2026-04-16T21:00:18.723905704Z Gathering data for ns/openshift-network-node-identity... 2026-04-16T21:00:19.135913214Z Gathering data for ns/openshift-network-operator... 2026-04-16T21:00:19.505680465Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-04-16T21:00:19.812965276Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-04-16T21:00:20.254495100Z Gathering data for ns/openshift-apiserver-operator... 2026-04-16T21:00:20.522749015Z Gathering data for ns/openshift-apiserver... 2026-04-16T21:00:20.871084708Z Gathering data for ns/openshift-controller-manager-operator... 2026-04-16T21:00:21.101788952Z Gathering data for ns/openshift-controller-manager... 2026-04-16T21:00:21.197820963Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-04-16T21:00:21.368734520Z Gathering data for ns/openshift-cluster-samples-operator... 2026-04-16T21:00:22.000804472Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-04-16T21:00:22.105623767Z Collecting kubelet logs for node ip-10-0-134-158.ec2.internal 2026-04-16T21:00:22.107229686Z Collecting kubelet logs for node ip-10-0-135-182.ec2.internal 2026-04-16T21:00:22.108855525Z Collecting kubelet logs for node ip-10-0-129-41.ec2.internal 2026-04-16T21:00:22.357259751Z Gathering data for ns/openshift-service-ca-operator... 2026-04-16T21:00:22.475374696Z daemonset.apps "perf-node-gather-daemonset" deleted 2026-04-16T21:00:22.479582280Z INFO: Node performance data collection complete. 2026-04-16T21:00:22.663243688Z Gathering data for ns/openshift-service-ca... 2026-04-16T21:00:22.987384085Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-04-16T21:00:23.820539676Z Wrote inspect data to must-gather. 2026-04-16T21:00:23.820585449Z error: inspection completed with the errors occurred while gathering data: 2026-04-16T21:00:23.820585449Z skipping gathering secrets/support due to error: secrets "support" not found 2026-04-16T21:00:23.949920108Z error: the server doesn't have a resource type "clusters" 2026-04-16T21:00:24.109600226Z Caches written to disk