2026-04-16T15:48:23.388100871Z [disk usage checker] Started 2026-04-16T15:48:23.422697761Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-04-16T15:48:23.617204732Z Gathering data for ns/openshift-cluster-version... 2026-04-16T15:48:23.768479440Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-04-16T15:48:23.865606517Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-16T15:48:23.962811850Z Gathering data for ns/default... 2026-04-16T15:48:24.299990764Z Gathering data for ns/openshift... 2026-04-16T15:48:24.757760023Z Gathering data for ns/kube-system... 2026-04-16T15:48:25.470971476Z Gathering data for ns/openshift-etcd... 2026-04-16T15:48:25.804570354Z Wrote inspect data to must-gather. 2026-04-16T15:48:25.804780712Z error: inspection completed with the errors occurred while gathering data: 2026-04-16T15:48:25.804780712Z namespaces "assisted-installer" not found 2026-04-16T15:48:25.906534730Z Waiting on subprocesses to finish execution. 2026-04-16T15:48:25.910967653Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-04-16T15:48:25.952578051Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-04-16T15:48:25.955683264Z INFO: Gathering machine config daemon's old logs from all nodes 2026-04-16T15:48:25.997360139Z INFO: Gathering HAProxy config files 2026-04-16T15:48:26.019658286Z INFO: Waiting for node performance related collection to complete ... 2026-04-16T15:48:26.066017018Z INFO: Collecting host service logs for crio 2026-04-16T15:48:26.066394926Z INFO: Collecting host service logs for kubelet 2026-04-16T15:48:26.066771096Z INFO: Collecting host service logs for rpm-ostreed 2026-04-16T15:48:26.067122116Z INFO: Collecting host service logs for ostree-finalize-staged 2026-04-16T15:48:26.067463802Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-04-16T15:48:26.067791044Z INFO: Collecting host service logs for machine-config-daemon-host 2026-04-16T15:48:26.068116122Z INFO: Collecting host service logs for NetworkManager 2026-04-16T15:48:26.068460629Z INFO: Collecting host service logs for openvswitch 2026-04-16T15:48:26.068787840Z INFO: Collecting host service logs for ovs-configuration 2026-04-16T15:48:26.069121548Z INFO: Collecting host service logs for ovsdb-server 2026-04-16T15:48:26.069457847Z INFO: Collecting host service logs for ovs-vswitchd 2026-04-16T15:48:26.069880065Z INFO: Waiting for worker host service log collection to complete ... 2026-04-16T15:48:26.436104440Z No resources found in openshift-etcd namespace. 2026-04-16T15:48:26.476515714Z INFO: "sriov-network-operator" not detected. Skipping. 2026-04-16T15:48:26.611482161Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-04-16T15:48:26.650329676Z INFO: Collecting Insights Archives from insights-operator-5785d4fcdd-fc75r 2026-04-16T15:48:26.650373301Z insights-runtime-extractor-2lhkw 2026-04-16T15:48:26.650393961Z insights-runtime-extractor-558cz 2026-04-16T15:48:26.650413714Z insights-runtime-extractor-d6m95 2026-04-16T15:48:26.676621063Z error: the server doesn't have a resource type "performanceprofile" 2026-04-16T15:48:26.691004055Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-04-16T15:48:26.691096012Z INFO: on-disk MachineConfig config collection complete. 2026-04-16T15:48:26.694588699Z INFO: Found 1 replicas - prometheus-k8s-0 2026-04-16T15:48:27.172318696Z ERROR: No running kube-apiserver pods found 2026-04-16T15:48:27.196904217Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-04-16T15:48:27.247955493Z No resources found 2026-04-16T15:48:27.311497019Z INFO: "metallb-operator" not detected. Skipping. 2026-04-16T15:48:27.329354073Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-04-16T15:48:27.329354073Z See 'oc rsync -h' for help and examples 2026-04-16T15:48:27.410905326Z INFO: Worker host service log collection to complete. 2026-04-16T15:48:27.559206321Z INFO: Waiting for HAProxy config collection to complete ... 2026-04-16T15:48:27.593617306Z Wrote inspect data to must-gather. 2026-04-16T15:48:27.681980698Z Wrote inspect data to must-gather. 2026-04-16T15:48:27.771822883Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-04-16T15:48:27.837820922Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-16T15:48:28.328102994Z tar: Removing leading `/' from member names 2026-04-16T15:48:28.338812551Z INFO: HAProxy config collection complete. 2026-04-16T15:48:28.356393160Z No resources found 2026-04-16T15:48:28.456852336Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-04-16T15:48:28.485759207Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-04-16T15:48:28.489006438Z Wrote inspect data to must-gather. 2026-04-16T15:48:28.504613030Z Wrote inspect data to must-gather. 2026-04-16T15:48:28.577768923Z Gathering data for ns/kuadrant-system... 2026-04-16T15:48:28.590299947Z Wrote inspect data to must-gather. 2026-04-16T15:48:28.621752707Z Wrote inspect data to must-gather. 2026-04-16T15:48:28.693135899Z error: the server doesn't have a resource type "machineconfigs" 2026-04-16T15:48:28.697620805Z INFO: 'previous-logs' folder not found on ip-10-0-129-254.ec2.internal, skipping... 2026-04-16T15:48:28.815773992Z INFO: Getting rules from prometheus-k8s-0 2026-04-16T15:48:28.983691093Z Wrote inspect data to must-gather. 2026-04-16T15:48:29.017029407Z Wrote inspect data to must-gather. 2026-04-16T15:48:29.030179969Z INFO: 'previous-logs' folder not found on ip-10-0-135-252.ec2.internal, skipping... 2026-04-16T15:48:29.335907560Z Wrote inspect data to must-gather. 2026-04-16T15:48:29.370506114Z INFO: Getting status/config from prometheus-k8s-0 2026-04-16T15:48:29.390549508Z Wrote inspect data to must-gather. 2026-04-16T15:48:29.423725720Z INFO: 'previous-logs' folder not found on ip-10-0-136-151.ec2.internal, skipping... 2026-04-16T15:48:29.423834530Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-04-16T15:48:29.423885627Z INFO: Machine Config Daemon termination log collection complete. 2026-04-16T15:48:29.649545677Z error: the server doesn't have a resource type "kubeletconfigs" 2026-04-16T15:48:29.753777319Z Wrote inspect data to must-gather. 2026-04-16T15:48:29.754051472Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-04-16T15:48:29.917317970Z INFO: Getting status/flags from prometheus-k8s-0 2026-04-16T15:48:29.986096733Z Wrote inspect data to must-gather. 2026-04-16T15:48:30.135476777Z INFO: INTERCONNECT MODE 2026-04-16T15:48:30.135562256Z INFO: Gathering ovn-kubernetes DBs 2026-04-16T15:48:30.199774820Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-04-16T15:48:30.284739979Z W0416 15:48:30.284693 1165 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-04-16T15:48:30.311265677Z Wrote inspect data to must-gather. 2026-04-16T15:48:30.325276581Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-16T15:48:30.398916102Z INFO: Gathering OVN_Northbound from ovnkube-node-8dtn7... 2026-04-16T15:48:30.399189367Z INFO: Gathering OVN_Northbound from ovnkube-node-9jzvn... 2026-04-16T15:48:30.399460150Z INFO: Gathering OVN_Northbound from ovnkube-node-n96lw... 2026-04-16T15:48:30.399779289Z INFO: Gathering OVN_Southbound from ovnkube-node-8dtn7... 2026-04-16T15:48:30.400047823Z INFO: Gathering OVN_Southbound from ovnkube-node-9jzvn... 2026-04-16T15:48:30.400409126Z INFO: Gathering OVN_Southbound from ovnkube-node-n96lw... 2026-04-16T15:48:30.549409334Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-04-16T15:48:30.774551973Z tar: Removing leading `/' from member names 2026-04-16T15:48:30.809053568Z Gathering data for ns/openshift-monitoring... 2026-04-16T15:48:30.834457167Z tar: Removing leading `/' from member names 2026-04-16T15:48:30.841330236Z tar: Removing leading `/' from member names 2026-04-16T15:48:30.846863253Z tar: Removing leading `/' from member names 2026-04-16T15:48:30.878709517Z tar: Removing leading `/' from member names 2026-04-16T15:48:30.921127579Z tar: Removing leading `/' from member names 2026-04-16T15:48:30.971735871Z Wrote inspect data to must-gather. 2026-04-16T15:48:31.019063356Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-04-16T15:48:31.408877845Z Wrote inspect data to must-gather. 2026-04-16T15:48:31.520560935Z INFO: Waiting for network log collection to complete ... 2026-04-16T15:48:31.520671580Z INFO: Waiting for ovnk database copies to complete ... 2026-04-16T15:48:31.521199545Z INFO: Copying ovnk databases complete. 2026-04-16T15:48:31.523383516Z 4.2M must-gather/network_logs/ovnk_database_store 2026-04-16T15:48:31.525935496Z ovnk_database_store/ 2026-04-16T15:48:31.525989578Z ovnk_database_store/ovnkube-node-n96lw_nbdb 2026-04-16T15:48:31.532290710Z INFO: Getting status from alertmanager-main-0 2026-04-16T15:48:31.541990320Z ovnk_database_store/ovnkube-node-8dtn7_nbdb 2026-04-16T15:48:31.554229180Z ovnk_database_store/ovnkube-node-8dtn7_sbdb 2026-04-16T15:48:31.605970542Z ovnk_database_store/ovnkube-node-9jzvn_sbdb 2026-04-16T15:48:31.637471358Z ovnk_database_store/ovnkube-node-n96lw_sbdb 2026-04-16T15:48:31.669942383Z ovnk_database_store/ovnkube-node-9jzvn_nbdb 2026-04-16T15:48:31.686157003Z INFO: Network log collection complete. 2026-04-16T15:48:31.762800722Z Wrote inspect data to must-gather. 2026-04-16T15:48:32.034479706Z Wrote inspect data to must-gather. 2026-04-16T15:48:32.361353225Z Wrote inspect data to must-gather. 2026-04-16T15:48:32.695514608Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-04-16T15:48:32.699065146Z INFO: Fallback to identify the container image from release info 2026-04-16T15:48:33.129298415Z Gathering data for ns/openshift-network-console... 2026-04-16T15:48:33.242571564Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc77fc5f1546189d6b57967ccc684e794bd846ea36c3e3c1e61d83a684f0f9b9 2026-04-16T15:48:33.379779145Z daemonset.apps/perf-node-gather-daemonset created 2026-04-16T15:48:33.474898507Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-04-16T15:48:33.532846065Z Waiting for performance profile collector pods to become ready: 1 2026-04-16T15:48:33.749532711Z Gathering data for ns/openshift-console-operator... 2026-04-16T15:48:34.242548186Z Gathering data for ns/openshift-console... 2026-04-16T15:48:34.684335175Z Waiting for performance profile collector pods to become ready: 2 2026-04-16T15:48:34.776168690Z Gathering data for ns/openshift-cluster-storage-operator... 2026-04-16T15:48:35.235161561Z Gathering data for ns/openshift-dns-operator... 2026-04-16T15:48:35.423033822Z Wrote inspect data to must-gather. 2026-04-16T15:48:35.564598102Z Gathering data for ns/openshift-dns... 2026-04-16T15:48:35.804892112Z Waiting for performance profile collector pods to become ready: 3 2026-04-16T15:48:36.183866921Z Gathering data for ns/openshift-image-registry... 2026-04-16T15:48:36.662476906Z Gathering data for ns/openshift-ingress-operator... 2026-04-16T15:48:36.932396071Z Waiting for performance profile collector pods to become ready: 4 2026-04-16T15:48:37.027162660Z Gathering data for ns/openshift-ingress... 2026-04-16T15:48:37.814885258Z Gathering data for ns/openshift-ingress-canary... 2026-04-16T15:48:38.057649741Z Waiting for performance profile collector pods to become ready: 5 2026-04-16T15:48:38.269686125Z Gathering data for ns/openshift-insights... 2026-04-16T15:48:38.479622037Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-04-16T15:48:39.173251908Z Waiting for performance profile collector pods to become ready: 6 2026-04-16T15:48:40.187362585Z Gathering data for ns/opendatahub... 2026-04-16T15:48:40.290999318Z Waiting for performance profile collector pods to become ready: 7 2026-04-16T15:48:41.416132102Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-04-16T15:48:41.673697548Z Collecting performance related data for node ip-10-0-129-254.ec2.internal 2026-04-16T15:48:41.680749838Z Collecting performance related data for node ip-10-0-135-252.ec2.internal 2026-04-16T15:48:41.688089253Z Collecting performance related data for node ip-10-0-136-151.ec2.internal 2026-04-16T15:48:41.851685612Z Gathering data for ns/openshift-lws-operator... 2026-04-16T15:48:43.358937045Z Gathering data for ns/openshift-config... 2026-04-16T15:48:43.483966096Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-04-16T15:48:43.679933814Z Gathering data for ns/openshift-config-managed... 2026-04-16T15:48:44.164147662Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-04-16T15:48:44.521208180Z Gathering data for ns/openshift-kube-apiserver... 2026-04-16T15:48:44.857401745Z Gathering data for ns/openshift-kube-controller-manager... 2026-04-16T15:48:45.188339710Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-04-16T15:48:45.506681080Z Gathering data for ns/openshift-kube-scheduler... 2026-04-16T15:48:45.820154314Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-04-16T15:48:46.171318288Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-04-16T15:48:46.546441144Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-04-16T15:48:46.905783936Z Gathering data for ns/openshift-user-workload-monitoring... 2026-04-16T15:48:47.526813469Z Gathering data for ns/openshift-multus... 2026-04-16T15:48:48.488429151Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-04-16T15:48:48.965957779Z Gathering data for ns/openshift-ovn-kubernetes... 2026-04-16T15:48:50.555618144Z Collecting kubelet logs for node ip-10-0-129-254.ec2.internal 2026-04-16T15:48:50.557819913Z Collecting kubelet logs for node ip-10-0-135-252.ec2.internal 2026-04-16T15:48:50.559855242Z Collecting kubelet logs for node ip-10-0-136-151.ec2.internal 2026-04-16T15:48:50.891007682Z Gathering data for ns/openshift-host-network... 2026-04-16T15:48:50.966225459Z daemonset.apps "perf-node-gather-daemonset" deleted 2026-04-16T15:48:50.972005269Z INFO: Node performance data collection complete. 2026-04-16T15:48:51.288058899Z Gathering data for ns/openshift-network-diagnostics... 2026-04-16T15:48:51.849032697Z Gathering data for ns/openshift-network-node-identity... 2026-04-16T15:48:52.313611624Z Gathering data for ns/openshift-network-operator... 2026-04-16T15:48:52.745939769Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-04-16T15:48:53.120230554Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-04-16T15:48:53.494413969Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-04-16T15:48:53.650351439Z Gathering data for ns/openshift-apiserver-operator... 2026-04-16T15:48:53.986470719Z Gathering data for ns/openshift-apiserver... 2026-04-16T15:48:54.438853484Z Gathering data for ns/openshift-controller-manager-operator... 2026-04-16T15:48:54.723843101Z Gathering data for ns/openshift-controller-manager... 2026-04-16T15:48:55.054814162Z Gathering data for ns/openshift-cluster-samples-operator... 2026-04-16T15:48:55.713981712Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-04-16T15:48:56.039213250Z Gathering data for ns/openshift-service-ca-operator... 2026-04-16T15:48:56.376107367Z Gathering data for ns/openshift-service-ca... 2026-04-16T15:48:56.766487052Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-04-16T15:48:57.763575674Z Wrote inspect data to must-gather. 2026-04-16T15:48:57.763615904Z error: inspection completed with the errors occurred while gathering data: 2026-04-16T15:48:57.763615904Z skipping gathering secrets/support due to error: secrets "support" not found 2026-04-16T15:48:57.893583365Z error: the server doesn't have a resource type "clusters" 2026-04-16T15:48:58.094448453Z Caches written to disk