2026-09-17T09:14:06.254934778Z [disk usage checker] Started 2026-09-17T09:14:06.258366253Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-17T09:14:06.436983485Z Gathering data for ns/openshift-cluster-version... 2026-09-17T09:14:06.562584902Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-17T09:14:06.648679183Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-17T09:14:06.727761097Z Gathering data for ns/default... 2026-09-17T09:14:06.992746777Z Gathering data for ns/openshift... 2026-09-17T09:14:07.402896476Z Gathering data for ns/kube-system... 2026-09-17T09:14:08.094217600Z Gathering data for ns/openshift-etcd... 2026-09-17T09:14:08.384250177Z Wrote inspect data to must-gather. 2026-09-17T09:14:08.384342101Z error: inspection completed with the errors occurred while gathering data: 2026-09-17T09:14:08.384342101Z namespaces "assisted-installer" not found 2026-09-17T09:14:08.753000322Z Waiting on subprocesses to finish execution. 2026-09-17T09:14:08.759039970Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-17T09:14:08.762724019Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-17T09:14:08.763019319Z INFO: Gathering HAProxy config files 2026-09-17T09:14:08.770425560Z Executing Istio gather script 2026-09-17T09:14:08.775947267Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-17T09:14:08.817199374Z INFO: Waiting for node performance related collection to complete ... 2026-09-17T09:14:08.822238238Z INFO: Collecting host service logs for crio 2026-09-17T09:14:08.822555910Z INFO: Collecting host service logs for kubelet 2026-09-17T09:14:08.822862770Z INFO: Collecting host service logs for rpm-ostreed 2026-09-17T09:14:08.823170428Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-17T09:14:08.823494441Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-17T09:14:08.823810882Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-17T09:14:08.824085401Z INFO: Collecting host service logs for NetworkManager 2026-09-17T09:14:08.824358605Z INFO: Collecting host service logs for openvswitch 2026-09-17T09:14:08.824659085Z INFO: Collecting host service logs for ovs-configuration 2026-09-17T09:14:08.824939875Z INFO: Collecting host service logs for ovsdb-server 2026-09-17T09:14:08.825226419Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-17T09:14:08.825683848Z INFO: Waiting for worker host service log collection to complete ... 2026-09-17T09:14:09.264264445Z No resources found 2026-09-17T09:14:09.366052809Z INFO: Collecting Insights Archives from insights-operator-5844b856db-qwkvf 2026-09-17T09:14:09.366052809Z insights-runtime-extractor-5j9xs 2026-09-17T09:14:09.366052809Z insights-runtime-extractor-9ctql 2026-09-17T09:14:09.366052809Z insights-runtime-extractor-n57rw 2026-09-17T09:14:09.487351974Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-17T09:14:09.488617691Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-17T09:14:09.528068161Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-17T09:14:09.742693864Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-17T09:14:09.742811967Z INFO: on-disk MachineConfig config collection complete. 2026-09-17T09:14:09.763434039Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-17T09:14:09.863404094Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-17T09:14:09.888479468Z error: the server doesn't have a resource type "performanceprofile" 2026-09-17T09:14:10.046986705Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-17T09:14:10.046986705Z See 'oc rsync -h' for help and examples 2026-09-17T09:14:10.093923521Z Wrote inspect data to must-gather. 2026-09-17T09:14:10.157677643Z INFO: Worker host service log collection to complete. 2026-09-17T09:14:10.173710406Z No resources found in openshift-etcd namespace. 2026-09-17T09:14:10.399276412Z INFO: "metallb-operator" not detected. Skipping. 2026-09-17T09:14:10.700511077Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-17T09:14:10.737447435Z Wrote inspect data to must-gather. 2026-09-17T09:14:10.885066340Z No resources found 2026-09-17T09:14:10.904063075Z tar: Removing leading `/' from member names 2026-09-17T09:14:10.908771274Z INFO: HAProxy config collection complete. 2026-09-17T09:14:10.958134710Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-17T09:14:11.031500199Z Wrote inspect data to must-gather. 2026-09-17T09:14:11.109593531Z Wrote inspect data to must-gather. 2026-09-17T09:14:11.160024324Z ERROR: No running kube-apiserver pods found 2026-09-17T09:14:11.169995407Z Wrote inspect data to must-gather. 2026-09-17T09:14:11.178445573Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-17T09:14:11.197968441Z Gathering data for ns/openshift-monitoring... 2026-09-17T09:14:11.271139577Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-17T09:14:11.274974635Z Wrote inspect data to must-gather. 2026-09-17T09:14:11.284461679Z INFO: 'previous-logs' folder not found on ip-10-0-132-66.ec2.internal, skipping... 2026-09-17T09:14:11.285427991Z Wrote inspect data to must-gather. 2026-09-17T09:14:11.379005316Z error: the server doesn't have a resource type "machineconfigs" 2026-09-17T09:14:11.533058162Z INFO: Getting rules from prometheus-k8s-0 2026-09-17T09:14:11.551657921Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-17T09:14:11.594569991Z Wrote inspect data to must-gather. 2026-09-17T09:14:11.597468160Z INFO: 'previous-logs' folder not found on ip-10-0-136-112.ec2.internal, skipping... 2026-09-17T09:14:11.735040932Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-17T09:14:11.788190470Z Wrote inspect data to must-gather. 2026-09-17T09:14:11.886023963Z INFO: 'previous-logs' folder not found on ip-10-0-142-156.ec2.internal, skipping... 2026-09-17T09:14:11.886132686Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-17T09:14:11.886201121Z INFO: Machine Config Daemon termination log collection complete. 2026-09-17T09:14:11.888726936Z Wrote inspect data to must-gather. 2026-09-17T09:14:11.939029784Z INFO: Getting status/config from prometheus-k8s-0 2026-09-17T09:14:11.949948680Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-17T09:14:12.125129368Z Wrote inspect data to must-gather. 2026-09-17T09:14:12.144879076Z Wrote inspect data to must-gather. 2026-09-17T09:14:12.343366634Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-17T09:14:12.344567668Z Wrote inspect data to must-gather. 2026-09-17T09:14:12.400468497Z INFO: INTERCONNECT MODE 2026-09-17T09:14:12.400511682Z INFO: Gathering ovn-kubernetes DBs 2026-09-17T09:14:12.554825417Z INFO: Gathering OVN_Northbound from ovnkube-node-kw5s6... 2026-09-17T09:14:12.555142028Z INFO: Gathering OVN_Northbound from ovnkube-node-ldxdr... 2026-09-17T09:14:12.555480503Z INFO: Gathering OVN_Northbound from ovnkube-node-mjzvq... 2026-09-17T09:14:12.555862384Z INFO: Gathering OVN_Southbound from ovnkube-node-kw5s6... 2026-09-17T09:14:12.556140779Z INFO: Gathering OVN_Southbound from ovnkube-node-ldxdr... 2026-09-17T09:14:12.556381630Z INFO: Gathering OVN_Southbound from ovnkube-node-mjzvq... 2026-09-17T09:14:12.578198573Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-17T09:14:12.583041999Z Wrote inspect data to must-gather. 2026-09-17T09:14:12.734079459Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-17T09:14:12.754792151Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-17T09:14:12.875850849Z tar: Removing leading `/' from member names 2026-09-17T09:14:12.908758485Z tar: Removing leading `/' from member names 2026-09-17T09:14:12.931196821Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-17T09:14:12.933675697Z tar: Removing leading `/' from member names 2026-09-17T09:14:12.941814761Z tar: Removing leading `/' from member names 2026-09-17T09:14:12.945999595Z tar: Removing leading `/' from member names 2026-09-17T09:14:12.974759034Z tar: Removing leading `/' from member names 2026-09-17T09:14:12.978888330Z Wrote inspect data to must-gather. 2026-09-17T09:14:13.037393542Z Wrote inspect data to /must-gather/istio. 2026-09-17T09:14:13.191097221Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-17T09:14:13.229592611Z Wrote inspect data to must-gather. 2026-09-17T09:14:13.293139810Z Wrote inspect data to /must-gather/istio. 2026-09-17T09:14:13.297285933Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-17T09:14:13.437596991Z INFO: Waiting for network log collection to complete ... 2026-09-17T09:14:13.437644756Z INFO: Waiting for ovnk database copies to complete ... 2026-09-17T09:14:13.438026612Z INFO: Copying ovnk databases complete. 2026-09-17T09:14:13.439758511Z 2.9M must-gather/network_logs/ovnk_database_store 2026-09-17T09:14:13.442359095Z ovnk_database_store/ 2026-09-17T09:14:13.442359095Z ovnk_database_store/ovnkube-node-mjzvq_nbdb 2026-09-17T09:14:13.452402367Z ovnk_database_store/ovnkube-node-ldxdr_nbdb 2026-09-17T09:14:13.459997339Z ovnk_database_store/ovnkube-node-kw5s6_sbdb 2026-09-17T09:14:13.470795813Z W0917 09:14:13.470757 1463 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-09-17T09:14:13.477483695Z ovnk_database_store/ovnkube-node-mjzvq_sbdb 2026-09-17T09:14:13.478313256Z Wrote inspect data to must-gather. 2026-09-17T09:14:13.495700021Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-17T09:14:13.499352197Z ovnk_database_store/ovnkube-node-ldxdr_sbdb 2026-09-17T09:14:13.520420391Z ovnk_database_store/ovnkube-node-kw5s6_nbdb 2026-09-17T09:14:13.531062069Z INFO: Network log collection complete. 2026-09-17T09:14:13.531914524Z Wrote inspect data to /must-gather/istio. 2026-09-17T09:14:13.611818729Z INFO: Getting status from alertmanager-main-0 2026-09-17T09:14:13.693352828Z Gathering data for ns/openshift-network-console... 2026-09-17T09:14:13.757057108Z Wrote inspect data to /must-gather/istio. 2026-09-17T09:14:13.760118684Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-17T09:14:13.803674175Z Wrote inspect data to must-gather. 2026-09-17T09:14:13.949364417Z Wrote inspect data to /must-gather/istio. 2026-09-17T09:14:14.054998514Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-17T09:14:14.057946712Z INFO: Fallback to identify the container image from release info 2026-09-17T09:14:14.129868559Z Gathering data for ns/openshift-console-operator... 2026-09-17T09:14:14.153253216Z Wrote inspect data to /must-gather/istio. 2026-09-17T09:14:14.155700179Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-17T09:14:14.364901907Z Wrote inspect data to /must-gather/istio. 2026-09-17T09:14:14.521885024Z Gathering data for ns/openshift-console... 2026-09-17T09:14:14.545601386Z Wrote inspect data to /must-gather/istio. 2026-09-17T09:14:14.547912186Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-17T09:14:14.703378125Z Wrote inspect data to /must-gather/istio. 2026-09-17T09:14:14.725692028Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a 2026-09-17T09:14:14.852125372Z daemonset.apps/perf-node-gather-daemonset created 2026-09-17T09:14:14.885557128Z Wrote inspect data to /must-gather/istio. 2026-09-17T09:14:14.987866141Z Waiting for performance profile collector pods to become ready: 1 2026-09-17T09:14:15.070862496Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-17T09:14:15.592161199Z Gathering data for ns/openshift-dns-operator... 2026-09-17T09:14:15.595163019Z Done executing Istio gather script 2026-09-17T09:14:15.895827164Z Gathering data for ns/openshift-dns... 2026-09-17T09:14:16.091562934Z Waiting for performance profile collector pods to become ready: 2 2026-09-17T09:14:16.276898499Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-17T09:14:16.437061293Z Gathering data for ns/openshift-image-registry... 2026-09-17T09:14:16.877778553Z Gathering data for ns/openshift-ingress-operator... 2026-09-17T09:14:17.194801959Z Gathering data for ns/openshift-ingress... 2026-09-17T09:14:17.205908996Z Waiting for performance profile collector pods to become ready: 3 2026-09-17T09:14:17.497195357Z Gathering data for ns/openshift-ingress-canary... 2026-09-17T09:14:17.853706447Z Gathering data for ns/openshift-insights... 2026-09-17T09:14:18.331031712Z Waiting for performance profile collector pods to become ready: 4 2026-09-17T09:14:19.432955353Z Waiting for performance profile collector pods to become ready: 5 2026-09-17T09:14:19.750477935Z Gathering data for ns/openshift-jobset-operator... 2026-09-17T09:14:20.532730662Z Waiting for performance profile collector pods to become ready: 6 2026-09-17T09:14:20.550375951Z Gathering data for ns/openshift-config... 2026-09-17T09:14:20.808032617Z Gathering data for ns/openshift-config-managed... 2026-09-17T09:14:21.212291488Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-17T09:14:21.281364795Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-17T09:14:21.458653268Z Gathering data for ns/openshift-kube-apiserver... 2026-09-17T09:14:21.642639151Z Waiting for performance profile collector pods to become ready: 7 2026-09-17T09:14:21.717751667Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-17T09:14:21.983695195Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-17T09:14:22.248647257Z Gathering data for ns/openshift-kube-scheduler... 2026-09-17T09:14:22.533456235Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-17T09:14:22.744787925Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-17T09:14:22.874646954Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-17T09:14:22.999242225Z Collecting performance related data for node ip-10-0-142-156.ec2.internal 2026-09-17T09:14:23.006369631Z Collecting performance related data for node ip-10-0-136-112.ec2.internal 2026-09-17T09:14:23.013543582Z Collecting performance related data for node ip-10-0-132-66.ec2.internal 2026-09-17T09:14:23.225761681Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-17T09:14:23.570466832Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-17T09:14:24.184765972Z Gathering data for ns/openshift-multus... 2026-09-17T09:14:25.600706636Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-17T09:14:26.285900195Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-17T09:14:26.787274097Z Gathering data for ns/openshift-host-network... 2026-09-17T09:14:27.091181513Z Gathering data for ns/openshift-network-diagnostics... 2026-09-17T09:14:27.625557531Z Gathering data for ns/openshift-network-node-identity... 2026-09-17T09:14:28.058008122Z Gathering data for ns/openshift-network-operator... 2026-09-17T09:14:28.414624992Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-17T09:14:28.678967941Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-17T09:14:29.239488584Z Gathering data for ns/openshift-apiserver-operator... 2026-09-17T09:14:29.581357950Z Gathering data for ns/openshift-apiserver... 2026-09-17T09:14:29.975046993Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-17T09:14:30.244719808Z Gathering data for ns/openshift-controller-manager... 2026-09-17T09:14:30.531981706Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-17T09:14:30.748549536Z Collecting kubelet logs for node ip-10-0-142-156.ec2.internal 2026-09-17T09:14:30.750665739Z Collecting kubelet logs for node ip-10-0-136-112.ec2.internal 2026-09-17T09:14:30.752729683Z Collecting kubelet logs for node ip-10-0-132-66.ec2.internal 2026-09-17T09:14:31.070727530Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-gcfwd namespace 2026-09-17T09:14:31.076966562Z INFO: Node performance data collection complete. 2026-09-17T09:14:31.227980163Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-17T09:14:31.296303140Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-17T09:14:31.501951243Z Gathering data for ns/openshift-service-ca-operator... 2026-09-17T09:14:31.811454005Z Gathering data for ns/openshift-service-ca... 2026-09-17T09:14:32.139803920Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-17T09:14:33.005842165Z Wrote inspect data to must-gather. 2026-09-17T09:14:33.005884304Z error: inspection completed with the errors occurred while gathering data: 2026-09-17T09:14:33.005884304Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-17T09:14:33.119285757Z error: the server doesn't have a resource type "clusters" 2026-09-17T09:14:33.312559938Z Caches written to disk