2026-09-18T09:07:59.755445017Z [disk usage checker] Started 2026-09-18T09:07:59.758940227Z [disk usage checker] Volume usage percentage: current = 26 ; allowed = 70 2026-09-18T09:08:00.019657516Z Gathering data for ns/openshift-cluster-version... 2026-09-18T09:08:00.138064535Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-18T09:08:00.223992757Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-18T09:08:00.310533147Z Gathering data for ns/default... 2026-09-18T09:08:00.578939974Z Gathering data for ns/openshift... 2026-09-18T09:08:00.974444955Z Gathering data for ns/kube-system... 2026-09-18T09:08:01.662944864Z Gathering data for ns/openshift-etcd... 2026-09-18T09:08:01.952089476Z Wrote inspect data to must-gather. 2026-09-18T09:08:01.952204661Z error: inspection completed with the errors occurred while gathering data: 2026-09-18T09:08:01.952204661Z namespaces "assisted-installer" not found 2026-09-18T09:08:02.343577949Z Waiting on subprocesses to finish execution. 2026-09-18T09:08:02.349262447Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-18T09:08:02.352218710Z INFO: Collecting host service logs for crio 2026-09-18T09:08:02.352524645Z INFO: Collecting host service logs for kubelet 2026-09-18T09:08:02.352855349Z INFO: Collecting host service logs for rpm-ostreed 2026-09-18T09:08:02.353177761Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-18T09:08:02.353497592Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-18T09:08:02.354030818Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-18T09:08:02.354356507Z INFO: Collecting host service logs for NetworkManager 2026-09-18T09:08:02.354689601Z INFO: Collecting host service logs for openvswitch 2026-09-18T09:08:02.355031396Z INFO: Collecting host service logs for ovs-configuration 2026-09-18T09:08:02.355371467Z INFO: Collecting host service logs for ovsdb-server 2026-09-18T09:08:02.355697717Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-18T09:08:02.356121551Z INFO: Waiting for worker host service log collection to complete ... 2026-09-18T09:08:02.359213200Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-18T09:08:02.360245926Z Executing Istio gather script 2026-09-18T09:08:02.360898193Z INFO: Gathering HAProxy config files 2026-09-18T09:08:02.375063385Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-18T09:08:02.482491343Z INFO: Waiting for node performance related collection to complete ... 2026-09-18T09:08:03.218722759Z Wrote inspect data to must-gather. 2026-09-18T09:08:03.233579897Z No resources found in openshift-etcd namespace. 2026-09-18T09:08:03.458635152Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-18T09:08:03.614144974Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-18T09:08:03.641041367Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-18T09:08:03.641041367Z INFO: on-disk MachineConfig config collection complete. 2026-09-18T09:08:03.746280267Z No resources found 2026-09-18T09:08:03.825141159Z ERROR: No running kube-apiserver pods found 2026-09-18T09:08:03.851688067Z error: the server doesn't have a resource type "performanceprofile" 2026-09-18T09:08:03.852857690Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-18T09:08:03.864455574Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-18T09:08:03.907413536Z INFO: Worker host service log collection to complete. 2026-09-18T09:08:03.916021089Z INFO: "metallb-operator" not detected. Skipping. 2026-09-18T09:08:04.052669071Z INFO: Collecting Insights Archives from insights-operator-6f9c644dd9-br8hf 2026-09-18T09:08:04.052720191Z insights-runtime-extractor-gfpx6 2026-09-18T09:08:04.052740473Z insights-runtime-extractor-sxzm2 2026-09-18T09:08:04.052760229Z insights-runtime-extractor-wgvmj 2026-09-18T09:08:04.485488041Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-18T09:08:04.613344816Z Gathering data for ns/openshift-monitoring... 2026-09-18T09:08:04.670221633Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-18T09:08:04.703953923Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-18T09:08:04.703953923Z See 'oc rsync -h' for help and examples 2026-09-18T09:08:04.742757323Z Wrote inspect data to must-gather. 2026-09-18T09:08:04.764584857Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-18T09:08:04.776626765Z Wrote inspect data to must-gather. 2026-09-18T09:08:04.787297295Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-18T09:08:04.860644171Z [disk usage checker] Volume usage percentage: current = 26 ; allowed = 70 2026-09-18T09:08:04.889052755Z No resources found 2026-09-18T09:08:04.941056860Z INFO: 'previous-logs' folder not found on ip-10-0-135-178.ec2.internal, skipping... 2026-09-18T09:08:04.977192763Z Wrote inspect data to must-gather. 2026-09-18T09:08:05.011502328Z Wrote inspect data to must-gather. 2026-09-18T09:08:05.050615297Z Wrote inspect data to must-gather. 2026-09-18T09:08:05.061154869Z Wrote inspect data to must-gather. 2026-09-18T09:08:05.210113608Z tar: Removing leading `/' from member names 2026-09-18T09:08:05.215566629Z INFO: HAProxy config collection complete. 2026-09-18T09:08:05.270464514Z Wrote inspect data to must-gather. 2026-09-18T09:08:05.272486629Z error: the server doesn't have a resource type "machineconfigs" 2026-09-18T09:08:05.303243547Z INFO: 'previous-logs' folder not found on ip-10-0-136-249.ec2.internal, skipping... 2026-09-18T09:08:05.424210071Z INFO: Getting rules from prometheus-k8s-0 2026-09-18T09:08:05.448692276Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-18T09:08:05.520763230Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-18T09:08:05.555485668Z Wrote inspect data to must-gather. 2026-09-18T09:08:05.642271238Z INFO: 'previous-logs' folder not found on ip-10-0-137-204.ec2.internal, skipping... 2026-09-18T09:08:05.642357279Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-18T09:08:05.642390584Z INFO: Machine Config Daemon termination log collection complete. 2026-09-18T09:08:05.693164196Z Wrote inspect data to must-gather. 2026-09-18T09:08:05.808846758Z Wrote inspect data to must-gather. 2026-09-18T09:08:05.852399071Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-18T09:08:05.873926504Z INFO: Getting status/config from prometheus-k8s-0 2026-09-18T09:08:06.053480246Z Wrote inspect data to must-gather. 2026-09-18T09:08:06.089770930Z INFO: INTERCONNECT MODE 2026-09-18T09:08:06.089813854Z INFO: Gathering ovn-kubernetes DBs 2026-09-18T09:08:06.261114244Z INFO: Gathering OVN_Northbound from ovnkube-node-6pbrk... 2026-09-18T09:08:06.261372790Z INFO: Gathering OVN_Northbound from ovnkube-node-rn8kf... 2026-09-18T09:08:06.261661114Z INFO: Gathering OVN_Northbound from ovnkube-node-rp4vd... 2026-09-18T09:08:06.262288531Z INFO: Gathering OVN_Southbound from ovnkube-node-6pbrk... 2026-09-18T09:08:06.262596772Z INFO: Gathering OVN_Southbound from ovnkube-node-rn8kf... 2026-09-18T09:08:06.262889578Z INFO: Gathering OVN_Southbound from ovnkube-node-rp4vd... 2026-09-18T09:08:06.272734364Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-18T09:08:06.303316327Z Wrote inspect data to must-gather. 2026-09-18T09:08:06.553608923Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-18T09:08:06.601658071Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-18T09:08:06.670816390Z tar: Removing leading `/' from member names 2026-09-18T09:08:06.685371382Z tar: Removing leading `/' from member names 2026-09-18T09:08:06.701659767Z tar: Removing leading `/' from member names 2026-09-18T09:08:06.702423332Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-18T09:08:06.705335742Z tar: Removing leading `/' from member names 2026-09-18T09:08:06.705494828Z tar: Removing leading `/' from member names 2026-09-18T09:08:06.706555121Z tar: Removing leading `/' from member names 2026-09-18T09:08:06.759852199Z Wrote inspect data to must-gather. 2026-09-18T09:08:06.826382445Z Wrote inspect data to /must-gather/istio. 2026-09-18T09:08:06.978132062Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-18T09:08:07.018180059Z Wrote inspect data to must-gather. 2026-09-18T09:08:07.078952354Z Wrote inspect data to /must-gather/istio. 2026-09-18T09:08:07.086683508Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-18T09:08:07.141907085Z INFO: Waiting for network log collection to complete ... 2026-09-18T09:08:07.141957414Z INFO: Waiting for ovnk database copies to complete ... 2026-09-18T09:08:07.142459125Z INFO: Copying ovnk databases complete. 2026-09-18T09:08:07.144196008Z 3.3M must-gather/network_logs/ovnk_database_store 2026-09-18T09:08:07.146438907Z ovnk_database_store/ 2026-09-18T09:08:07.146438907Z ovnk_database_store/ovnkube-node-rn8kf_sbdb 2026-09-18T09:08:07.171008439Z ovnk_database_store/ovnkube-node-6pbrk_nbdb 2026-09-18T09:08:07.179380527Z ovnk_database_store/ovnkube-node-6pbrk_sbdb 2026-09-18T09:08:07.199506111Z ovnk_database_store/ovnkube-node-rp4vd_sbdb 2026-09-18T09:08:07.217469925Z ovnk_database_store/ovnkube-node-rn8kf_nbdb 2026-09-18T09:08:07.227650305Z ovnk_database_store/ovnkube-node-rp4vd_nbdb 2026-09-18T09:08:07.236869576Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-18T09:08:07.238208637Z INFO: Network log collection complete. 2026-09-18T09:08:07.240218176Z W0918 09:08:07.240156 1494 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-09-18T09:08:07.249211845Z Wrote inspect data to must-gather. 2026-09-18T09:08:07.298087520Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-18T09:08:07.305049440Z Wrote inspect data to /must-gather/istio. 2026-09-18T09:08:07.487893664Z Gathering data for ns/openshift-network-console... 2026-09-18T09:08:07.536378402Z Wrote inspect data to /must-gather/istio. 2026-09-18T09:08:07.542255776Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-18T09:08:07.555405608Z Wrote inspect data to must-gather. 2026-09-18T09:08:07.638476320Z INFO: Getting status from alertmanager-main-0 2026-09-18T09:08:07.754908550Z Wrote inspect data to /must-gather/istio. 2026-09-18T09:08:07.872683778Z Wrote inspect data to must-gather. 2026-09-18T09:08:07.952339315Z Gathering data for ns/openshift-console-operator... 2026-09-18T09:08:07.964306180Z Wrote inspect data to /must-gather/istio. 2026-09-18T09:08:07.967212922Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-18T09:08:08.130500884Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-18T09:08:08.133354488Z INFO: Fallback to identify the container image from release info 2026-09-18T09:08:08.190034204Z Wrote inspect data to /must-gather/istio. 2026-09-18T09:08:08.345550367Z Gathering data for ns/openshift-console... 2026-09-18T09:08:08.383458800Z Wrote inspect data to /must-gather/istio. 2026-09-18T09:08:08.386432979Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-18T09:08:08.547693560Z Wrote inspect data to /must-gather/istio. 2026-09-18T09:08:08.745829547Z Wrote inspect data to /must-gather/istio. 2026-09-18T09:08:08.752676489Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0720c8b078cc0cebb01b4c5cde21180aee7c2bb1a0d322b21890c0d3707406ea 2026-09-18T09:08:08.864870745Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-18T09:08:08.887300686Z daemonset.apps/perf-node-gather-daemonset created 2026-09-18T09:08:09.032166628Z Waiting for performance profile collector pods to become ready: 1 2026-09-18T09:08:09.388677972Z Gathering data for ns/openshift-dns-operator... 2026-09-18T09:08:09.463337358Z Done executing Istio gather script 2026-09-18T09:08:09.681461505Z Gathering data for ns/openshift-dns... 2026-09-18T09:08:09.884818657Z [disk usage checker] Volume usage percentage: current = 26 ; allowed = 70 2026-09-18T09:08:10.153727307Z Waiting for performance profile collector pods to become ready: 2 2026-09-18T09:08:10.217849181Z Gathering data for ns/openshift-image-registry... 2026-09-18T09:08:10.639969026Z Gathering data for ns/openshift-ingress-operator... 2026-09-18T09:08:10.969578472Z Gathering data for ns/openshift-ingress... 2026-09-18T09:08:11.275087353Z Waiting for performance profile collector pods to become ready: 3 2026-09-18T09:08:11.322188202Z Gathering data for ns/openshift-ingress-canary... 2026-09-18T09:08:11.680952299Z Gathering data for ns/openshift-insights... 2026-09-18T09:08:12.384711621Z Waiting for performance profile collector pods to become ready: 4 2026-09-18T09:08:13.448312258Z Gathering data for ns/openshift-jobset-operator... 2026-09-18T09:08:13.486215603Z Waiting for performance profile collector pods to become ready: 5 2026-09-18T09:08:14.212679642Z Gathering data for ns/openshift-config... 2026-09-18T09:08:14.502758597Z Gathering data for ns/openshift-config-managed... 2026-09-18T09:08:14.609493089Z Waiting for performance profile collector pods to become ready: 6 2026-09-18T09:08:14.890725715Z [disk usage checker] Volume usage percentage: current = 26 ; allowed = 70 2026-09-18T09:08:14.906733121Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-18T09:08:15.147019084Z Gathering data for ns/openshift-kube-apiserver... 2026-09-18T09:08:15.381629084Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-18T09:08:15.711696570Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-18T09:08:15.729818061Z Waiting for performance profile collector pods to become ready: 7 2026-09-18T09:08:16.008713455Z Gathering data for ns/openshift-kube-scheduler... 2026-09-18T09:08:16.244276195Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-18T09:08:16.499445775Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-18T09:08:16.789247236Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-18T09:08:16.851381858Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-18T09:08:17.099786489Z Collecting performance related data for node ip-10-0-135-178.ec2.internal 2026-09-18T09:08:17.107571562Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-18T09:08:17.108015381Z Collecting performance related data for node ip-10-0-137-204.ec2.internal 2026-09-18T09:08:17.115636814Z Collecting performance related data for node ip-10-0-136-249.ec2.internal 2026-09-18T09:08:17.690604311Z Gathering data for ns/openshift-multus... 2026-09-18T09:08:19.062963771Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-18T09:08:19.896398976Z [disk usage checker] Volume usage percentage: current = 27 ; allowed = 70 2026-09-18T09:08:20.367548692Z Gathering data for ns/openshift-host-network... 2026-09-18T09:08:20.677361599Z Gathering data for ns/openshift-network-diagnostics... 2026-09-18T09:08:21.221763310Z Gathering data for ns/openshift-network-node-identity... 2026-09-18T09:08:21.620163311Z Gathering data for ns/openshift-network-operator... 2026-09-18T09:08:22.009082585Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-18T09:08:22.310164864Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-18T09:08:22.820758033Z Gathering data for ns/openshift-apiserver-operator... 2026-09-18T09:08:23.137555274Z Gathering data for ns/openshift-apiserver... 2026-09-18T09:08:23.489237454Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-18T09:08:23.742359512Z Gathering data for ns/openshift-controller-manager... 2026-09-18T09:08:24.006182345Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-18T09:08:24.660713409Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-18T09:08:24.823358186Z Collecting kubelet logs for node ip-10-0-135-178.ec2.internal 2026-09-18T09:08:24.825341601Z Collecting kubelet logs for node ip-10-0-137-204.ec2.internal 2026-09-18T09:08:24.827380506Z Collecting kubelet logs for node ip-10-0-136-249.ec2.internal 2026-09-18T09:08:24.904990006Z [disk usage checker] Volume usage percentage: current = 27 ; allowed = 70 2026-09-18T09:08:24.956425030Z Gathering data for ns/openshift-service-ca-operator... 2026-09-18T09:08:25.175276570Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-6b5d2 namespace 2026-09-18T09:08:25.180867850Z INFO: Node performance data collection complete. 2026-09-18T09:08:25.278517744Z Gathering data for ns/openshift-service-ca... 2026-09-18T09:08:25.627679704Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-18T09:08:26.467487443Z Wrote inspect data to must-gather. 2026-09-18T09:08:26.467639028Z error: inspection completed with the errors occurred while gathering data: 2026-09-18T09:08:26.467639028Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-18T09:08:26.591019458Z error: the server doesn't have a resource type "clusters" 2026-09-18T09:08:26.720522569Z Caches written to disk