2026-09-09T21:09:41.428931571Z [disk usage checker] Started 2026-09-09T21:09:41.432272184Z [disk usage checker] Volume usage percentage: current = 38 ; allowed = 70 2026-09-09T21:09:41.782278980Z Gathering data for ns/openshift-cluster-version... 2026-09-09T21:09:41.978071585Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-09T21:09:42.077539229Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-09T21:09:42.172427810Z Gathering data for ns/default... 2026-09-09T21:09:42.558545872Z Gathering data for ns/openshift... 2026-09-09T21:09:43.001628917Z Gathering data for ns/kube-system... 2026-09-09T21:09:43.841118283Z Gathering data for ns/openshift-etcd... 2026-09-09T21:09:44.211640867Z Wrote inspect data to must-gather. 2026-09-09T21:09:44.211690594Z error: inspection completed with the errors occurred while gathering data: 2026-09-09T21:09:44.211690594Z namespaces "assisted-installer" not found 2026-09-09T21:09:44.481314419Z Waiting on subprocesses to finish execution. 2026-09-09T21:09:44.489844846Z INFO: Gathering HAProxy config files 2026-09-09T21:09:44.491965128Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-09T21:09:44.496167149Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-09T21:09:44.498242518Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-09T21:09:44.502922803Z Executing Istio gather script 2026-09-09T21:09:44.503379108Z INFO: Collecting host service logs for crio 2026-09-09T21:09:44.503845765Z INFO: Collecting host service logs for kubelet 2026-09-09T21:09:44.504203058Z INFO: Collecting host service logs for rpm-ostreed 2026-09-09T21:09:44.504594770Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-09T21:09:44.504977324Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-09T21:09:44.505341933Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-09T21:09:44.505733678Z INFO: Collecting host service logs for NetworkManager 2026-09-09T21:09:44.506054710Z INFO: Collecting host service logs for openvswitch 2026-09-09T21:09:44.506386619Z INFO: Collecting host service logs for ovs-configuration 2026-09-09T21:09:44.506747131Z INFO: Collecting host service logs for ovsdb-server 2026-09-09T21:09:44.507261444Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-09T21:09:44.507711654Z INFO: Waiting for worker host service log collection to complete ... 2026-09-09T21:09:44.586667476Z INFO: Waiting for node performance related collection to complete ... 2026-09-09T21:09:45.144537440Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-09T21:09:45.352701015Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-09T21:09:45.414054348Z error: the server doesn't have a resource type "performanceprofile" 2026-09-09T21:09:45.630239896Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-09T21:09:45.709454090Z INFO: Collecting Insights Archives from insights-operator-76bcd6ccf5-hzlkc 2026-09-09T21:09:45.709709844Z insights-runtime-extractor-rp476 2026-09-09T21:09:45.709738352Z insights-runtime-extractor-s4s4k 2026-09-09T21:09:45.709758696Z insights-runtime-extractor-vjq59 2026-09-09T21:09:45.728200542Z INFO: "metallb-operator" not detected. Skipping. 2026-09-09T21:09:45.744365222Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-09T21:09:45.744451694Z INFO: on-disk MachineConfig config collection complete. 2026-09-09T21:09:45.898335021Z No resources found in openshift-etcd namespace. 2026-09-09T21:09:45.904959584Z Wrote inspect data to must-gather. 2026-09-09T21:09:45.952827542Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-09T21:09:46.134342284Z No resources found 2026-09-09T21:09:46.138427025Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-09T21:09:46.139846859Z INFO: Worker host service log collection to complete. 2026-09-09T21:09:46.265636094Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-09T21:09:46.270152724Z Wrote inspect data to must-gather. 2026-09-09T21:09:46.381914507Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-09T21:09:46.381914507Z See 'oc rsync -h' for help and examples 2026-09-09T21:09:46.397247242Z Wrote inspect data to must-gather. 2026-09-09T21:09:46.450146647Z [disk usage checker] Volume usage percentage: current = 38 ; allowed = 70 2026-09-09T21:09:46.841716766Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-09T21:09:46.934059338Z ERROR: No running kube-apiserver pods found 2026-09-09T21:09:47.058158113Z Wrote inspect data to must-gather. 2026-09-09T21:09:47.074582582Z Wrote inspect data to must-gather. 2026-09-09T21:09:47.102429053Z INFO: 'previous-logs' folder not found on ip-10-0-129-130.ec2.internal, skipping... 2026-09-09T21:09:47.181336728Z Wrote inspect data to must-gather. 2026-09-09T21:09:47.211216050Z tar: Removing leading `/' from member names 2026-09-09T21:09:47.229243458Z INFO: HAProxy config collection complete. 2026-09-09T21:09:47.255943145Z Wrote inspect data to must-gather. 2026-09-09T21:09:47.256611975Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-09T21:09:47.293327042Z No resources found 2026-09-09T21:09:47.451692897Z INFO: Getting rules from prometheus-k8s-0 2026-09-09T21:09:47.504986563Z error: the server doesn't have a resource type "machineconfigs" 2026-09-09T21:09:47.632154508Z INFO: 'previous-logs' folder not found on ip-10-0-129-45.ec2.internal, skipping... 2026-09-09T21:09:47.635732176Z Gathering data for ns/openshift-monitoring... 2026-09-09T21:09:47.640836666Z Wrote inspect data to must-gather. 2026-09-09T21:09:47.741489031Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-09T21:09:47.969689091Z Wrote inspect data to must-gather. 2026-09-09T21:09:47.976887771Z INFO: 'previous-logs' folder not found on ip-10-0-137-8.ec2.internal, skipping... 2026-09-09T21:09:47.976929776Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-09T21:09:47.976960467Z INFO: Machine Config Daemon termination log collection complete. 2026-09-09T21:09:48.000775541Z INFO: Getting status/config from prometheus-k8s-0 2026-09-09T21:09:48.040453417Z Wrote inspect data to must-gather. 2026-09-09T21:09:48.220733691Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-09T21:09:48.256588607Z Wrote inspect data to must-gather. 2026-09-09T21:09:48.466576107Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-09T21:09:48.504246114Z Wrote inspect data to must-gather. 2026-09-09T21:09:48.533360259Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-09T21:09:48.610756849Z INFO: INTERCONNECT MODE 2026-09-09T21:09:48.610801033Z INFO: Gathering ovn-kubernetes DBs 2026-09-09T21:09:48.727494993Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-09T21:09:48.746090460Z Wrote inspect data to must-gather. 2026-09-09T21:09:48.789147338Z INFO: Gathering OVN_Northbound from ovnkube-node-9v47s... 2026-09-09T21:09:48.789413047Z INFO: Gathering OVN_Northbound from ovnkube-node-m9fx7... 2026-09-09T21:09:48.789636305Z INFO: Gathering OVN_Northbound from ovnkube-node-z9m2s... 2026-09-09T21:09:48.790364455Z INFO: Gathering OVN_Southbound from ovnkube-node-9v47s... 2026-09-09T21:09:48.790619527Z INFO: Gathering OVN_Southbound from ovnkube-node-m9fx7... 2026-09-09T21:09:48.790928415Z INFO: Gathering OVN_Southbound from ovnkube-node-z9m2s... 2026-09-09T21:09:49.028109505Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-09T21:09:49.103360475Z tar: Removing leading `/' from member names 2026-09-09T21:09:49.127442538Z Wrote inspect data to must-gather. 2026-09-09T21:09:49.166455046Z tar: Removing leading `/' from member names 2026-09-09T21:09:49.169858730Z tar: Removing leading `/' from member names 2026-09-09T21:09:49.177307056Z tar: Removing leading `/' from member names 2026-09-09T21:09:49.249346488Z tar: Removing leading `/' from member names 2026-09-09T21:09:49.249531787Z tar: Removing leading `/' from member names 2026-09-09T21:09:49.375897058Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-09T21:09:49.409093563Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-09T21:09:49.431830480Z Wrote inspect data to must-gather. 2026-09-09T21:09:49.582061348Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-09T21:09:49.687523361Z Wrote inspect data to must-gather. 2026-09-09T21:09:49.786797781Z INFO: Waiting for network log collection to complete ... 2026-09-09T21:09:49.786797781Z INFO: Waiting for ovnk database copies to complete ... 2026-09-09T21:09:49.787180168Z INFO: Copying ovnk databases complete. 2026-09-09T21:09:49.789339632Z 4.7M must-gather/network_logs/ovnk_database_store 2026-09-09T21:09:49.792019458Z ovnk_database_store/ 2026-09-09T21:09:49.792075312Z ovnk_database_store/ovnkube-node-z9m2s_nbdb 2026-09-09T21:09:49.804239630Z ovnk_database_store/ovnkube-node-m9fx7_sbdb 2026-09-09T21:09:49.815804306Z Wrote inspect data to /must-gather/istio. 2026-09-09T21:09:49.834635084Z ovnk_database_store/ovnkube-node-z9m2s_sbdb 2026-09-09T21:09:49.868598862Z ovnk_database_store/ovnkube-node-9v47s_nbdb 2026-09-09T21:09:49.880870740Z ovnk_database_store/ovnkube-node-m9fx7_nbdb 2026-09-09T21:09:49.892877733Z ovnk_database_store/ovnkube-node-9v47s_sbdb 2026-09-09T21:09:49.898969919Z INFO: Getting status from alertmanager-main-0 2026-09-09T21:09:49.917389968Z INFO: Network log collection complete. 2026-09-09T21:09:49.963265576Z W0909 21:09:49.963207 1495 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-09-09T21:09:49.969528826Z Wrote inspect data to must-gather. 2026-09-09T21:09:50.083262126Z Wrote inspect data to /must-gather/istio. 2026-09-09T21:09:50.089801988Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-09T21:09:50.090831857Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-09T21:09:50.301344779Z Gathering data for ns/openshift-network-console... 2026-09-09T21:09:50.319143008Z Wrote inspect data to must-gather. 2026-09-09T21:09:50.321316945Z Wrote inspect data to /must-gather/istio. 2026-09-09T21:09:50.554510150Z Wrote inspect data to /must-gather/istio. 2026-09-09T21:09:50.559309931Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-09T21:09:50.631252176Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-09T21:09:50.636360778Z INFO: Fallback to identify the container image from release info 2026-09-09T21:09:50.801344309Z Wrote inspect data to /must-gather/istio. 2026-09-09T21:09:50.903066744Z Gathering data for ns/openshift-console-operator... 2026-09-09T21:09:51.013858238Z Wrote inspect data to /must-gather/istio. 2026-09-09T21:09:51.020367712Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-09T21:09:51.189129757Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f39c1daa89e19c5e987a3b759e646ffab6c1b2ab94a97ded81ddf13e73d0399 2026-09-09T21:09:51.263640166Z Wrote inspect data to /must-gather/istio. 2026-09-09T21:09:51.343109581Z daemonset.apps/perf-node-gather-daemonset created 2026-09-09T21:09:51.353711125Z Gathering data for ns/openshift-console... 2026-09-09T21:09:51.499604671Z Wrote inspect data to /must-gather/istio. 2026-09-09T21:09:51.504100809Z [disk usage checker] Volume usage percentage: current = 38 ; allowed = 70 2026-09-09T21:09:51.506794467Z Waiting for performance profile collector pods to become ready: 1 2026-09-09T21:09:51.507066781Z Inspecting resource crd/inferencepools.inference.networking.k8s.io 2026-09-09T21:09:51.691024105Z Wrote inspect data to /must-gather/istio. 2026-09-09T21:09:51.929843489Z Wrote inspect data to /must-gather/istio. 2026-09-09T21:09:51.934746814Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io 2026-09-09T21:09:51.983242411Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-09T21:09:52.128670726Z Wrote inspect data to /must-gather/istio. 2026-09-09T21:09:52.335738779Z Wrote inspect data to /must-gather/istio. 2026-09-09T21:09:52.341409221Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-09T21:09:52.551517450Z Wrote inspect data to /must-gather/istio. 2026-09-09T21:09:52.563022658Z Gathering data for ns/openshift-dns-operator... 2026-09-09T21:09:52.676507585Z Waiting for performance profile collector pods to become ready: 2 2026-09-09T21:09:52.786112077Z Wrote inspect data to /must-gather/istio. 2026-09-09T21:09:53.022683363Z Gathering data for ns/openshift-dns... 2026-09-09T21:09:53.489092543Z Done executing Istio gather script 2026-09-09T21:09:53.720157828Z Gathering data for ns/openshift-image-registry... 2026-09-09T21:09:53.806484781Z Waiting for performance profile collector pods to become ready: 3 2026-09-09T21:09:54.216475959Z Gathering data for ns/openshift-ingress-operator... 2026-09-09T21:09:54.693365961Z Gathering data for ns/openshift-ingress... 2026-09-09T21:09:54.944105739Z Waiting for performance profile collector pods to become ready: 4 2026-09-09T21:09:55.067493256Z Gathering data for ns/openshift-ingress-canary... 2026-09-09T21:09:55.487030073Z Gathering data for ns/openshift-insights... 2026-09-09T21:09:56.091935511Z Waiting for performance profile collector pods to become ready: 5 2026-09-09T21:09:56.510484225Z [disk usage checker] Volume usage percentage: current = 38 ; allowed = 70 2026-09-09T21:09:57.223804885Z Waiting for performance profile collector pods to become ready: 6 2026-09-09T21:09:58.098710765Z Gathering data for ns/kserve... 2026-09-09T21:09:58.354172110Z Waiting for performance profile collector pods to become ready: 7 2026-09-09T21:09:59.494280050Z Waiting for performance profile collector pods to become ready: 8 2026-09-09T21:10:00.402189105Z Gathering data for ns/openshift-config... 2026-09-09T21:10:00.636519255Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-09T21:10:00.710559389Z Gathering data for ns/openshift-config-managed... 2026-09-09T21:10:00.964718220Z Collecting performance related data for node ip-10-0-137-8.ec2.internal 2026-09-09T21:10:00.974722855Z Collecting performance related data for node ip-10-0-129-45.ec2.internal 2026-09-09T21:10:00.982940504Z Collecting performance related data for node ip-10-0-129-130.ec2.internal 2026-09-09T21:10:01.239067532Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-09T21:10:01.516829793Z [disk usage checker] Volume usage percentage: current = 38 ; allowed = 70 2026-09-09T21:10:01.569907165Z Gathering data for ns/openshift-kube-apiserver... 2026-09-09T21:10:01.950263944Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-09T21:10:02.405289559Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-09T21:10:02.785115400Z Gathering data for ns/openshift-kube-scheduler... 2026-09-09T21:10:03.156780364Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-09T21:10:03.541839496Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-09T21:10:03.937898478Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-09T21:10:04.288161775Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-09T21:10:04.826039016Z Gathering data for ns/openshift-multus... 2026-09-09T21:10:06.268037901Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-09T21:10:06.522152797Z [disk usage checker] Volume usage percentage: current = 38 ; allowed = 70 2026-09-09T21:10:07.501238896Z Gathering data for ns/openshift-host-network... 2026-09-09T21:10:07.842823727Z Gathering data for ns/openshift-network-diagnostics... 2026-09-09T21:10:08.372225022Z Gathering data for ns/openshift-network-node-identity... 2026-09-09T21:10:08.799325203Z Gathering data for ns/openshift-network-operator... 2026-09-09T21:10:09.189843234Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-09T21:10:09.462336783Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-09T21:10:09.657020671Z Collecting kubelet logs for node ip-10-0-137-8.ec2.internal 2026-09-09T21:10:09.658883870Z Collecting kubelet logs for node ip-10-0-129-45.ec2.internal 2026-09-09T21:10:09.660744639Z Collecting kubelet logs for node ip-10-0-129-130.ec2.internal 2026-09-09T21:10:10.006365029Z Gathering data for ns/openshift-apiserver-operator... 2026-09-09T21:10:10.111202257Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-w9tpv namespace 2026-09-09T21:10:10.117348678Z INFO: Node performance data collection complete. 2026-09-09T21:10:10.364011539Z Gathering data for ns/openshift-apiserver... 2026-09-09T21:10:10.805762560Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-09T21:10:11.091203791Z Gathering data for ns/openshift-controller-manager... 2026-09-09T21:10:11.361530644Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-09T21:10:11.526764128Z [disk usage checker] Volume usage percentage: current = 38 ; allowed = 70 2026-09-09T21:10:12.000021757Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-09T21:10:12.281600022Z Gathering data for ns/openshift-service-ca-operator... 2026-09-09T21:10:12.603461688Z Gathering data for ns/openshift-service-ca... 2026-09-09T21:10:12.963637016Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-09T21:10:13.838638903Z Wrote inspect data to must-gather. 2026-09-09T21:10:13.838704269Z error: inspection completed with the errors occurred while gathering data: 2026-09-09T21:10:13.838704269Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-09T21:10:13.970357084Z error: the server doesn't have a resource type "clusters" 2026-09-09T21:10:14.208997949Z Caches written to disk