2026-07-06T15:37:31.451606389Z [disk usage checker] Started 2026-07-06T15:37:31.455313247Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-06T15:37:31.664232334Z Gathering data for ns/openshift-cluster-version... 2026-07-06T15:37:31.818777928Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-06T15:37:31.921604186Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-06T15:37:32.013751982Z Gathering data for ns/default... 2026-07-06T15:37:32.346603363Z Gathering data for ns/openshift... 2026-07-06T15:37:32.816391808Z Gathering data for ns/kube-system... 2026-07-06T15:37:33.771727024Z Gathering data for ns/openshift-etcd... 2026-07-06T15:37:34.167904686Z Wrote inspect data to must-gather. 2026-07-06T15:37:34.168066013Z error: inspection completed with the errors occurred while gathering data: 2026-07-06T15:37:34.168066013Z namespaces "assisted-installer" not found 2026-07-06T15:37:34.224647200Z Waiting on subprocesses to finish execution. 2026-07-06T15:37:34.232626214Z INFO: Gathering HAProxy config files 2026-07-06T15:37:34.233594525Z INFO: Gathering machine config daemon's old logs from all nodes 2026-07-06T15:37:34.245703409Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-07-06T15:37:34.251250547Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-07-06T15:37:34.256600934Z Executing Istio gather script 2026-07-06T15:37:34.259077032Z INFO: Waiting for node performance related collection to complete ... 2026-07-06T15:37:34.352705778Z INFO: Collecting host service logs for crio 2026-07-06T15:37:34.352979284Z INFO: Collecting host service logs for kubelet 2026-07-06T15:37:34.353274966Z INFO: Collecting host service logs for rpm-ostreed 2026-07-06T15:37:34.353593424Z INFO: Collecting host service logs for ostree-finalize-staged 2026-07-06T15:37:34.354012966Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-07-06T15:37:34.354307071Z INFO: Collecting host service logs for machine-config-daemon-host 2026-07-06T15:37:34.354591434Z INFO: Collecting host service logs for NetworkManager 2026-07-06T15:37:34.354871771Z INFO: Collecting host service logs for openvswitch 2026-07-06T15:37:34.355138123Z INFO: Collecting host service logs for ovs-configuration 2026-07-06T15:37:34.355441268Z INFO: Collecting host service logs for ovsdb-server 2026-07-06T15:37:34.355729238Z INFO: Collecting host service logs for ovs-vswitchd 2026-07-06T15:37:34.356077030Z INFO: Waiting for worker host service log collection to complete ... 2026-07-06T15:37:34.917262464Z INFO: Found 1 replicas - prometheus-k8s-0 2026-07-06T15:37:35.076310363Z No resources found in openshift-etcd namespace. 2026-07-06T15:37:35.177092781Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-07-06T15:37:35.257951312Z INFO: Collecting Insights Archives from insights-operator-5c847dcfc4-zsgbz 2026-07-06T15:37:35.258266108Z insights-runtime-extractor-27497 2026-07-06T15:37:35.258325772Z insights-runtime-extractor-bg9nt 2026-07-06T15:37:35.258354247Z insights-runtime-extractor-l9jx9 2026-07-06T15:37:35.327175715Z error: the server doesn't have a resource type "performanceprofile" 2026-07-06T15:37:35.435749028Z INFO: "sriov-network-operator" not detected. Skipping. 2026-07-06T15:37:35.598933485Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-07-06T15:37:35.599033723Z INFO: on-disk MachineConfig config collection complete. 2026-07-06T15:37:35.613444205Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-07-06T15:37:35.655354071Z No resources found 2026-07-06T15:37:35.737186720Z Wrote inspect data to must-gather. 2026-07-06T15:37:35.771108587Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-07-06T15:37:35.771108587Z See 'oc rsync -h' for help and examples 2026-07-06T15:37:35.797715993Z INFO: "metallb-operator" not detected. Skipping. 2026-07-06T15:37:35.945409680Z Wrote inspect data to must-gather. 2026-07-06T15:37:36.095458146Z INFO: Worker host service log collection to complete. 2026-07-06T15:37:36.125770183Z ERROR: No running kube-apiserver pods found 2026-07-06T15:37:36.282804682Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-07-06T15:37:36.313255587Z Wrote inspect data to must-gather. 2026-07-06T15:37:36.489961295Z No resources found 2026-07-06T15:37:36.510519517Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-07-06T15:37:36.562542608Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-06T15:37:36.701210357Z Wrote inspect data to must-gather. 2026-07-06T15:37:36.772937042Z Wrote inspect data to must-gather. 2026-07-06T15:37:36.802493950Z Gathering data for ns/openshift-monitoring... 2026-07-06T15:37:36.805182655Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-06T15:37:36.866741347Z INFO: Waiting for HAProxy config collection to complete ... 2026-07-06T15:37:36.890649096Z INFO: 'previous-logs' folder not found on ip-10-0-128-213.ec2.internal, skipping... 2026-07-06T15:37:36.962762700Z error: the server doesn't have a resource type "machineconfigs" 2026-07-06T15:37:36.986928829Z Wrote inspect data to must-gather. 2026-07-06T15:37:37.081900744Z Wrote inspect data to must-gather. 2026-07-06T15:37:37.137818568Z INFO: Getting rules from prometheus-k8s-0 2026-07-06T15:37:37.194995953Z error: the server doesn't have a resource type "machineconfigpools" 2026-07-06T15:37:37.254054422Z INFO: HAProxy config collection complete. 2026-07-06T15:37:37.303073489Z INFO: 'previous-logs' folder not found on ip-10-0-131-86.ec2.internal, skipping... 2026-07-06T15:37:37.407324981Z Wrote inspect data to must-gather. 2026-07-06T15:37:37.482144667Z Wrote inspect data to must-gather. 2026-07-06T15:37:37.585693476Z INFO: 'previous-logs' folder not found on ip-10-0-139-31.ec2.internal, skipping... 2026-07-06T15:37:37.585729637Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-07-06T15:37:37.585763058Z INFO: Machine Config Daemon termination log collection complete. 2026-07-06T15:37:37.627236309Z INFO: Getting status/config from prometheus-k8s-0 2026-07-06T15:37:37.659143522Z error: the server doesn't have a resource type "kubeletconfigs" 2026-07-06T15:37:37.700012960Z Wrote inspect data to must-gather. 2026-07-06T15:37:37.901275359Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-07-06T15:37:37.919003605Z Wrote inspect data to must-gather. 2026-07-06T15:37:37.949569349Z Wrote inspect data to must-gather. 2026-07-06T15:37:38.075202578Z INFO: Getting status/flags from prometheus-k8s-0 2026-07-06T15:37:38.144168902Z Wrote inspect data to must-gather. 2026-07-06T15:37:38.270502961Z INFO: INTERCONNECT MODE 2026-07-06T15:37:38.270535665Z INFO: Gathering ovn-kubernetes DBs 2026-07-06T15:37:38.330618980Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-07-06T15:37:38.373307453Z Wrote inspect data to must-gather. 2026-07-06T15:37:38.434751100Z INFO: Gathering OVN_Northbound from ovnkube-node-7szf4... 2026-07-06T15:37:38.435100657Z INFO: Gathering OVN_Northbound from ovnkube-node-9v2mt... 2026-07-06T15:37:38.435366694Z INFO: Gathering OVN_Northbound from ovnkube-node-czwzb... 2026-07-06T15:37:38.435651337Z INFO: Gathering OVN_Southbound from ovnkube-node-7szf4... 2026-07-06T15:37:38.435933397Z INFO: Gathering OVN_Southbound from ovnkube-node-9v2mt... 2026-07-06T15:37:38.436187657Z INFO: Gathering OVN_Southbound from ovnkube-node-czwzb... 2026-07-06T15:37:38.625990122Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-07-06T15:37:38.738618964Z Wrote inspect data to must-gather. 2026-07-06T15:37:38.754389149Z tar: Removing leading `/' from member names 2026-07-06T15:37:38.807218639Z tar: Removing leading `/' from member names 2026-07-06T15:37:38.814745124Z tar: Removing leading `/' from member names 2026-07-06T15:37:38.843822263Z tar: Removing leading `/' from member names 2026-07-06T15:37:38.844855470Z tar: Removing leading `/' from member names 2026-07-06T15:37:38.889482132Z tar: Removing leading `/' from member names 2026-07-06T15:37:38.921727375Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-07-06T15:37:38.948546499Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-06T15:37:39.008130666Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-07-06T15:37:39.009848844Z Wrote inspect data to must-gather. 2026-07-06T15:37:39.151839731Z Wrote inspect data to /must-gather/istio. 2026-07-06T15:37:39.278871694Z Wrote inspect data to must-gather. 2026-07-06T15:37:39.382157096Z INFO: Waiting for network log collection to complete ... 2026-07-06T15:37:39.382404732Z INFO: Waiting for ovnk database copies to complete ... 2026-07-06T15:37:39.382691012Z INFO: Copying ovnk databases complete. 2026-07-06T15:37:39.384757900Z 3.0M must-gather/network_logs/ovnk_database_store 2026-07-06T15:37:39.390802537Z ovnk_database_store/ 2026-07-06T15:37:39.390802537Z ovnk_database_store/ovnkube-node-czwzb_nbdb 2026-07-06T15:37:39.402668923Z ovnk_database_store/ovnkube-node-9v2mt_nbdb 2026-07-06T15:37:39.403694318Z Wrote inspect data to /must-gather/istio. 2026-07-06T15:37:39.407064353Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-07-06T15:37:39.412234865Z ovnk_database_store/ovnkube-node-7szf4_sbdb 2026-07-06T15:37:39.430518176Z ovnk_database_store/ovnkube-node-czwzb_sbdb 2026-07-06T15:37:39.454143219Z ovnk_database_store/ovnkube-node-9v2mt_sbdb 2026-07-06T15:37:39.474004075Z ovnk_database_store/ovnkube-node-7szf4_nbdb 2026-07-06T15:37:39.482909201Z INFO: Getting status from alertmanager-main-0 2026-07-06T15:37:39.483658384Z INFO: Network log collection complete. 2026-07-06T15:37:39.532112881Z W0706 15:37:39.532064 1508 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-07-06T15:37:39.538422662Z Wrote inspect data to must-gather. 2026-07-06T15:37:39.633546381Z Wrote inspect data to /must-gather/istio. 2026-07-06T15:37:39.643828334Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-06T15:37:39.837893520Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-07-06T15:37:39.837940245Z Gathering data for ns/openshift-network-console... 2026-07-06T15:37:39.841525309Z INFO: Fallback to identify the container image from release info 2026-07-06T15:37:39.861251189Z Wrote inspect data to /must-gather/istio. 2026-07-06T15:37:39.863627497Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-07-06T15:37:40.066948768Z Wrote inspect data to /must-gather/istio. 2026-07-06T15:37:40.271058425Z Wrote inspect data to /must-gather/istio. 2026-07-06T15:37:40.274199250Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-07-06T15:37:40.316267354Z Gathering data for ns/openshift-console-operator... 2026-07-06T15:37:40.491512885Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5be8cdcf34f35a89a70b6bfa49e66897641871237570b3e09766f83a0a473e6a 2026-07-06T15:37:40.500861981Z Wrote inspect data to /must-gather/istio. 2026-07-06T15:37:40.630266706Z daemonset.apps/perf-node-gather-daemonset created 2026-07-06T15:37:40.720627589Z Wrote inspect data to /must-gather/istio. 2026-07-06T15:37:40.725147622Z Inspecting resource crd/inferencemodelrewrites.inference.networking.x-k8s.io 2026-07-06T15:37:40.786321619Z Waiting for performance profile collector pods to become ready: 1 2026-07-06T15:37:40.795803253Z Gathering data for ns/openshift-console... 2026-07-06T15:37:40.905418166Z Wrote inspect data to /must-gather/istio. 2026-07-06T15:37:41.111903203Z Wrote inspect data to /must-gather/istio. 2026-07-06T15:37:41.115832418Z Inspecting resource crd/inferenceobjectives.inference.networking.x-k8s.io 2026-07-06T15:37:41.299224642Z Wrote inspect data to /must-gather/istio. 2026-07-06T15:37:41.374748896Z Gathering data for ns/openshift-cluster-storage-operator... 2026-07-06T15:37:41.498791408Z Wrote inspect data to /must-gather/istio. 2026-07-06T15:37:41.501106052Z Inspecting resource crd/inferencepoolimports.inference.networking.x-k8s.io 2026-07-06T15:37:41.635465253Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-06T15:37:41.672701984Z Wrote inspect data to /must-gather/istio. 2026-07-06T15:37:41.883030485Z Wrote inspect data to /must-gather/istio. 2026-07-06T15:37:41.887088021Z Inspecting resource crd/inferencepools.inference.networking.k8s.io 2026-07-06T15:37:41.936414302Z Waiting for performance profile collector pods to become ready: 2 2026-07-06T15:37:42.000804502Z Gathering data for ns/openshift-dns-operator... 2026-07-06T15:37:42.081252768Z Wrote inspect data to /must-gather/istio. 2026-07-06T15:37:42.271065993Z Wrote inspect data to /must-gather/istio. 2026-07-06T15:37:42.274158572Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io 2026-07-06T15:37:42.336160049Z Gathering data for ns/openshift-dns... 2026-07-06T15:37:42.443044622Z Wrote inspect data to /must-gather/istio. 2026-07-06T15:37:42.609581477Z Wrote inspect data to /must-gather/istio. 2026-07-06T15:37:42.611688030Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-07-06T15:37:42.780522881Z Wrote inspect data to /must-gather/istio. 2026-07-06T15:37:42.990242367Z Wrote inspect data to /must-gather/istio. 2026-07-06T15:37:43.072252154Z Gathering data for ns/openshift-image-registry... 2026-07-06T15:37:43.089700156Z Waiting for performance profile collector pods to become ready: 3 2026-07-06T15:37:43.637844326Z Gathering data for ns/openshift-ingress-operator... 2026-07-06T15:37:43.685065830Z Done executing Istio gather script 2026-07-06T15:37:44.005469056Z Gathering data for ns/openshift-ingress... 2026-07-06T15:37:44.208470567Z Waiting for performance profile collector pods to become ready: 4 2026-07-06T15:37:44.390747514Z Gathering data for ns/openshift-ingress-canary... 2026-07-06T15:37:44.848961285Z Gathering data for ns/openshift-insights... 2026-07-06T15:37:45.334077971Z Waiting for performance profile collector pods to become ready: 5 2026-07-06T15:37:46.456952559Z Waiting for performance profile collector pods to become ready: 6 2026-07-06T15:37:46.642439423Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-06T15:37:47.234936592Z Gathering data for ns/kserve... 2026-07-06T15:37:47.590294814Z Waiting for performance profile collector pods to become ready: 7 2026-07-06T15:37:48.709735844Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-07-06T15:37:48.711399392Z Gathering data for ns/openshift-config... 2026-07-06T15:37:48.975075724Z Collecting performance related data for node ip-10-0-131-86.ec2.internal 2026-07-06T15:37:48.984840560Z Collecting performance related data for node ip-10-0-139-31.ec2.internal 2026-07-06T15:37:48.992538616Z Collecting performance related data for node ip-10-0-128-213.ec2.internal 2026-07-06T15:37:49.036095521Z Gathering data for ns/openshift-config-managed... 2026-07-06T15:37:49.637685135Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-07-06T15:37:50.006668606Z Gathering data for ns/openshift-kube-apiserver... 2026-07-06T15:37:50.403099844Z Gathering data for ns/openshift-kube-controller-manager... 2026-07-06T15:37:50.789718195Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-07-06T15:37:51.164635941Z Gathering data for ns/openshift-kube-scheduler... 2026-07-06T15:37:51.499028717Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-07-06T15:37:51.647116358Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-06T15:37:51.914800405Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-07-06T15:37:52.332702557Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-07-06T15:37:52.684560754Z Gathering data for ns/openshift-user-workload-monitoring... 2026-07-06T15:37:53.337730940Z Gathering data for ns/openshift-multus... 2026-07-06T15:37:55.148932072Z Gathering data for ns/openshift-ovn-kubernetes... 2026-07-06T15:37:56.653255381Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-06T15:37:56.729497815Z Gathering data for ns/openshift-host-network... 2026-07-06T15:37:56.809599682Z Collecting kubelet logs for node ip-10-0-131-86.ec2.internal 2026-07-06T15:37:56.811349555Z Collecting kubelet logs for node ip-10-0-139-31.ec2.internal 2026-07-06T15:37:56.812976054Z Collecting kubelet logs for node ip-10-0-128-213.ec2.internal 2026-07-06T15:37:57.178568935Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-gmfmg namespace 2026-07-06T15:37:57.184609620Z INFO: Node performance data collection complete. 2026-07-06T15:37:57.246542805Z Gathering data for ns/openshift-network-diagnostics... 2026-07-06T15:37:57.982116488Z Gathering data for ns/openshift-network-node-identity... 2026-07-06T15:37:58.473940235Z Gathering data for ns/openshift-network-operator... 2026-07-06T15:37:58.965452545Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-07-06T15:37:59.359214333Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-07-06T15:38:00.007215800Z Gathering data for ns/openshift-apiserver-operator... 2026-07-06T15:38:00.398991078Z Gathering data for ns/openshift-apiserver... 2026-07-06T15:38:00.917779186Z Gathering data for ns/openshift-controller-manager-operator... 2026-07-06T15:38:01.324369187Z Gathering data for ns/openshift-controller-manager... 2026-07-06T15:38:01.651445954Z Gathering data for ns/openshift-cluster-samples-operator... 2026-07-06T15:38:01.657739546Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-06T15:38:02.349045557Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-07-06T15:38:02.755812418Z Gathering data for ns/openshift-service-ca-operator... 2026-07-06T15:38:03.112883120Z Gathering data for ns/openshift-service-ca... 2026-07-06T15:38:03.567732070Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-07-06T15:38:04.648963185Z Wrote inspect data to must-gather. 2026-07-06T15:38:04.649005602Z error: inspection completed with the errors occurred while gathering data: 2026-07-06T15:38:04.649005602Z skipping gathering secrets/support due to error: secrets "support" not found 2026-07-06T15:38:04.777753662Z error: the server doesn't have a resource type "clusters" 2026-07-06T15:38:04.913778518Z Caches written to disk