2026-07-29T11:05:48.277105864Z [disk usage checker] Started
2026-07-29T11:05:48.280813810Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-29T11:05:48.468533677Z Gathering data for ns/openshift-cluster-version...
2026-07-29T11:05:48.577915036Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-29T11:05:48.664275995Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-29T11:05:48.741868510Z Gathering data for ns/default...
2026-07-29T11:05:49.036703215Z Gathering data for ns/openshift...
2026-07-29T11:05:49.433315448Z Gathering data for ns/kube-system...
2026-07-29T11:05:50.121825055Z Gathering data for ns/openshift-etcd...
2026-07-29T11:05:50.406868022Z Wrote inspect data to must-gather.
2026-07-29T11:05:50.407031754Z error: inspection completed with the errors occurred while gathering data:
2026-07-29T11:05:50.407031754Z     namespaces "assisted-installer" not found
2026-07-29T11:05:51.015446159Z Waiting on subprocesses to finish execution.
2026-07-29T11:05:51.021479502Z INFO: Gathering HAProxy config files
2026-07-29T11:05:51.025975247Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-29T11:05:51.026063726Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-29T11:05:51.027495876Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-29T11:05:51.030039058Z INFO: Collecting host service logs for crio
2026-07-29T11:05:51.030435543Z INFO: Collecting host service logs for kubelet
2026-07-29T11:05:51.030772204Z INFO: Collecting host service logs for rpm-ostreed
2026-07-29T11:05:51.031150137Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-29T11:05:51.031503313Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-29T11:05:51.031835714Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-29T11:05:51.032212956Z INFO: Collecting host service logs for NetworkManager
2026-07-29T11:05:51.032549075Z INFO: Collecting host service logs for openvswitch
2026-07-29T11:05:51.032896999Z INFO: Collecting host service logs for ovs-configuration
2026-07-29T11:05:51.033337930Z INFO: Collecting host service logs for ovsdb-server
2026-07-29T11:05:51.033687388Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-29T11:05:51.034091172Z INFO: Waiting for worker host service log collection to complete ...
2026-07-29T11:05:51.035268102Z Executing Istio gather script
2026-07-29T11:05:51.115147984Z INFO: Waiting for node performance related collection to complete ...
2026-07-29T11:05:51.673155865Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-29T11:05:51.731955945Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-vm85k
2026-07-29T11:05:51.731995921Z insights-runtime-extractor-brmw9
2026-07-29T11:05:51.732015267Z insights-runtime-extractor-llm74
2026-07-29T11:05:51.732034082Z insights-runtime-extractor-s87xx
2026-07-29T11:05:51.743462976Z INFO: "metallb-operator" not detected. Skipping.
2026-07-29T11:05:51.778876343Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-29T11:05:51.778962107Z INFO: on-disk MachineConfig config collection complete.
2026-07-29T11:05:51.877951706Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-29T11:05:52.138498490Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-29T11:05:52.409947015Z No resources found in openshift-etcd namespace.
2026-07-29T11:05:52.443218556Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-29T11:05:52.512289844Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-29T11:05:52.512289844Z See 'oc rsync -h' for help and examples
2026-07-29T11:05:52.553367264Z No resources found
2026-07-29T11:05:52.590408392Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-29T11:05:52.642326980Z Wrote inspect data to must-gather.
2026-07-29T11:05:52.800168817Z Wrote inspect data to must-gather.
2026-07-29T11:05:52.804599584Z No resources found
2026-07-29T11:05:52.806822112Z INFO: 'previous-logs' folder not found on ip-10-0-133-233.ec2.internal, skipping...
2026-07-29T11:05:52.890230402Z INFO: Worker host service log collection to complete.
2026-07-29T11:05:52.933957973Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-29T11:05:53.262449562Z error: the server doesn't have a resource type "performanceprofile"
2026-07-29T11:05:53.274410365Z Wrote inspect data to must-gather.
2026-07-29T11:05:53.311027480Z ERROR: No running kube-apiserver pods found
2026-07-29T11:05:53.346588669Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-29T11:05:53.347358773Z Wrote inspect data to must-gather.
2026-07-29T11:05:53.348324002Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-29T11:05:53.361104409Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-29T11:05:53.414433298Z Wrote inspect data to must-gather.
2026-07-29T11:05:53.543992323Z INFO: 'previous-logs' folder not found on ip-10-0-137-111.ec2.internal, skipping...
2026-07-29T11:05:53.620982767Z Wrote inspect data to must-gather.
2026-07-29T11:05:53.736942875Z Wrote inspect data to must-gather.
2026-07-29T11:05:53.782341996Z tar: Removing leading `/' from member names
2026-07-29T11:05:53.786264551Z INFO: HAProxy config collection complete.
2026-07-29T11:05:53.873456737Z Wrote inspect data to must-gather.
2026-07-29T11:05:53.945011748Z INFO: 'previous-logs' folder not found on ip-10-0-138-43.ec2.internal, skipping...
2026-07-29T11:05:53.945139636Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-29T11:05:53.945196334Z INFO: Machine Config Daemon termination log collection complete.
2026-07-29T11:05:53.982314487Z error: the server doesn't have a resource type "machineconfigs"
2026-07-29T11:05:54.021267953Z INFO: Getting rules from prometheus-k8s-0
2026-07-29T11:05:54.046212675Z Gathering data for ns/openshift-monitoring...
2026-07-29T11:05:54.172841754Z Wrote inspect data to must-gather.
2026-07-29T11:05:54.184202172Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-29T11:05:54.441451670Z Wrote inspect data to must-gather.
2026-07-29T11:05:54.465053093Z Wrote inspect data to must-gather.
2026-07-29T11:05:54.477186614Z INFO: Getting status/config from prometheus-k8s-0
2026-07-29T11:05:54.638144442Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-29T11:05:54.791571362Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-29T11:05:54.799543701Z INFO: INTERCONNECT MODE
2026-07-29T11:05:54.799579922Z INFO: Gathering ovn-kubernetes DBs
2026-07-29T11:05:54.875747126Z Wrote inspect data to must-gather.
2026-07-29T11:05:54.947979099Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-29T11:05:55.064309659Z INFO: Gathering OVN_Northbound from ovnkube-node-6jzgn...
2026-07-29T11:05:55.064669899Z INFO: Gathering OVN_Northbound from ovnkube-node-psdkf...
2026-07-29T11:05:55.064988188Z INFO: Gathering OVN_Northbound from ovnkube-node-t7g4l...
2026-07-29T11:05:55.065353616Z INFO: Gathering OVN_Southbound from ovnkube-node-6jzgn...
2026-07-29T11:05:55.065647771Z INFO: Gathering OVN_Southbound from ovnkube-node-psdkf...
2026-07-29T11:05:55.065938386Z INFO: Gathering OVN_Southbound from ovnkube-node-t7g4l...
2026-07-29T11:05:55.165094534Z Wrote inspect data to must-gather.
2026-07-29T11:05:55.281735245Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-29T11:05:55.330051871Z tar: Removing leading `/' from member names
2026-07-29T11:05:55.366244246Z tar: Removing leading `/' from member names
2026-07-29T11:05:55.383307946Z tar: Removing leading `/' from member names
2026-07-29T11:05:55.385483109Z tar: Removing leading `/' from member names
2026-07-29T11:05:55.422188503Z tar: Removing leading `/' from member names
2026-07-29T11:05:55.425856390Z tar: Removing leading `/' from member names
2026-07-29T11:05:55.538827288Z Wrote inspect data to must-gather.
2026-07-29T11:05:55.546244081Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-29T11:05:55.590443497Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-29T11:05:55.831904666Z Wrote inspect data to must-gather.
2026-07-29T11:05:55.863514070Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-29T11:05:55.888398147Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-29T11:05:55.974110799Z INFO: Waiting for network log collection to complete ...
2026-07-29T11:05:55.974238314Z INFO: Waiting for ovnk database copies to complete ...
2026-07-29T11:05:55.974759265Z INFO: Copying ovnk databases complete.
2026-07-29T11:05:55.977016422Z 3.5M	must-gather/network_logs/ovnk_database_store
2026-07-29T11:05:55.979972212Z ovnk_database_store/
2026-07-29T11:05:55.979972212Z ovnk_database_store/ovnkube-node-psdkf_sbdb
2026-07-29T11:05:56.001781634Z ovnk_database_store/ovnkube-node-t7g4l_sbdb
2026-07-29T11:05:56.025754684Z ovnk_database_store/ovnkube-node-psdkf_nbdb
2026-07-29T11:05:56.035047560Z ovnk_database_store/ovnkube-node-t7g4l_nbdb
2026-07-29T11:05:56.043342398Z ovnk_database_store/ovnkube-node-6jzgn_sbdb
2026-07-29T11:05:56.066578723Z ovnk_database_store/ovnkube-node-6jzgn_nbdb
2026-07-29T11:05:56.077201263Z INFO: Network log collection complete.
2026-07-29T11:05:56.080532774Z Wrote inspect data to must-gather.
2026-07-29T11:05:56.085094984Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:05:56.244496370Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-29T11:05:56.332626224Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:05:56.337263174Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-29T11:05:56.341409918Z INFO: Getting status from alertmanager-main-0
2026-07-29T11:05:56.402143336Z Wrote inspect data to must-gather.
2026-07-29T11:05:56.453956508Z Gathering data for ns/openshift-network-console...
2026-07-29T11:05:56.552962666Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:05:56.724083981Z Wrote inspect data to must-gather.
2026-07-29T11:05:56.777176576Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:05:56.780615294Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-29T11:05:56.895622645Z Gathering data for ns/openshift-console-operator...
2026-07-29T11:05:56.990452912Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:05:57.036139353Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-29T11:05:57.038678288Z INFO: Fallback to identify the container image from release info
2026-07-29T11:05:57.201771636Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:05:57.204824856Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-29T11:05:57.298638452Z Gathering data for ns/openshift-console...
2026-07-29T11:05:57.439450423Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:05:57.620627246Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:05:57.622944413Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-07-29T11:05:57.799920472Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:05:57.812236914Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-29T11:05:57.994224316Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:05:57.998585298Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-07-29T11:05:58.170408042Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-29T11:05:58.191478400Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:05:58.320856905Z daemonset.apps/perf-node-gather-daemonset created
2026-07-29T11:05:58.369167791Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-29T11:05:58.386412840Z Gathering data for ns/openshift-dns-operator...
2026-07-29T11:05:58.423888419Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:05:58.427304763Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-29T11:05:58.483668668Z Waiting for performance profile collector pods to become ready: 1
2026-07-29T11:05:58.603745964Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:05:58.693979195Z Gathering data for ns/openshift-dns...
2026-07-29T11:05:58.812153671Z Wrote inspect data to /must-gather/istio.
2026-07-29T11:05:59.268309667Z Gathering data for ns/openshift-image-registry...
2026-07-29T11:05:59.512435106Z Done executing Istio gather script
2026-07-29T11:05:59.612836781Z Waiting for performance profile collector pods to become ready: 2
2026-07-29T11:05:59.722631445Z Gathering data for ns/openshift-ingress-operator...
2026-07-29T11:06:00.016415455Z Gathering data for ns/openshift-ingress...
2026-07-29T11:06:00.311195294Z Gathering data for ns/openshift-ingress-canary...
2026-07-29T11:06:00.650223567Z Gathering data for ns/openshift-insights...
2026-07-29T11:06:00.731467348Z Waiting for performance profile collector pods to become ready: 3
2026-07-29T11:06:01.859458587Z Waiting for performance profile collector pods to become ready: 4
2026-07-29T11:06:02.922059357Z Gathering data for ns/kserve...
2026-07-29T11:06:02.982280002Z Waiting for performance profile collector pods to become ready: 5
2026-07-29T11:06:03.375665398Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-29T11:06:04.108064373Z Waiting for performance profile collector pods to become ready: 6
2026-07-29T11:06:04.463185794Z Gathering data for ns/openshift-config...
2026-07-29T11:06:04.708748229Z Gathering data for ns/openshift-config-managed...
2026-07-29T11:06:05.099323577Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-29T11:06:05.228057622Z Waiting for performance profile collector pods to become ready: 7
2026-07-29T11:06:05.344392558Z Gathering data for ns/openshift-kube-apiserver...
2026-07-29T11:06:05.583203417Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-29T11:06:05.848808952Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-29T11:06:06.117634687Z Gathering data for ns/openshift-kube-scheduler...
2026-07-29T11:06:06.348131125Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-29T11:06:06.418658614Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-29T11:06:06.632111757Z Collecting performance related data for node ip-10-0-137-111.ec2.internal
2026-07-29T11:06:06.640884030Z Collecting performance related data for node ip-10-0-138-43.ec2.internal
2026-07-29T11:06:06.654863652Z Collecting performance related data for node ip-10-0-133-233.ec2.internal
2026-07-29T11:06:06.760372613Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-29T11:06:07.128453837Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-29T11:06:07.488825926Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-29T11:06:08.134019230Z Gathering data for ns/openshift-multus...
2026-07-29T11:06:08.380166978Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-29T11:06:09.504096308Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-29T11:06:10.673498956Z Gathering data for ns/openshift-host-network...
2026-07-29T11:06:10.964597286Z Gathering data for ns/openshift-network-diagnostics...
2026-07-29T11:06:11.459134405Z Gathering data for ns/openshift-network-node-identity...
2026-07-29T11:06:11.990214691Z Gathering data for ns/openshift-network-operator...
2026-07-29T11:06:12.351390540Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-29T11:06:12.646281310Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-29T11:06:13.098406983Z Gathering data for ns/openshift-apiserver-operator...
2026-07-29T11:06:13.386527443Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-29T11:06:13.431161273Z Gathering data for ns/openshift-apiserver...
2026-07-29T11:06:13.875915619Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-29T11:06:14.162525898Z Gathering data for ns/openshift-controller-manager...
2026-07-29T11:06:14.433055762Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-29T11:06:14.841146683Z Collecting kubelet logs for node ip-10-0-137-111.ec2.internal
2026-07-29T11:06:14.842873973Z Collecting kubelet logs for node ip-10-0-138-43.ec2.internal
2026-07-29T11:06:14.844605986Z Collecting kubelet logs for node ip-10-0-133-233.ec2.internal
2026-07-29T11:06:15.111339372Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-29T11:06:15.237062570Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-mxlpr namespace
2026-07-29T11:06:15.241323030Z INFO: Node performance data collection complete.
2026-07-29T11:06:15.403695559Z Gathering data for ns/openshift-service-ca-operator...
2026-07-29T11:06:15.698345838Z Gathering data for ns/openshift-service-ca...
2026-07-29T11:06:16.042901712Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-29T11:06:16.859401317Z Wrote inspect data to must-gather.
2026-07-29T11:06:16.859447217Z error: inspection completed with the errors occurred while gathering data:
2026-07-29T11:06:16.859447217Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-29T11:06:16.986615842Z error: the server doesn't have a resource type "clusters"
2026-07-29T11:06:17.114429465Z Caches written to disk