2026-06-08T17:35:01.005099442Z [disk usage checker] Started
2026-06-08T17:35:01.008793817Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-08T17:35:01.210078416Z Gathering data for ns/openshift-cluster-version...
2026-06-08T17:35:01.325406649Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-08T17:35:01.402472967Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-08T17:35:01.474649756Z Gathering data for ns/default...
2026-06-08T17:35:01.766653765Z Gathering data for ns/openshift...
2026-06-08T17:35:02.145851980Z Gathering data for ns/kube-system...
2026-06-08T17:35:02.773719894Z Gathering data for ns/openshift-etcd...
2026-06-08T17:35:03.029434707Z Wrote inspect data to must-gather.
2026-06-08T17:35:03.029471623Z error: inspection completed with the errors occurred while gathering data:
2026-06-08T17:35:03.029471623Z     namespaces "assisted-installer" not found
2026-06-08T17:35:03.624462084Z Waiting on subprocesses to finish execution.
2026-06-08T17:35:03.635181887Z INFO: Gathering HAProxy config files
2026-06-08T17:35:03.637887481Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-08T17:35:03.640588426Z Executing Istio gather script
2026-06-08T17:35:03.672639921Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-08T17:35:03.679551324Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-08T17:35:03.683344997Z INFO: Collecting host service logs for crio
2026-06-08T17:35:03.683634888Z INFO: Collecting host service logs for kubelet
2026-06-08T17:35:03.683984900Z INFO: Collecting host service logs for rpm-ostreed
2026-06-08T17:35:03.684320385Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-08T17:35:03.684650065Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-08T17:35:03.684966843Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-08T17:35:03.685372260Z INFO: Collecting host service logs for NetworkManager
2026-06-08T17:35:03.685695512Z INFO: Collecting host service logs for openvswitch
2026-06-08T17:35:03.686022922Z INFO: Collecting host service logs for ovs-configuration
2026-06-08T17:35:03.686374052Z INFO: Collecting host service logs for ovsdb-server
2026-06-08T17:35:03.686790466Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-08T17:35:03.687090433Z INFO: Waiting for worker host service log collection to complete ...
2026-06-08T17:35:03.719032006Z INFO: Waiting for node performance related collection to complete ...
2026-06-08T17:35:04.316961372Z INFO: "metallb-operator" not detected. Skipping.
2026-06-08T17:35:04.404505260Z Wrote inspect data to must-gather.
2026-06-08T17:35:04.415101555Z INFO: Collecting Insights Archives from insights-operator-896549976-wh7p4
2026-06-08T17:35:04.415147124Z insights-runtime-extractor-2nckh
2026-06-08T17:35:04.415189490Z insights-runtime-extractor-jpgww
2026-06-08T17:35:04.415220756Z insights-runtime-extractor-qxxqs
2026-06-08T17:35:04.516639499Z No resources found in openshift-etcd namespace.
2026-06-08T17:35:04.685512057Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-08T17:35:04.695287596Z error: the server doesn't have a resource type "performanceprofile"
2026-06-08T17:35:04.783690681Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-08T17:35:04.863487717Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-08T17:35:04.863487717Z See 'oc rsync -h' for help and examples
2026-06-08T17:35:04.869097102Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-08T17:35:04.869140485Z INFO: on-disk MachineConfig config collection complete.
2026-06-08T17:35:04.960467762Z INFO: Worker host service log collection to complete.
2026-06-08T17:35:05.043248426Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-08T17:35:05.135946560Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-08T17:35:05.172596360Z No resources found
2026-06-08T17:35:05.246572379Z Wrote inspect data to must-gather.
2026-06-08T17:35:05.246655456Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-08T17:35:05.514810261Z ERROR: No running kube-apiserver pods found
2026-06-08T17:35:05.714084973Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-08T17:35:05.786285139Z Wrote inspect data to must-gather.
2026-06-08T17:35:05.862433690Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-08T17:35:05.950821858Z Wrote inspect data to must-gather.
2026-06-08T17:35:05.996112429Z Wrote inspect data to must-gather.
2026-06-08T17:35:05.997897817Z No resources found
2026-06-08T17:35:06.029116962Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-08T17:35:06.113380477Z Wrote inspect data to must-gather.
2026-06-08T17:35:06.116195378Z tar: Removing leading `/' from member names
2026-06-08T17:35:06.120052221Z INFO: HAProxy config collection complete.
2026-06-08T17:35:06.120943334Z Wrote inspect data to must-gather.
2026-06-08T17:35:06.150590531Z INFO: 'previous-logs' folder not found on ip-10-0-128-56.ec2.internal, skipping...
2026-06-08T17:35:06.176445615Z Gathering data for ns/openshift-monitoring...
2026-06-08T17:35:06.211077810Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-08T17:35:06.221435880Z error: the server doesn't have a resource type "machineconfigs"
2026-06-08T17:35:06.300707527Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-08T17:35:06.406179462Z Wrote inspect data to must-gather.
2026-06-08T17:35:06.422543240Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-08T17:35:06.467634431Z INFO: 'previous-logs' folder not found on ip-10-0-131-225.ec2.internal, skipping...
2026-06-08T17:35:06.661231143Z INFO: Getting rules from prometheus-k8s-0
2026-06-08T17:35:06.664308942Z Wrote inspect data to must-gather.
2026-06-08T17:35:06.683741245Z Wrote inspect data to must-gather.
2026-06-08T17:35:06.755046936Z INFO: 'previous-logs' folder not found on ip-10-0-140-198.ec2.internal, skipping...
2026-06-08T17:35:06.755086930Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-08T17:35:06.755086930Z INFO: Machine Config Daemon termination log collection complete.
2026-06-08T17:35:06.829422436Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-08T17:35:06.933578366Z Wrote inspect data to must-gather.
2026-06-08T17:35:07.032653372Z Wrote inspect data to must-gather.
2026-06-08T17:35:07.054892244Z INFO: Getting status/config from prometheus-k8s-0
2026-06-08T17:35:07.233340990Z INFO: INTERCONNECT MODE
2026-06-08T17:35:07.233371630Z INFO: Gathering ovn-kubernetes DBs
2026-06-08T17:35:07.242008629Z Wrote inspect data to must-gather.
2026-06-08T17:35:07.372405990Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-08T17:35:07.415856253Z INFO: Gathering OVN_Northbound from ovnkube-node-bvfmv...
2026-06-08T17:35:07.416096680Z INFO: Gathering OVN_Northbound from ovnkube-node-hdhzw...
2026-06-08T17:35:07.416604621Z INFO: Gathering OVN_Northbound from ovnkube-node-p5pkv...
2026-06-08T17:35:07.416835699Z INFO: Gathering OVN_Southbound from ovnkube-node-bvfmv...
2026-06-08T17:35:07.417183241Z INFO: Gathering OVN_Southbound from ovnkube-node-hdhzw...
2026-06-08T17:35:07.417484191Z INFO: Gathering OVN_Southbound from ovnkube-node-p5pkv...
2026-06-08T17:35:07.528816820Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-08T17:35:07.536003120Z Wrote inspect data to must-gather.
2026-06-08T17:35:07.538393479Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-08T17:35:07.673546853Z Wrote inspect data to /must-gather/istio.
2026-06-08T17:35:07.713940247Z tar: Removing leading `/' from member names
2026-06-08T17:35:07.739807288Z tar: Removing leading `/' from member names
2026-06-08T17:35:07.748302589Z tar: Removing leading `/' from member names
2026-06-08T17:35:07.759825269Z tar: Removing leading `/' from member names
2026-06-08T17:35:07.771310396Z tar: Removing leading `/' from member names
2026-06-08T17:35:07.798802556Z tar: Removing leading `/' from member names
2026-06-08T17:35:07.886869623Z Wrote inspect data to must-gather.
2026-06-08T17:35:07.904313430Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-08T17:35:07.948282985Z Wrote inspect data to /must-gather/istio.
2026-06-08T17:35:07.951821058Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-08T17:35:08.141670043Z Wrote inspect data to must-gather.
2026-06-08T17:35:08.163281475Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-08T17:35:08.188360671Z Wrote inspect data to /must-gather/istio.
2026-06-08T17:35:08.302206032Z INFO: Waiting for network log collection to complete ...
2026-06-08T17:35:08.302337425Z INFO: Waiting for ovnk database copies to complete ...
2026-06-08T17:35:08.302881289Z INFO: Copying ovnk databases complete.
2026-06-08T17:35:08.305322837Z 2.6M	must-gather/network_logs/ovnk_database_store
2026-06-08T17:35:08.308786986Z ovnk_database_store/
2026-06-08T17:35:08.308838883Z ovnk_database_store/ovnkube-node-bvfmv_nbdb
2026-06-08T17:35:08.314105815Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-08T17:35:08.316432408Z ovnk_database_store/ovnkube-node-bvfmv_sbdb
2026-06-08T17:35:08.332495668Z ovnk_database_store/ovnkube-node-hdhzw_nbdb
2026-06-08T17:35:08.340242273Z ovnk_database_store/ovnkube-node-p5pkv_sbdb
2026-06-08T17:35:08.359701578Z ovnk_database_store/ovnkube-node-p5pkv_nbdb
2026-06-08T17:35:08.366761298Z ovnk_database_store/ovnkube-node-hdhzw_sbdb
2026-06-08T17:35:08.389259886Z INFO: Network log collection complete.
2026-06-08T17:35:08.431696730Z Wrote inspect data to /must-gather/istio.
2026-06-08T17:35:08.434794367Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-08T17:35:08.468834809Z Wrote inspect data to must-gather.
2026-06-08T17:35:08.488257718Z Gathering data for ns/openshift-network-console...
2026-06-08T17:35:08.509756807Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-08T17:35:08.639536978Z Wrote inspect data to /must-gather/istio.
2026-06-08T17:35:08.776130594Z Wrote inspect data to must-gather.
2026-06-08T17:35:08.845763287Z Wrote inspect data to /must-gather/istio.
2026-06-08T17:35:08.848976162Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-08T17:35:08.942712427Z INFO: Getting status from alertmanager-main-0
2026-06-08T17:35:08.954087151Z Gathering data for ns/openshift-console-operator...
2026-06-08T17:35:09.089687039Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-08T17:35:09.091785658Z INFO: Fallback to identify the container image from release info
2026-06-08T17:35:09.111622011Z Wrote inspect data to /must-gather/istio.
2026-06-08T17:35:09.313139167Z Wrote inspect data to /must-gather/istio.
2026-06-08T17:35:09.315231601Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-08T17:35:09.363687197Z Gathering data for ns/openshift-console...
2026-06-08T17:35:09.482994274Z Wrote inspect data to /must-gather/istio.
2026-06-08T17:35:09.650188413Z Wrote inspect data to /must-gather/istio.
2026-06-08T17:35:09.840871054Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e
2026-06-08T17:35:09.860256589Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-08T17:35:09.977423716Z daemonset.apps/perf-node-gather-daemonset created
2026-06-08T17:35:10.108442707Z Waiting for performance profile collector pods to become ready: 1
2026-06-08T17:35:10.381717794Z Done executing Istio gather script
2026-06-08T17:35:10.389734746Z Gathering data for ns/openshift-dns-operator...
2026-06-08T17:35:10.672639232Z Gathering data for ns/openshift-dns...
2026-06-08T17:35:11.072273004Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-08T17:35:11.196154193Z Gathering data for ns/openshift-image-registry...
2026-06-08T17:35:11.227755038Z Waiting for performance profile collector pods to become ready: 2
2026-06-08T17:35:11.595872585Z Gathering data for ns/openshift-ingress-operator...
2026-06-08T17:35:11.885139971Z Gathering data for ns/openshift-ingress...
2026-06-08T17:35:12.185246948Z Gathering data for ns/openshift-ingress-canary...
2026-06-08T17:35:12.353953933Z Waiting for performance profile collector pods to become ready: 3
2026-06-08T17:35:12.523177049Z Gathering data for ns/openshift-insights...
2026-06-08T17:35:13.463908757Z Waiting for performance profile collector pods to become ready: 4
2026-06-08T17:35:14.284306229Z Gathering data for ns/openshift-config...
2026-06-08T17:35:14.573513616Z Gathering data for ns/openshift-config-managed...
2026-06-08T17:35:14.594893530Z Waiting for performance profile collector pods to become ready: 5
2026-06-08T17:35:14.925409373Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-08T17:35:15.188725463Z Gathering data for ns/openshift-kube-apiserver...
2026-06-08T17:35:15.402324421Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-08T17:35:15.642872722Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-08T17:35:15.726340265Z Waiting for performance profile collector pods to become ready: 6
2026-06-08T17:35:15.891255106Z Gathering data for ns/openshift-kube-scheduler...
2026-06-08T17:35:16.077512733Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-08T17:35:16.102107135Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-08T17:35:16.333460671Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-08T17:35:16.598086463Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-08T17:35:16.848922951Z Waiting for performance profile collector pods to become ready: 7
2026-06-08T17:35:16.867296776Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-08T17:35:17.292121073Z Gathering data for ns/openshift-multus...
2026-06-08T17:35:17.969846420Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-08T17:35:18.226001611Z Collecting performance related data for node ip-10-0-131-225.ec2.internal
2026-06-08T17:35:18.238094157Z Collecting performance related data for node ip-10-0-128-56.ec2.internal
2026-06-08T17:35:18.249854706Z Collecting performance related data for node ip-10-0-140-198.ec2.internal
2026-06-08T17:35:18.516015848Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-08T17:35:19.764993068Z Gathering data for ns/openshift-host-network...
2026-06-08T17:35:20.048982312Z Gathering data for ns/openshift-network-diagnostics...
2026-06-08T17:35:20.549502547Z Gathering data for ns/openshift-network-node-identity...
2026-06-08T17:35:20.938826250Z Gathering data for ns/openshift-network-operator...
2026-06-08T17:35:21.084355906Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-08T17:35:21.282325786Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-08T17:35:21.499631602Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-08T17:35:21.973884775Z Gathering data for ns/openshift-apiserver-operator...
2026-06-08T17:35:22.242468499Z Gathering data for ns/openshift-apiserver...
2026-06-08T17:35:22.581768436Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-08T17:35:22.822561709Z Gathering data for ns/openshift-controller-manager...
2026-06-08T17:35:23.075512295Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-08T17:35:23.700385486Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-08T17:35:23.954245049Z Gathering data for ns/openshift-service-ca-operator...
2026-06-08T17:35:24.226434992Z Gathering data for ns/openshift-service-ca...
2026-06-08T17:35:24.540441404Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-08T17:35:25.352375487Z Wrote inspect data to must-gather.
2026-06-08T17:35:25.352420439Z error: inspection completed with the errors occurred while gathering data:
2026-06-08T17:35:25.352420439Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-08T17:35:26.089610381Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-08T17:35:26.640349030Z Collecting kubelet logs for node ip-10-0-131-225.ec2.internal
2026-06-08T17:35:26.642127736Z Collecting kubelet logs for node ip-10-0-128-56.ec2.internal
2026-06-08T17:35:26.643940155Z Collecting kubelet logs for node ip-10-0-140-198.ec2.internal
2026-06-08T17:35:26.942182983Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-z62ll namespace
2026-06-08T17:35:26.946359212Z INFO: Node performance data collection complete.
2026-06-08T17:35:27.071722241Z error: the server doesn't have a resource type "clusters"
2026-06-08T17:35:27.232322777Z Caches written to disk