2026-07-14T12:02:55.361256076Z [disk usage checker] Started
2026-07-14T12:02:55.365074467Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-14T12:02:55.561640768Z Gathering data for ns/openshift-cluster-version...
2026-07-14T12:02:55.705321647Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-14T12:02:55.796881196Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-14T12:02:55.883231846Z Gathering data for ns/default...
2026-07-14T12:02:56.163664539Z Gathering data for ns/openshift...
2026-07-14T12:02:56.590647427Z Gathering data for ns/kube-system...
2026-07-14T12:02:57.275934873Z Gathering data for ns/openshift-etcd...
2026-07-14T12:02:57.574282035Z Wrote inspect data to must-gather.
2026-07-14T12:02:57.574317127Z error: inspection completed with the errors occurred while gathering data:
2026-07-14T12:02:57.574317127Z     namespaces "assisted-installer" not found
2026-07-14T12:02:57.925342691Z Waiting on subprocesses to finish execution.
2026-07-14T12:02:57.933199082Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-14T12:02:57.935683107Z Executing Istio gather script
2026-07-14T12:02:57.936474084Z INFO: Collecting host service logs for crio
2026-07-14T12:02:57.936583685Z INFO: Collecting host service logs for kubelet
2026-07-14T12:02:57.936910682Z INFO: Collecting host service logs for rpm-ostreed
2026-07-14T12:02:57.937224445Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-14T12:02:57.937535325Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-14T12:02:57.937832901Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-14T12:02:57.938209661Z INFO: Collecting host service logs for NetworkManager
2026-07-14T12:02:57.938578501Z INFO: Collecting host service logs for openvswitch
2026-07-14T12:02:57.938868433Z INFO: Collecting host service logs for ovs-configuration
2026-07-14T12:02:57.939161190Z INFO: Collecting host service logs for ovsdb-server
2026-07-14T12:02:57.939393700Z INFO: Gathering HAProxy config files
2026-07-14T12:02:57.939532123Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-14T12:02:57.939899464Z INFO: Waiting for worker host service log collection to complete ...
2026-07-14T12:02:57.941131199Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-14T12:02:57.943663151Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-14T12:02:57.971611565Z INFO: Waiting for node performance related collection to complete ...
2026-07-14T12:02:58.797411001Z No resources found
2026-07-14T12:02:58.847363239Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-14T12:02:58.891553643Z error: the server doesn't have a resource type "performanceprofile"
2026-07-14T12:02:59.036483933Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-14T12:02:59.036532191Z INFO: on-disk MachineConfig config collection complete.
2026-07-14T12:02:59.165471737Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-14T12:02:59.194140121Z No resources found
2026-07-14T12:02:59.200727990Z No resources found in openshift-etcd namespace.
2026-07-14T12:02:59.461606062Z INFO: Worker host service log collection to complete.
2026-07-14T12:02:59.603714205Z INFO: Collecting Insights Archives from insights-operator-68d9dd94cf-bdfpz
2026-07-14T12:02:59.603860522Z insights-runtime-extractor-5xpdr
2026-07-14T12:02:59.603882570Z insights-runtime-extractor-7mth5
2026-07-14T12:02:59.603902765Z insights-runtime-extractor-bjz69
2026-07-14T12:02:59.714317084Z Wrote inspect data to must-gather.
2026-07-14T12:02:59.772526052Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-14T12:02:59.788812543Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-14T12:02:59.913933861Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-14T12:02:59.938920462Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-14T12:02:59.956523960Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-14T12:02:59.973348054Z INFO: "metallb-operator" not detected. Skipping.
2026-07-14T12:03:00.001929083Z Wrote inspect data to must-gather.
2026-07-14T12:03:00.033645339Z Wrote inspect data to must-gather.
2026-07-14T12:03:00.037672391Z Wrote inspect data to must-gather.
2026-07-14T12:03:00.096447701Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-14T12:03:00.097880225Z Gathering data for ns/openshift-monitoring...
2026-07-14T12:03:00.128144534Z Wrote inspect data to must-gather.
2026-07-14T12:03:00.191418569Z tar: Removing leading `/' from member names
2026-07-14T12:03:00.197159528Z INFO: HAProxy config collection complete.
2026-07-14T12:03:00.213214412Z INFO: 'previous-logs' folder not found on ip-10-0-133-208.ec2.internal, skipping...
2026-07-14T12:03:00.379765718Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-14T12:03:00.379765718Z See 'oc rsync -h' for help and examples
2026-07-14T12:03:00.390253562Z Wrote inspect data to must-gather.
2026-07-14T12:03:00.394451053Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-14T12:03:00.422674313Z error: the server doesn't have a resource type "machineconfigs"
2026-07-14T12:03:00.439136600Z ERROR: No running kube-apiserver pods found
2026-07-14T12:03:00.495410943Z Wrote inspect data to must-gather.
2026-07-14T12:03:00.553003211Z Wrote inspect data to must-gather.
2026-07-14T12:03:00.637390094Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-14T12:03:00.640916872Z INFO: 'previous-logs' folder not found on ip-10-0-138-106.ec2.internal, skipping...
2026-07-14T12:03:00.755767930Z INFO: Getting rules from prometheus-k8s-0
2026-07-14T12:03:00.908204327Z Wrote inspect data to must-gather.
2026-07-14T12:03:00.915641958Z Wrote inspect data to must-gather.
2026-07-14T12:03:00.946129377Z INFO: 'previous-logs' folder not found on ip-10-0-143-41.ec2.internal, skipping...
2026-07-14T12:03:00.946161240Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-14T12:03:00.946220151Z INFO: Machine Config Daemon termination log collection complete.
2026-07-14T12:03:01.040739022Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-14T12:03:01.090763735Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-14T12:03:01.186474738Z Wrote inspect data to must-gather.
2026-07-14T12:03:01.204810620Z INFO: Getting status/config from prometheus-k8s-0
2026-07-14T12:03:01.319426949Z Wrote inspect data to must-gather.
2026-07-14T12:03:01.493024195Z INFO: INTERCONNECT MODE
2026-07-14T12:03:01.493064045Z INFO: Gathering ovn-kubernetes DBs
2026-07-14T12:03:01.544102532Z Wrote inspect data to must-gather.
2026-07-14T12:03:01.597986952Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-14T12:03:01.656210935Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-14T12:03:01.713405778Z INFO: Gathering OVN_Northbound from ovnkube-node-g69rz...
2026-07-14T12:03:01.713751118Z INFO: Gathering OVN_Northbound from ovnkube-node-pkv79...
2026-07-14T12:03:01.714034313Z INFO: Gathering OVN_Northbound from ovnkube-node-tpm5l...
2026-07-14T12:03:01.714367920Z INFO: Gathering OVN_Southbound from ovnkube-node-g69rz...
2026-07-14T12:03:01.717382583Z INFO: Gathering OVN_Southbound from ovnkube-node-pkv79...
2026-07-14T12:03:01.717759256Z INFO: Gathering OVN_Southbound from ovnkube-node-tpm5l...
2026-07-14T12:03:01.831096083Z Wrote inspect data to must-gather.
2026-07-14T12:03:01.849698756Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-14T12:03:01.963786242Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-14T12:03:02.001983121Z Wrote inspect data to /must-gather/istio.
2026-07-14T12:03:02.078195919Z tar: Removing leading `/' from member names
2026-07-14T12:03:02.091969456Z tar: Removing leading `/' from member names
2026-07-14T12:03:02.098029494Z tar: Removing leading `/' from member names
2026-07-14T12:03:02.106490541Z tar: Removing leading `/' from member names
2026-07-14T12:03:02.115947499Z tar: Removing leading `/' from member names
2026-07-14T12:03:02.124309572Z tar: Removing leading `/' from member names
2026-07-14T12:03:02.149095172Z Wrote inspect data to must-gather.
2026-07-14T12:03:02.266266713Z Wrote inspect data to /must-gather/istio.
2026-07-14T12:03:02.273574959Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-14T12:03:02.289296222Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-14T12:03:02.400088942Z Wrote inspect data to must-gather.
2026-07-14T12:03:02.526204548Z Wrote inspect data to /must-gather/istio.
2026-07-14T12:03:02.590837520Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-14T12:03:02.625712164Z INFO: Waiting for network log collection to complete ...
2026-07-14T12:03:02.625825683Z INFO: Waiting for ovnk database copies to complete ...
2026-07-14T12:03:02.626343128Z INFO: Copying ovnk databases complete.
2026-07-14T12:03:02.629190021Z 2.4M	must-gather/network_logs/ovnk_database_store
2026-07-14T12:03:02.631571740Z ovnk_database_store/
2026-07-14T12:03:02.631571740Z ovnk_database_store/ovnkube-node-tpm5l_nbdb
2026-07-14T12:03:02.639840034Z ovnk_database_store/ovnkube-node-pkv79_sbdb
2026-07-14T12:03:02.650780267Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-14T12:03:02.655304261Z ovnk_database_store/ovnkube-node-g69rz_sbdb
2026-07-14T12:03:02.672354313Z ovnk_database_store/ovnkube-node-g69rz_nbdb
2026-07-14T12:03:02.682516519Z ovnk_database_store/ovnkube-node-pkv79_nbdb
2026-07-14T12:03:02.690209517Z ovnk_database_store/ovnkube-node-tpm5l_sbdb
2026-07-14T12:03:02.709097889Z INFO: Network log collection complete.
2026-07-14T12:03:02.744440223Z Wrote inspect data to must-gather.
2026-07-14T12:03:02.774322458Z Wrote inspect data to /must-gather/istio.
2026-07-14T12:03:02.778053275Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-14T12:03:02.862967231Z Gathering data for ns/openshift-network-console...
2026-07-14T12:03:03.001346967Z Wrote inspect data to /must-gather/istio.
2026-07-14T12:03:03.016883130Z INFO: Getting status from alertmanager-main-0
2026-07-14T12:03:03.032652047Z Wrote inspect data to must-gather.
2026-07-14T12:03:03.209342002Z Wrote inspect data to /must-gather/istio.
2026-07-14T12:03:03.212563814Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-14T12:03:03.324425802Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-14T12:03:03.327002877Z INFO: Fallback to identify the container image from release info
2026-07-14T12:03:03.343768815Z Gathering data for ns/openshift-console-operator...
2026-07-14T12:03:03.438677204Z Wrote inspect data to /must-gather/istio.
2026-07-14T12:03:03.615833885Z Wrote inspect data to /must-gather/istio.
2026-07-14T12:03:03.618108254Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-14T12:03:03.730381938Z Gathering data for ns/openshift-console...
2026-07-14T12:03:03.776374895Z Wrote inspect data to /must-gather/istio.
2026-07-14T12:03:03.911086823Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d8d5dc4ea635b13544e6f92103089219e732369ae30557b7beee2e8a11bf500d
2026-07-14T12:03:03.973242583Z Wrote inspect data to /must-gather/istio.
2026-07-14T12:03:04.036437547Z daemonset.apps/perf-node-gather-daemonset created
2026-07-14T12:03:04.167369286Z Waiting for performance profile collector pods to become ready: 1
2026-07-14T12:03:04.304244006Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-14T12:03:04.699441554Z Done executing Istio gather script
2026-07-14T12:03:04.840340472Z Gathering data for ns/openshift-dns-operator...
2026-07-14T12:03:05.129775082Z Gathering data for ns/openshift-dns...
2026-07-14T12:03:05.270453348Z Waiting for performance profile collector pods to become ready: 2
2026-07-14T12:03:05.412214771Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-14T12:03:05.771594345Z Gathering data for ns/openshift-image-registry...
2026-07-14T12:03:06.310485724Z Gathering data for ns/openshift-ingress-operator...
2026-07-14T12:03:06.390519404Z Waiting for performance profile collector pods to become ready: 3
2026-07-14T12:03:06.698452330Z Gathering data for ns/openshift-ingress...
2026-07-14T12:03:07.059853634Z Gathering data for ns/openshift-ingress-canary...
2026-07-14T12:03:07.449918991Z Gathering data for ns/openshift-insights...
2026-07-14T12:03:07.508456137Z Waiting for performance profile collector pods to become ready: 4
2026-07-14T12:03:08.613379138Z Waiting for performance profile collector pods to become ready: 5
2026-07-14T12:03:09.382566705Z Gathering data for ns/openshift-config...
2026-07-14T12:03:09.719752859Z Gathering data for ns/openshift-config-managed...
2026-07-14T12:03:09.736117532Z Waiting for performance profile collector pods to become ready: 6
2026-07-14T12:03:10.189083178Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-14T12:03:10.417106528Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-14T12:03:10.528435542Z Gathering data for ns/openshift-kube-apiserver...
2026-07-14T12:03:10.864085234Z Waiting for performance profile collector pods to become ready: 7
2026-07-14T12:03:10.904754042Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-14T12:03:11.181737562Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-14T12:03:11.465632829Z Gathering data for ns/openshift-kube-scheduler...
2026-07-14T12:03:11.728447679Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-14T12:03:11.985305665Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-14T12:03:12.056094410Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-14T12:03:12.233592451Z Collecting performance related data for node ip-10-0-143-41.ec2.internal
2026-07-14T12:03:12.240310027Z Collecting performance related data for node ip-10-0-133-208.ec2.internal
2026-07-14T12:03:12.247624790Z Collecting performance related data for node ip-10-0-138-106.ec2.internal
2026-07-14T12:03:12.433805458Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-14T12:03:12.777374638Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-14T12:03:13.543213800Z Gathering data for ns/openshift-multus...
2026-07-14T12:03:15.423385307Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-14T12:03:15.486318587Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-14T12:03:17.099643762Z Gathering data for ns/openshift-host-network...
2026-07-14T12:03:17.673441781Z Gathering data for ns/openshift-network-diagnostics...
2026-07-14T12:03:18.297976781Z Gathering data for ns/openshift-network-node-identity...
2026-07-14T12:03:18.737752267Z Gathering data for ns/openshift-network-operator...
2026-07-14T12:03:19.160394088Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-14T12:03:19.433824773Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-14T12:03:19.940725165Z Gathering data for ns/openshift-apiserver-operator...
2026-07-14T12:03:20.274236553Z Gathering data for ns/openshift-apiserver...
2026-07-14T12:03:20.430623282Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-14T12:03:20.441917018Z Collecting kubelet logs for node ip-10-0-143-41.ec2.internal
2026-07-14T12:03:20.444102121Z Collecting kubelet logs for node ip-10-0-133-208.ec2.internal
2026-07-14T12:03:20.446411403Z Collecting kubelet logs for node ip-10-0-138-106.ec2.internal
2026-07-14T12:03:20.742188510Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-14T12:03:20.799080234Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-lrzsl namespace
2026-07-14T12:03:20.804898866Z INFO: Node performance data collection complete.
2026-07-14T12:03:21.024514053Z Gathering data for ns/openshift-controller-manager...
2026-07-14T12:03:21.340018133Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-14T12:03:22.089086491Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-14T12:03:22.389548793Z Gathering data for ns/openshift-service-ca-operator...
2026-07-14T12:03:22.715565714Z Gathering data for ns/openshift-service-ca...
2026-07-14T12:03:23.075793276Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-14T12:03:24.027729237Z Wrote inspect data to must-gather.
2026-07-14T12:03:24.027775207Z error: inspection completed with the errors occurred while gathering data:
2026-07-14T12:03:24.027775207Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-14T12:03:24.149622544Z error: the server doesn't have a resource type "clusters"
2026-07-14T12:03:24.624288339Z Caches written to disk