2026-09-16T08:56:37.452243757Z [disk usage checker] Started
2026-09-16T08:56:37.455457918Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-16T08:56:37.640987008Z Gathering data for ns/openshift-cluster-version...
2026-09-16T08:56:37.747274551Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-16T08:56:37.826001848Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T08:56:37.905983360Z Gathering data for ns/default...
2026-09-16T08:56:38.168394815Z Gathering data for ns/openshift...
2026-09-16T08:56:38.532758270Z Gathering data for ns/kube-system...
2026-09-16T08:56:39.217245813Z Gathering data for ns/openshift-etcd...
2026-09-16T08:56:39.495527485Z Wrote inspect data to must-gather.
2026-09-16T08:56:39.495558152Z error: inspection completed with the errors occurred while gathering data:
2026-09-16T08:56:39.495558152Z     namespaces "assisted-installer" not found
2026-09-16T08:56:39.879257579Z Waiting on subprocesses to finish execution.
2026-09-16T08:56:39.886722863Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-16T08:56:39.889312904Z INFO: Gathering HAProxy config files
2026-09-16T08:56:39.894386770Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-16T08:56:39.897200626Z INFO: Collecting host service logs for crio
2026-09-16T08:56:39.897524668Z INFO: Collecting host service logs for kubelet
2026-09-16T08:56:39.897846617Z INFO: Collecting host service logs for rpm-ostreed
2026-09-16T08:56:39.898174257Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-16T08:56:39.898474766Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-16T08:56:39.898559136Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-16T08:56:39.898779721Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-16T08:56:39.899142058Z INFO: Collecting host service logs for NetworkManager
2026-09-16T08:56:39.899472656Z INFO: Collecting host service logs for openvswitch
2026-09-16T08:56:39.899793549Z INFO: Collecting host service logs for ovs-configuration
2026-09-16T08:56:39.900163355Z INFO: Collecting host service logs for ovsdb-server
2026-09-16T08:56:39.900489211Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-16T08:56:39.900855016Z INFO: Waiting for worker host service log collection to complete ...
2026-09-16T08:56:39.921050248Z INFO: Waiting for node performance related collection to complete ...
2026-09-16T08:56:39.931929156Z Executing Istio gather script
2026-09-16T08:56:40.820737747Z No resources found
2026-09-16T08:56:40.849692570Z No resources found in openshift-etcd namespace.
2026-09-16T08:56:40.911265499Z error: the server doesn't have a resource type "performanceprofile"
2026-09-16T08:56:40.954731083Z INFO: Collecting Insights Archives from insights-operator-5844b856db-fcskd
2026-09-16T08:56:40.954793962Z insights-runtime-extractor-d4kt7
2026-09-16T08:56:40.954815578Z insights-runtime-extractor-dxp79
2026-09-16T08:56:40.954836004Z insights-runtime-extractor-l9ngt
2026-09-16T08:56:40.957145575Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-16T08:56:41.053507350Z Wrote inspect data to must-gather.
2026-09-16T08:56:41.121849778Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-16T08:56:41.122833525Z INFO: Worker host service log collection to complete.
2026-09-16T08:56:41.180161084Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-16T08:56:41.180244920Z INFO: on-disk MachineConfig config collection complete.
2026-09-16T08:56:41.310968740Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-16T08:56:41.325832155Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-16T08:56:41.325832155Z See 'oc rsync -h' for help and examples
2026-09-16T08:56:41.332440587Z INFO: "metallb-operator" not detected. Skipping.
2026-09-16T08:56:41.519334426Z Wrote inspect data to must-gather.
2026-09-16T08:56:41.650204624Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-16T08:56:41.693560577Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-16T08:56:41.745817774Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T08:56:41.758603478Z ERROR: No running kube-apiserver pods found
2026-09-16T08:56:41.865190122Z No resources found
2026-09-16T08:56:41.911979143Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-16T08:56:42.035548875Z Wrote inspect data to must-gather.
2026-09-16T08:56:42.103260473Z Wrote inspect data to must-gather.
2026-09-16T08:56:42.168843998Z Wrote inspect data to must-gather.
2026-09-16T08:56:42.175657236Z INFO: 'previous-logs' folder not found on ip-10-0-130-237.ec2.internal, skipping...
2026-09-16T08:56:42.250770277Z Wrote inspect data to must-gather.
2026-09-16T08:56:42.267253845Z tar: Removing leading `/' from member names
2026-09-16T08:56:42.272092824Z Gathering data for ns/openshift-monitoring...
2026-09-16T08:56:42.272533304Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-16T08:56:42.274940827Z INFO: HAProxy config collection complete.
2026-09-16T08:56:42.295827762Z Wrote inspect data to must-gather.
2026-09-16T08:56:42.410848631Z error: the server doesn't have a resource type "machineconfigs"
2026-09-16T08:56:42.465573991Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-16T08:56:42.503789867Z INFO: 'previous-logs' folder not found on ip-10-0-136-123.ec2.internal, skipping...
2026-09-16T08:56:42.591490198Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-16T08:56:42.609841781Z Wrote inspect data to must-gather.
2026-09-16T08:56:42.719331905Z INFO: Getting rules from prometheus-k8s-0
2026-09-16T08:56:42.722031863Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-16T08:56:42.799328147Z INFO: 'previous-logs' folder not found on ip-10-0-143-47.ec2.internal, skipping...
2026-09-16T08:56:42.799441928Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-16T08:56:42.799500341Z INFO: Machine Config Daemon termination log collection complete.
2026-09-16T08:56:42.834895035Z Wrote inspect data to must-gather.
2026-09-16T08:56:42.890127374Z Wrote inspect data to must-gather.
2026-09-16T08:56:42.980569164Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-16T08:56:43.115773365Z INFO: Getting status/config from prometheus-k8s-0
2026-09-16T08:56:43.133374699Z Wrote inspect data to must-gather.
2026-09-16T08:56:43.191937863Z Wrote inspect data to must-gather.
2026-09-16T08:56:43.394377698Z INFO: INTERCONNECT MODE
2026-09-16T08:56:43.394377698Z INFO: Gathering ovn-kubernetes DBs
2026-09-16T08:56:43.394413596Z Wrote inspect data to must-gather.
2026-09-16T08:56:43.519594010Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-16T08:56:43.575226475Z INFO: Gathering OVN_Northbound from ovnkube-node-4pkd9...
2026-09-16T08:56:43.575572590Z INFO: Gathering OVN_Northbound from ovnkube-node-4rb62...
2026-09-16T08:56:43.575870397Z INFO: Gathering OVN_Northbound from ovnkube-node-zmbf7...
2026-09-16T08:56:43.576197708Z INFO: Gathering OVN_Southbound from ovnkube-node-4pkd9...
2026-09-16T08:56:43.576474825Z INFO: Gathering OVN_Southbound from ovnkube-node-4rb62...
2026-09-16T08:56:43.576738980Z INFO: Gathering OVN_Southbound from ovnkube-node-zmbf7...
2026-09-16T08:56:43.595379445Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-16T08:56:43.641565831Z Wrote inspect data to must-gather.
2026-09-16T08:56:43.703241992Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-16T08:56:43.848556092Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-16T08:56:43.860090599Z tar: Removing leading `/' from member names
2026-09-16T08:56:43.893009148Z tar: Removing leading `/' from member names
2026-09-16T08:56:43.908280503Z tar: Removing leading `/' from member names
2026-09-16T08:56:43.920703368Z tar: Removing leading `/' from member names
2026-09-16T08:56:43.923150280Z tar: Removing leading `/' from member names
2026-09-16T08:56:43.931509632Z tar: Removing leading `/' from member names
2026-09-16T08:56:43.978266613Z Wrote inspect data to /must-gather/istio.
2026-09-16T08:56:44.009259737Z Wrote inspect data to must-gather.
2026-09-16T08:56:44.133546359Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-16T08:56:44.235312540Z Wrote inspect data to /must-gather/istio.
2026-09-16T08:56:44.238608253Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-16T08:56:44.269408410Z Wrote inspect data to must-gather.
2026-09-16T08:56:44.399599299Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-16T08:56:44.424570529Z INFO: Waiting for network log collection to complete ...
2026-09-16T08:56:44.424688579Z INFO: Waiting for ovnk database copies to complete ...
2026-09-16T08:56:44.425219250Z INFO: Copying ovnk databases complete.
2026-09-16T08:56:44.427596127Z 2.9M	must-gather/network_logs/ovnk_database_store
2026-09-16T08:56:44.430935323Z ovnk_database_store/
2026-09-16T08:56:44.430935323Z ovnk_database_store/ovnkube-node-zmbf7_sbdb
2026-09-16T08:56:44.457123233Z ovnk_database_store/ovnkube-node-4pkd9_sbdb
2026-09-16T08:56:44.459341258Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T08:56:44.476880361Z ovnk_database_store/ovnkube-node-4pkd9_nbdb
2026-09-16T08:56:44.483513437Z Wrote inspect data to /must-gather/istio.
2026-09-16T08:56:44.483850328Z ovnk_database_store/ovnkube-node-4rb62_sbdb
2026-09-16T08:56:44.502617344Z ovnk_database_store/ovnkube-node-4rb62_nbdb
2026-09-16T08:56:44.510974935Z ovnk_database_store/ovnkube-node-zmbf7_nbdb
2026-09-16T08:56:44.523643887Z INFO: Network log collection complete.
2026-09-16T08:56:44.590473717Z Wrote inspect data to must-gather.
2026-09-16T08:56:44.645509523Z Gathering data for ns/openshift-network-console...
2026-09-16T08:56:44.702631887Z Wrote inspect data to /must-gather/istio.
2026-09-16T08:56:44.707384132Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-16T08:56:44.803476623Z INFO: Getting status from alertmanager-main-0
2026-09-16T08:56:44.901379730Z Wrote inspect data to must-gather.
2026-09-16T08:56:44.910258048Z Wrote inspect data to /must-gather/istio.
2026-09-16T08:56:45.124858086Z Gathering data for ns/openshift-console-operator...
2026-09-16T08:56:45.134266928Z Wrote inspect data to /must-gather/istio.
2026-09-16T08:56:45.137517585Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-16T08:56:45.177814370Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-16T08:56:45.180121690Z INFO: Fallback to identify the container image from release info
2026-09-16T08:56:45.351579594Z Wrote inspect data to /must-gather/istio.
2026-09-16T08:56:45.507695389Z Gathering data for ns/openshift-console...
2026-09-16T08:56:45.538252823Z Wrote inspect data to /must-gather/istio.
2026-09-16T08:56:45.540862456Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-16T08:56:45.691324123Z Wrote inspect data to /must-gather/istio.
2026-09-16T08:56:45.860905565Z Wrote inspect data to /must-gather/istio.
2026-09-16T08:56:46.041091004Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-16T08:56:46.078161290Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a
2026-09-16T08:56:46.200552470Z daemonset.apps/perf-node-gather-daemonset created
2026-09-16T08:56:46.327843722Z Waiting for performance profile collector pods to become ready: 1
2026-09-16T08:56:46.532619727Z Done executing Istio gather script
2026-09-16T08:56:46.572339308Z Gathering data for ns/openshift-dns-operator...
2026-09-16T08:56:46.868501547Z Gathering data for ns/openshift-dns...
2026-09-16T08:56:47.438951721Z Gathering data for ns/openshift-image-registry...
2026-09-16T08:56:47.442062145Z Waiting for performance profile collector pods to become ready: 2
2026-09-16T08:56:47.470642864Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-16T08:56:47.864223649Z Gathering data for ns/openshift-ingress-operator...
2026-09-16T08:56:48.163458670Z Gathering data for ns/openshift-ingress...
2026-09-16T08:56:48.451376659Z Gathering data for ns/openshift-ingress-canary...
2026-09-16T08:56:48.542789373Z Waiting for performance profile collector pods to become ready: 3
2026-09-16T08:56:48.812421392Z Gathering data for ns/openshift-insights...
2026-09-16T08:56:49.649657238Z Waiting for performance profile collector pods to become ready: 4
2026-09-16T08:56:50.559957817Z Gathering data for ns/openshift-jobset-operator...
2026-09-16T08:56:50.754921642Z Waiting for performance profile collector pods to become ready: 5
2026-09-16T08:56:51.327941887Z Gathering data for ns/openshift-config...
2026-09-16T08:56:51.611517148Z Gathering data for ns/openshift-config-managed...
2026-09-16T08:56:51.857748503Z Waiting for performance profile collector pods to become ready: 6
2026-09-16T08:56:52.004454860Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-16T08:56:52.233280118Z Gathering data for ns/openshift-kube-apiserver...
2026-09-16T08:56:52.470311535Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-16T08:56:52.475232206Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-16T08:56:52.729149637Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-16T08:56:52.979440555Z Waiting for performance profile collector pods to become ready: 7
2026-09-16T08:56:52.996277659Z Gathering data for ns/openshift-kube-scheduler...
2026-09-16T08:56:53.277089642Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-16T08:56:53.533992757Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-16T08:56:53.820273874Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-16T08:56:54.102017309Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-16T08:56:54.138213863Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-16T08:56:54.352227457Z Collecting performance related data for node ip-10-0-136-123.ec2.internal
2026-09-16T08:56:54.360416226Z Collecting performance related data for node ip-10-0-130-237.ec2.internal
2026-09-16T08:56:54.369090194Z Collecting performance related data for node ip-10-0-143-47.ec2.internal
2026-09-16T08:56:54.663396851Z Gathering data for ns/openshift-multus...
2026-09-16T08:56:56.053702416Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-16T08:56:57.242085608Z Gathering data for ns/openshift-host-network...
2026-09-16T08:56:57.480422815Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-16T08:56:57.530746305Z Gathering data for ns/openshift-network-diagnostics...
2026-09-16T08:56:58.031713157Z Gathering data for ns/openshift-network-node-identity...
2026-09-16T08:56:58.405096496Z Gathering data for ns/openshift-network-operator...
2026-09-16T08:56:58.768902245Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-16T08:56:59.029432871Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-16T08:56:59.563942959Z Gathering data for ns/openshift-apiserver-operator...
2026-09-16T08:56:59.901842508Z Gathering data for ns/openshift-apiserver...
2026-09-16T08:57:00.350712683Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-16T08:57:00.582831609Z Gathering data for ns/openshift-controller-manager...
2026-09-16T08:57:00.830175062Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-16T08:57:01.443377824Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-16T08:57:01.708958117Z Gathering data for ns/openshift-service-ca-operator...
2026-09-16T08:57:02.004029048Z Gathering data for ns/openshift-service-ca...
2026-09-16T08:57:02.143503126Z Collecting kubelet logs for node ip-10-0-136-123.ec2.internal
2026-09-16T08:57:02.145279881Z Collecting kubelet logs for node ip-10-0-130-237.ec2.internal
2026-09-16T08:57:02.147354078Z Collecting kubelet logs for node ip-10-0-143-47.ec2.internal
2026-09-16T08:57:02.428124347Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-16T08:57:02.485277793Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-16T08:57:02.493890961Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-5h62s namespace
2026-09-16T08:57:02.498788280Z INFO: Node performance data collection complete.
2026-09-16T08:57:03.311405545Z Wrote inspect data to must-gather.
2026-09-16T08:57:03.311446729Z error: inspection completed with the errors occurred while gathering data:
2026-09-16T08:57:03.311446729Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-16T08:57:03.432444011Z error: the server doesn't have a resource type "clusters"
2026-09-16T08:57:03.563309212Z Caches written to disk