2026-09-18T21:10:47.644447007Z [disk usage checker] Started
2026-09-18T21:10:47.647743042Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-18T21:10:47.855643219Z Gathering data for ns/openshift-cluster-version...
2026-09-18T21:10:47.982412939Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-18T21:10:48.084604002Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-18T21:10:48.178550968Z Gathering data for ns/default...
2026-09-18T21:10:48.492811421Z Gathering data for ns/openshift...
2026-09-18T21:10:48.994348570Z Gathering data for ns/kube-system...
2026-09-18T21:10:49.929871194Z Gathering data for ns/openshift-etcd...
2026-09-18T21:10:50.268572947Z Wrote inspect data to must-gather.
2026-09-18T21:10:50.268605116Z error: inspection completed with the errors occurred while gathering data:
2026-09-18T21:10:50.268605116Z     namespaces "assisted-installer" not found
2026-09-18T21:10:50.458582173Z Waiting on subprocesses to finish execution.
2026-09-18T21:10:50.466192046Z Executing Istio gather script
2026-09-18T21:10:50.469928599Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-18T21:10:50.473229260Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-18T21:10:50.474092650Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-18T21:10:50.486678618Z INFO: Gathering HAProxy config files
2026-09-18T21:10:50.530151451Z INFO: Collecting host service logs for crio
2026-09-18T21:10:50.530432382Z INFO: Collecting host service logs for kubelet
2026-09-18T21:10:50.533756194Z INFO: Collecting host service logs for rpm-ostreed
2026-09-18T21:10:50.534136501Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-18T21:10:50.534515029Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-18T21:10:50.534873383Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-18T21:10:50.535239600Z INFO: Collecting host service logs for NetworkManager
2026-09-18T21:10:50.535597863Z INFO: Collecting host service logs for openvswitch
2026-09-18T21:10:50.535969754Z INFO: Collecting host service logs for ovs-configuration
2026-09-18T21:10:50.536333825Z INFO: Collecting host service logs for ovsdb-server
2026-09-18T21:10:50.537023216Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-18T21:10:50.539052271Z INFO: Waiting for worker host service log collection to complete ...
2026-09-18T21:10:50.541985552Z INFO: Waiting for node performance related collection to complete ...
2026-09-18T21:10:51.103363674Z No resources found
2026-09-18T21:10:51.527587657Z No resources found in openshift-etcd namespace.
2026-09-18T21:10:51.725652557Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-18T21:10:51.796801154Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-18T21:10:51.838369992Z INFO: Collecting Insights Archives from insights-operator-6f9c644dd9-lgdjd
2026-09-18T21:10:51.838419225Z insights-runtime-extractor-b2jts
2026-09-18T21:10:51.838438527Z insights-runtime-extractor-pd9m9
2026-09-18T21:10:51.838457784Z insights-runtime-extractor-vqldl
2026-09-18T21:10:51.949493202Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-18T21:10:52.077517112Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-18T21:10:52.077517112Z INFO: on-disk MachineConfig config collection complete.
2026-09-18T21:10:52.304475096Z INFO: "metallb-operator" not detected. Skipping.
2026-09-18T21:10:52.334614091Z INFO: Worker host service log collection to complete.
2026-09-18T21:10:52.396269956Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-18T21:10:52.416308291Z ERROR: No running kube-apiserver pods found
2026-09-18T21:10:52.595404615Z error: the server doesn't have a resource type "performanceprofile"
2026-09-18T21:10:52.598139782Z Wrote inspect data to must-gather.
2026-09-18T21:10:52.642613723Z Wrote inspect data to must-gather.
2026-09-18T21:10:52.647366962Z No resources found
2026-09-18T21:10:52.665302780Z Wrote inspect data to must-gather.
2026-09-18T21:10:52.670836087Z Wrote inspect data to must-gather.
2026-09-18T21:10:52.672125725Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-18T21:10:52.726269617Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-18T21:10:52.839971902Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-18T21:10:52.860730074Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-18T21:10:52.901887634Z Wrote inspect data to must-gather.
2026-09-18T21:10:52.924941605Z Wrote inspect data to must-gather.
2026-09-18T21:10:52.962417207Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-18T21:10:52.962417207Z See 'oc rsync -h' for help and examples
2026-09-18T21:10:52.993133400Z INFO: 'previous-logs' folder not found on ip-10-0-136-235.ec2.internal, skipping...
2026-09-18T21:10:53.090468307Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-18T21:10:53.129802860Z Gathering data for ns/openshift-monitoring...
2026-09-18T21:10:53.171568324Z Wrote inspect data to must-gather.
2026-09-18T21:10:53.218969221Z tar: Removing leading `/' from member names
2026-09-18T21:10:53.223469965Z INFO: HAProxy config collection complete.
2026-09-18T21:10:53.273411955Z Wrote inspect data to must-gather.
2026-09-18T21:10:53.356487132Z error: the server doesn't have a resource type "machineconfigs"
2026-09-18T21:10:53.374498754Z INFO: 'previous-logs' folder not found on ip-10-0-137-48.ec2.internal, skipping...
2026-09-18T21:10:53.551749219Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-18T21:10:53.563687581Z Wrote inspect data to must-gather.
2026-09-18T21:10:53.662060722Z INFO: 'previous-logs' folder not found on ip-10-0-140-180.ec2.internal, skipping...
2026-09-18T21:10:53.662182557Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-18T21:10:53.662257317Z INFO: Machine Config Daemon termination log collection complete.
2026-09-18T21:10:53.665945322Z INFO: Getting rules from prometheus-k8s-0
2026-09-18T21:10:53.814458010Z Wrote inspect data to must-gather.
2026-09-18T21:10:53.850744152Z Wrote inspect data to must-gather.
2026-09-18T21:10:54.012358860Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-18T21:10:54.076381780Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-18T21:10:54.123001404Z INFO: Getting status/config from prometheus-k8s-0
2026-09-18T21:10:54.181937146Z INFO: INTERCONNECT MODE
2026-09-18T21:10:54.181974000Z INFO: Gathering ovn-kubernetes DBs
2026-09-18T21:10:54.274569140Z Wrote inspect data to must-gather.
2026-09-18T21:10:54.349933129Z INFO: Gathering OVN_Northbound from ovnkube-node-7nqtg...
2026-09-18T21:10:54.350201915Z INFO: Gathering OVN_Northbound from ovnkube-node-bfsh2...
2026-09-18T21:10:54.350466747Z INFO: Gathering OVN_Northbound from ovnkube-node-vgfrj...
2026-09-18T21:10:54.350805525Z INFO: Gathering OVN_Southbound from ovnkube-node-7nqtg...
2026-09-18T21:10:54.351064297Z INFO: Gathering OVN_Southbound from ovnkube-node-bfsh2...
2026-09-18T21:10:54.352952076Z INFO: Gathering OVN_Southbound from ovnkube-node-vgfrj...
2026-09-18T21:10:54.647403091Z Wrote inspect data to must-gather.
2026-09-18T21:10:54.705510919Z tar: Removing leading `/' from member names
2026-09-18T21:10:54.705611549Z tar: Removing leading `/' from member names
2026-09-18T21:10:54.706273230Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-18T21:10:54.710495662Z tar: Removing leading `/' from member names
2026-09-18T21:10:54.725752997Z tar: Removing leading `/' from member names
2026-09-18T21:10:54.742539977Z tar: Removing leading `/' from member names
2026-09-18T21:10:54.746824632Z tar: Removing leading `/' from member names
2026-09-18T21:10:54.926436231Z Wrote inspect data to must-gather.
2026-09-18T21:10:54.984994327Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-18T21:10:55.015690707Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-18T21:10:55.172709919Z Wrote inspect data to must-gather.
2026-09-18T21:10:55.258985784Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-18T21:10:55.269824889Z INFO: Waiting for network log collection to complete ...
2026-09-18T21:10:55.269954693Z INFO: Waiting for ovnk database copies to complete ...
2026-09-18T21:10:55.270452488Z INFO: Copying ovnk databases complete.
2026-09-18T21:10:55.272369230Z 3.9M	must-gather/network_logs/ovnk_database_store
2026-09-18T21:10:55.275042138Z ovnk_database_store/
2026-09-18T21:10:55.275091709Z ovnk_database_store/ovnkube-node-vgfrj_nbdb
2026-09-18T21:10:55.285991799Z ovnk_database_store/ovnkube-node-vgfrj_sbdb
2026-09-18T21:10:55.311018329Z ovnk_database_store/ovnkube-node-7nqtg_nbdb
2026-09-18T21:10:55.322301630Z ovnk_database_store/ovnkube-node-bfsh2_sbdb
2026-09-18T21:10:55.347036158Z ovnk_database_store/ovnkube-node-7nqtg_sbdb
2026-09-18T21:10:55.347388834Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-18T21:10:55.373970559Z ovnk_database_store/ovnkube-node-bfsh2_nbdb
2026-09-18T21:10:55.385933191Z INFO: Network log collection complete.
2026-09-18T21:10:55.415007412Z Wrote inspect data to must-gather.
2026-09-18T21:10:55.547560777Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-18T21:10:55.548941714Z Wrote inspect data to /must-gather/istio.
2026-09-18T21:10:55.648302042Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-18T21:10:55.715567835Z Wrote inspect data to must-gather.
2026-09-18T21:10:55.794250349Z Wrote inspect data to /must-gather/istio.
2026-09-18T21:10:55.796900742Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-18T21:10:55.875120488Z Gathering data for ns/openshift-network-console...
2026-09-18T21:10:56.025938315Z INFO: Getting status from alertmanager-main-0
2026-09-18T21:10:56.033235563Z Wrote inspect data to must-gather.
2026-09-18T21:10:56.034577508Z Wrote inspect data to /must-gather/istio.
2026-09-18T21:10:56.275175516Z Wrote inspect data to /must-gather/istio.
2026-09-18T21:10:56.278436077Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-18T21:10:56.331901683Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-18T21:10:56.334060956Z INFO: Fallback to identify the container image from release info
2026-09-18T21:10:56.410007506Z Gathering data for ns/openshift-console-operator...
2026-09-18T21:10:56.503995430Z Wrote inspect data to /must-gather/istio.
2026-09-18T21:10:56.715428841Z Wrote inspect data to /must-gather/istio.
2026-09-18T21:10:56.718120177Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-18T21:10:56.842563921Z Gathering data for ns/openshift-console...
2026-09-18T21:10:56.936686194Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0720c8b078cc0cebb01b4c5cde21180aee7c2bb1a0d322b21890c0d3707406ea
2026-09-18T21:10:56.957449283Z Wrote inspect data to /must-gather/istio.
2026-09-18T21:10:57.078230508Z daemonset.apps/perf-node-gather-daemonset created
2026-09-18T21:10:57.182593597Z Wrote inspect data to /must-gather/istio.
2026-09-18T21:10:57.185907739Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-09-18T21:10:57.238226989Z Waiting for performance profile collector pods to become ready: 1
2026-09-18T21:10:57.386142851Z Wrote inspect data to /must-gather/istio.
2026-09-18T21:10:57.444585086Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-18T21:10:57.609947130Z Wrote inspect data to /must-gather/istio.
2026-09-18T21:10:57.612203506Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-09-18T21:10:57.698491046Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-18T21:10:57.783282656Z Wrote inspect data to /must-gather/istio.
2026-09-18T21:10:57.999740343Z Wrote inspect data to /must-gather/istio.
2026-09-18T21:10:58.002228604Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-18T21:10:58.043030029Z Gathering data for ns/openshift-dns-operator...
2026-09-18T21:10:58.187533115Z Wrote inspect data to /must-gather/istio.
2026-09-18T21:10:58.395653504Z Wrote inspect data to /must-gather/istio.
2026-09-18T21:10:58.398261717Z Waiting for performance profile collector pods to become ready: 2
2026-09-18T21:10:58.425129847Z Gathering data for ns/openshift-dns...
2026-09-18T21:10:59.074987073Z Done executing Istio gather script
2026-09-18T21:10:59.132212726Z Gathering data for ns/openshift-image-registry...
2026-09-18T21:10:59.516590404Z Waiting for performance profile collector pods to become ready: 3
2026-09-18T21:10:59.679689025Z Gathering data for ns/openshift-ingress-operator...
2026-09-18T21:11:00.064293263Z Gathering data for ns/openshift-ingress...
2026-09-18T21:11:00.411800521Z Gathering data for ns/openshift-ingress-canary...
2026-09-18T21:11:00.641002647Z Waiting for performance profile collector pods to become ready: 4
2026-09-18T21:11:00.822373968Z Gathering data for ns/openshift-insights...
2026-09-18T21:11:01.752199377Z Waiting for performance profile collector pods to become ready: 5
2026-09-18T21:11:02.703477498Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-18T21:11:02.878611693Z Waiting for performance profile collector pods to become ready: 6
2026-09-18T21:11:03.360098848Z Gathering data for ns/kserve...
2026-09-18T21:11:04.007448399Z Waiting for performance profile collector pods to become ready: 7
2026-09-18T21:11:05.126682151Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-18T21:11:05.168957790Z Gathering data for ns/openshift-config...
2026-09-18T21:11:05.427519138Z Collecting performance related data for node ip-10-0-140-180.ec2.internal
2026-09-18T21:11:05.435261606Z Collecting performance related data for node ip-10-0-136-235.ec2.internal
2026-09-18T21:11:05.445139214Z Collecting performance related data for node ip-10-0-137-48.ec2.internal
2026-09-18T21:11:05.513972755Z Gathering data for ns/openshift-config-managed...
2026-09-18T21:11:06.074357253Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-18T21:11:06.609592615Z Gathering data for ns/openshift-kube-apiserver...
2026-09-18T21:11:06.997363175Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-18T21:11:07.344710876Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-18T21:11:07.700123873Z Gathering data for ns/openshift-kube-scheduler...
2026-09-18T21:11:07.708831720Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-18T21:11:08.071695460Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-18T21:11:08.433083490Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-18T21:11:08.863366606Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-18T21:11:09.201315283Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-18T21:11:09.781329760Z Gathering data for ns/openshift-multus...
2026-09-18T21:11:11.422620201Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-18T21:11:12.714877000Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-18T21:11:12.788753268Z Gathering data for ns/openshift-host-network...
2026-09-18T21:11:13.121711488Z Gathering data for ns/openshift-network-diagnostics...
2026-09-18T21:11:13.712696670Z Gathering data for ns/openshift-network-node-identity...
2026-09-18T21:11:13.957828498Z Collecting kubelet logs for node ip-10-0-140-180.ec2.internal
2026-09-18T21:11:13.960104565Z Collecting kubelet logs for node ip-10-0-136-235.ec2.internal
2026-09-18T21:11:13.962327541Z Collecting kubelet logs for node ip-10-0-137-48.ec2.internal
2026-09-18T21:11:14.272094603Z Gathering data for ns/openshift-network-operator...
2026-09-18T21:11:14.366264727Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-wnf42 namespace
2026-09-18T21:11:14.381080265Z INFO: Node performance data collection complete.
2026-09-18T21:11:14.782031739Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-18T21:11:15.070480461Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-18T21:11:15.587925879Z Gathering data for ns/openshift-apiserver-operator...
2026-09-18T21:11:15.915846032Z Gathering data for ns/openshift-apiserver...
2026-09-18T21:11:16.328525080Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-18T21:11:16.610115929Z Gathering data for ns/openshift-controller-manager...
2026-09-18T21:11:16.895460298Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-18T21:11:17.605344499Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-18T21:11:17.719311739Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-18T21:11:17.913010623Z Gathering data for ns/openshift-service-ca-operator...
2026-09-18T21:11:18.250229023Z Gathering data for ns/openshift-service-ca...
2026-09-18T21:11:18.645672726Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-18T21:11:19.600577447Z Wrote inspect data to must-gather.
2026-09-18T21:11:19.600621927Z error: inspection completed with the errors occurred while gathering data:
2026-09-18T21:11:19.600621927Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-18T21:11:19.727792410Z error: the server doesn't have a resource type "clusters"
2026-09-18T21:11:19.866355428Z Caches written to disk