2026-09-15T16:55:57.109276107Z [disk usage checker] Started
2026-09-15T16:55:57.113265790Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-09-15T16:55:57.296720984Z Gathering data for ns/openshift-cluster-version...
2026-09-15T16:55:57.431686753Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-15T16:55:57.512460429Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-15T16:55:57.592607840Z Gathering data for ns/default...
2026-09-15T16:55:57.875129490Z Gathering data for ns/openshift...
2026-09-15T16:55:58.301053649Z Gathering data for ns/kube-system...
2026-09-15T16:55:58.962516212Z Gathering data for ns/openshift-etcd...
2026-09-15T16:55:59.263292562Z Wrote inspect data to must-gather.
2026-09-15T16:55:59.263408089Z error: inspection completed with the errors occurred while gathering data:
2026-09-15T16:55:59.263408089Z     namespaces "assisted-installer" not found
2026-09-15T16:55:59.788105292Z Waiting on subprocesses to finish execution.
2026-09-15T16:55:59.795873106Z INFO: Gathering HAProxy config files
2026-09-15T16:55:59.797679908Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-15T16:55:59.799885365Z Executing Istio gather script
2026-09-15T16:55:59.801126953Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-15T16:55:59.807948401Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-15T16:55:59.814671007Z INFO: Collecting host service logs for crio
2026-09-15T16:55:59.815005746Z INFO: Collecting host service logs for kubelet
2026-09-15T16:55:59.815346750Z INFO: Collecting host service logs for rpm-ostreed
2026-09-15T16:55:59.815703800Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-15T16:55:59.816024058Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-15T16:55:59.816332401Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-15T16:55:59.816670662Z INFO: Collecting host service logs for NetworkManager
2026-09-15T16:55:59.817003814Z INFO: Collecting host service logs for openvswitch
2026-09-15T16:55:59.817769401Z INFO: Collecting host service logs for ovs-configuration
2026-09-15T16:55:59.818096033Z INFO: Collecting host service logs for ovsdb-server
2026-09-15T16:55:59.818445748Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-15T16:55:59.818825236Z INFO: Waiting for worker host service log collection to complete ...
2026-09-15T16:55:59.909048877Z INFO: Waiting for node performance related collection to complete ...
2026-09-15T16:56:00.506303850Z INFO: Collecting Insights Archives from insights-operator-5844b856db-nxlhs
2026-09-15T16:56:00.506354821Z insights-runtime-extractor-27hdl
2026-09-15T16:56:00.506374294Z insights-runtime-extractor-8jm8c
2026-09-15T16:56:00.506441460Z insights-runtime-extractor-nhhcz
2026-09-15T16:56:00.690171312Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-15T16:56:00.690242608Z INFO: on-disk MachineConfig config collection complete.
2026-09-15T16:56:00.851546060Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-15T16:56:00.938064046Z error: the server doesn't have a resource type "performanceprofile"
2026-09-15T16:56:01.083810057Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-15T16:56:01.152102213Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-15T16:56:01.323248960Z No resources found
2026-09-15T16:56:01.400275892Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-15T16:56:01.404827963Z No resources found in openshift-etcd namespace.
2026-09-15T16:56:01.467250787Z INFO: Worker host service log collection to complete.
2026-09-15T16:56:01.479764956Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-15T16:56:01.479764956Z See 'oc rsync -h' for help and examples
2026-09-15T16:56:01.503218415Z INFO: "metallb-operator" not detected. Skipping.
2026-09-15T16:56:01.580968308Z Wrote inspect data to must-gather.
2026-09-15T16:56:01.682754170Z ERROR: No running kube-apiserver pods found
2026-09-15T16:56:01.750892364Z Wrote inspect data to must-gather.
2026-09-15T16:56:01.772279874Z Wrote inspect data to must-gather.
2026-09-15T16:56:01.773339035Z INFO: 'previous-logs' folder not found on ip-10-0-130-221.ec2.internal, skipping...
2026-09-15T16:56:01.804121248Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-15T16:56:01.822344454Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-15T16:56:02.181665884Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-15T16:56:02.192031985Z No resources found
2026-09-15T16:56:02.205411380Z Gathering data for ns/openshift-monitoring...
2026-09-15T16:56:02.310486891Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-09-15T16:56:02.405906598Z Wrote inspect data to must-gather.
2026-09-15T16:56:02.438600260Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-15T16:56:02.558494332Z Wrote inspect data to must-gather.
2026-09-15T16:56:02.584580310Z tar: Removing leading `/' from member names
2026-09-15T16:56:02.590274820Z INFO: HAProxy config collection complete.
2026-09-15T16:56:02.633498763Z Wrote inspect data to must-gather.
2026-09-15T16:56:02.659888950Z error: the server doesn't have a resource type "machineconfigs"
2026-09-15T16:56:02.702501889Z INFO: 'previous-logs' folder not found on ip-10-0-141-197.ec2.internal, skipping...
2026-09-15T16:56:02.739513353Z Wrote inspect data to must-gather.
2026-09-15T16:56:02.906096946Z INFO: Getting rules from prometheus-k8s-0
2026-09-15T16:56:02.916743462Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-15T16:56:02.958156366Z Wrote inspect data to must-gather.
2026-09-15T16:56:03.024463623Z INFO: 'previous-logs' folder not found on ip-10-0-141-35.ec2.internal, skipping...
2026-09-15T16:56:03.024566742Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-15T16:56:03.024614146Z INFO: Machine Config Daemon termination log collection complete.
2026-09-15T16:56:03.151983468Z Wrote inspect data to must-gather.
2026-09-15T16:56:03.219768980Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-15T16:56:03.246187845Z Wrote inspect data to must-gather.
2026-09-15T16:56:03.321960557Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-15T16:56:03.365760621Z INFO: Getting status/config from prometheus-k8s-0
2026-09-15T16:56:03.509229555Z Wrote inspect data to must-gather.
2026-09-15T16:56:03.560722882Z Wrote inspect data to must-gather.
2026-09-15T16:56:03.806269371Z Wrote inspect data to must-gather.
2026-09-15T16:56:03.821879610Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-15T16:56:03.841740072Z INFO: INTERCONNECT MODE
2026-09-15T16:56:03.841784030Z INFO: Gathering ovn-kubernetes DBs
2026-09-15T16:56:04.011288120Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-15T16:56:04.018706503Z INFO: Gathering OVN_Northbound from ovnkube-node-749vc...
2026-09-15T16:56:04.018975713Z INFO: Gathering OVN_Northbound from ovnkube-node-b4rdn...
2026-09-15T16:56:04.019198564Z INFO: Gathering OVN_Northbound from ovnkube-node-jcbrc...
2026-09-15T16:56:04.019554930Z INFO: Gathering OVN_Southbound from ovnkube-node-749vc...
2026-09-15T16:56:04.019788538Z INFO: Gathering OVN_Southbound from ovnkube-node-b4rdn...
2026-09-15T16:56:04.019963445Z INFO: Gathering OVN_Southbound from ovnkube-node-jcbrc...
2026-09-15T16:56:04.065559386Z Wrote inspect data to must-gather.
2026-09-15T16:56:04.113680780Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-15T16:56:04.310447316Z tar: Removing leading `/' from member names
2026-09-15T16:56:04.315943086Z tar: Removing leading `/' from member names
2026-09-15T16:56:04.369838216Z tar: Removing leading `/' from member names
2026-09-15T16:56:04.382483827Z tar: Removing leading `/' from member names
2026-09-15T16:56:04.383480721Z tar: Removing leading `/' from member names
2026-09-15T16:56:04.402808322Z tar: Removing leading `/' from member names
2026-09-15T16:56:04.497651687Z Wrote inspect data to must-gather.
2026-09-15T16:56:04.526896206Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-15T16:56:04.768434486Z W0915 16:56:04.768365    1405 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-15T16:56:04.773167656Z Wrote inspect data to must-gather.
2026-09-15T16:56:04.775861861Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-15T16:56:04.810790476Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-15T16:56:04.856065782Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-15T16:56:04.934928006Z INFO: Waiting for network log collection to complete ...
2026-09-15T16:56:04.934970034Z INFO: Waiting for ovnk database copies to complete ...
2026-09-15T16:56:04.935308746Z INFO: Copying ovnk databases complete.
2026-09-15T16:56:04.937225348Z 3.7M	must-gather/network_logs/ovnk_database_store
2026-09-15T16:56:04.942814845Z ovnk_database_store/
2026-09-15T16:56:04.942814845Z ovnk_database_store/ovnkube-node-749vc_nbdb
2026-09-15T16:56:04.951221286Z ovnk_database_store/ovnkube-node-749vc_sbdb
2026-09-15T16:56:04.971044661Z ovnk_database_store/ovnkube-node-b4rdn_sbdb
2026-09-15T16:56:04.993694600Z ovnk_database_store/ovnkube-node-jcbrc_sbdb
2026-09-15T16:56:05.019464791Z Wrote inspect data to /must-gather/istio.
2026-09-15T16:56:05.020602219Z ovnk_database_store/ovnkube-node-b4rdn_nbdb
2026-09-15T16:56:05.028283698Z ovnk_database_store/ovnkube-node-jcbrc_nbdb
2026-09-15T16:56:05.040679434Z Gathering data for ns/openshift-network-console...
2026-09-15T16:56:05.041126953Z INFO: Network log collection complete.
2026-09-15T16:56:05.128882262Z Wrote inspect data to must-gather.
2026-09-15T16:56:05.255204787Z Wrote inspect data to /must-gather/istio.
2026-09-15T16:56:05.257958909Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-15T16:56:05.305132770Z INFO: Getting status from alertmanager-main-0
2026-09-15T16:56:05.484087541Z Wrote inspect data to must-gather.
2026-09-15T16:56:05.495571155Z Wrote inspect data to /must-gather/istio.
2026-09-15T16:56:05.541105647Z Gathering data for ns/openshift-console-operator...
2026-09-15T16:56:05.711726651Z Wrote inspect data to /must-gather/istio.
2026-09-15T16:56:05.715642433Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-15T16:56:05.770796726Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-15T16:56:05.773333039Z INFO: Fallback to identify the container image from release info
2026-09-15T16:56:05.922086800Z Wrote inspect data to /must-gather/istio.
2026-09-15T16:56:05.952463927Z Gathering data for ns/openshift-console...
2026-09-15T16:56:06.122258566Z Wrote inspect data to /must-gather/istio.
2026-09-15T16:56:06.124546406Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-15T16:56:06.342076873Z Wrote inspect data to /must-gather/istio.
2026-09-15T16:56:06.375414363Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a
2026-09-15T16:56:06.502435759Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-15T16:56:06.519330475Z daemonset.apps/perf-node-gather-daemonset created
2026-09-15T16:56:06.580313048Z Wrote inspect data to /must-gather/istio.
2026-09-15T16:56:06.583603048Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-09-15T16:56:06.679876608Z Waiting for performance profile collector pods to become ready: 1
2026-09-15T16:56:06.766917435Z Wrote inspect data to /must-gather/istio.
2026-09-15T16:56:06.988416321Z Wrote inspect data to /must-gather/istio.
2026-09-15T16:56:06.991133545Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-09-15T16:56:07.077240462Z Gathering data for ns/openshift-dns-operator...
2026-09-15T16:56:07.176508021Z Wrote inspect data to /must-gather/istio.
2026-09-15T16:56:07.317924765Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-09-15T16:56:07.375198994Z Gathering data for ns/openshift-dns...
2026-09-15T16:56:07.387255121Z Wrote inspect data to /must-gather/istio.
2026-09-15T16:56:07.389885593Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-15T16:56:07.535891481Z Wrote inspect data to /must-gather/istio.
2026-09-15T16:56:07.725105913Z Wrote inspect data to /must-gather/istio.
2026-09-15T16:56:07.817340401Z Waiting for performance profile collector pods to become ready: 2
2026-09-15T16:56:08.010247250Z Gathering data for ns/openshift-image-registry...
2026-09-15T16:56:08.439095893Z Done executing Istio gather script
2026-09-15T16:56:08.482276306Z Gathering data for ns/openshift-ingress-operator...
2026-09-15T16:56:08.796535841Z Gathering data for ns/openshift-ingress...
2026-09-15T16:56:08.947135494Z Waiting for performance profile collector pods to become ready: 3
2026-09-15T16:56:09.119668617Z Gathering data for ns/openshift-ingress-canary...
2026-09-15T16:56:09.493014949Z Gathering data for ns/openshift-insights...
2026-09-15T16:56:10.076352692Z Waiting for performance profile collector pods to become ready: 4
2026-09-15T16:56:11.188415361Z Waiting for performance profile collector pods to become ready: 5
2026-09-15T16:56:11.825942505Z Gathering data for ns/kserve...
2026-09-15T16:56:12.312370044Z Waiting for performance profile collector pods to become ready: 6
2026-09-15T16:56:12.322526006Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-09-15T16:56:13.425107101Z Gathering data for ns/openshift-config...
2026-09-15T16:56:13.427576998Z Waiting for performance profile collector pods to become ready: 7
2026-09-15T16:56:13.712467345Z Gathering data for ns/openshift-config-managed...
2026-09-15T16:56:14.142944419Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-15T16:56:14.393421423Z Gathering data for ns/openshift-kube-apiserver...
2026-09-15T16:56:14.552452537Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-15T16:56:14.682370304Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-15T16:56:14.827705711Z Collecting performance related data for node ip-10-0-141-35.ec2.internal
2026-09-15T16:56:14.836138831Z Collecting performance related data for node ip-10-0-141-197.ec2.internal
2026-09-15T16:56:14.846700584Z Collecting performance related data for node ip-10-0-130-221.ec2.internal
2026-09-15T16:56:15.014567970Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-15T16:56:15.380037164Z Gathering data for ns/openshift-kube-scheduler...
2026-09-15T16:56:15.735494098Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-15T16:56:16.177215169Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-15T16:56:16.553237630Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-15T16:56:16.843511687Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-15T16:56:17.329986617Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-09-15T16:56:17.393341733Z Gathering data for ns/openshift-multus...
2026-09-15T16:56:18.893084319Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-15T16:56:20.270459075Z Gathering data for ns/openshift-host-network...
2026-09-15T16:56:20.615506088Z Gathering data for ns/openshift-network-diagnostics...
2026-09-15T16:56:21.110328820Z Gathering data for ns/openshift-network-node-identity...
2026-09-15T16:56:21.535669570Z Gathering data for ns/openshift-network-operator...
2026-09-15T16:56:21.889600670Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-15T16:56:22.149368835Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-15T16:56:22.335091723Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-09-15T16:56:22.618218871Z Gathering data for ns/openshift-apiserver-operator...
2026-09-15T16:56:22.856073042Z Collecting kubelet logs for node ip-10-0-141-35.ec2.internal
2026-09-15T16:56:22.858605694Z Collecting kubelet logs for node ip-10-0-141-197.ec2.internal
2026-09-15T16:56:22.860414894Z Collecting kubelet logs for node ip-10-0-130-221.ec2.internal
2026-09-15T16:56:22.971506708Z Gathering data for ns/openshift-apiserver...
2026-09-15T16:56:23.281109415Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-bwj65 namespace
2026-09-15T16:56:23.286175953Z INFO: Node performance data collection complete.
2026-09-15T16:56:23.384956361Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-15T16:56:23.648154468Z Gathering data for ns/openshift-controller-manager...
2026-09-15T16:56:23.908325704Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-15T16:56:24.524550200Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-15T16:56:24.785723097Z Gathering data for ns/openshift-service-ca-operator...
2026-09-15T16:56:25.077820920Z Gathering data for ns/openshift-service-ca...
2026-09-15T16:56:25.419256852Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-15T16:56:26.274591045Z Wrote inspect data to must-gather.
2026-09-15T16:56:26.274635806Z error: inspection completed with the errors occurred while gathering data:
2026-09-15T16:56:26.274635806Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-15T16:56:26.403306668Z error: the server doesn't have a resource type "clusters"
2026-09-15T16:56:26.550211605Z Caches written to disk