2026-09-17T18:00:25.404288044Z [disk usage checker] Started
2026-09-17T18:00:25.407489782Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-09-17T18:00:25.612402511Z Gathering data for ns/openshift-cluster-version...
2026-09-17T18:00:25.755677358Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-17T18:00:25.857728735Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T18:00:25.945713014Z Gathering data for ns/default...
2026-09-17T18:00:26.314833883Z Gathering data for ns/openshift...
2026-09-17T18:00:26.740506458Z Gathering data for ns/kube-system...
2026-09-17T18:00:27.524908728Z Gathering data for ns/openshift-etcd...
2026-09-17T18:00:27.888075570Z Wrote inspect data to must-gather.
2026-09-17T18:00:27.888117517Z error: inspection completed with the errors occurred while gathering data:
2026-09-17T18:00:27.888117517Z     namespaces "assisted-installer" not found
2026-09-17T18:00:28.121777509Z Waiting on subprocesses to finish execution.
2026-09-17T18:00:28.126642755Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-17T18:00:28.132152659Z Executing Istio gather script
2026-09-17T18:00:28.132471901Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-17T18:00:28.133240606Z INFO: Gathering HAProxy config files
2026-09-17T18:00:28.154186071Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-17T18:00:28.169673053Z INFO: Collecting host service logs for crio
2026-09-17T18:00:28.170037500Z INFO: Collecting host service logs for kubelet
2026-09-17T18:00:28.170377796Z INFO: Collecting host service logs for rpm-ostreed
2026-09-17T18:00:28.170679011Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-17T18:00:28.170978722Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-17T18:00:28.171290703Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-17T18:00:28.171582329Z INFO: Collecting host service logs for NetworkManager
2026-09-17T18:00:28.171862535Z INFO: Collecting host service logs for openvswitch
2026-09-17T18:00:28.172171259Z INFO: Collecting host service logs for ovs-configuration
2026-09-17T18:00:28.172473086Z INFO: Collecting host service logs for ovsdb-server
2026-09-17T18:00:28.172769102Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-17T18:00:28.173129756Z INFO: Waiting for worker host service log collection to complete ...
2026-09-17T18:00:28.213321717Z INFO: Waiting for node performance related collection to complete ...
2026-09-17T18:00:28.793139740Z INFO: Collecting Insights Archives from insights-operator-6f9c644dd9-d8vvv
2026-09-17T18:00:28.793189895Z insights-runtime-extractor-bldj9
2026-09-17T18:00:28.793209970Z insights-runtime-extractor-kpnbq
2026-09-17T18:00:28.793228931Z insights-runtime-extractor-r6jkw
2026-09-17T18:00:28.940712140Z No resources found in openshift-etcd namespace.
2026-09-17T18:00:28.992603276Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-17T18:00:29.013299011Z error: the server doesn't have a resource type "performanceprofile"
2026-09-17T18:00:29.150017577Z INFO: "metallb-operator" not detected. Skipping.
2026-09-17T18:00:29.232138691Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-17T18:00:29.232138691Z See 'oc rsync -h' for help and examples
2026-09-17T18:00:29.332128189Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-17T18:00:29.332217727Z INFO: on-disk MachineConfig config collection complete.
2026-09-17T18:00:29.392129443Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-17T18:00:29.711565409Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-17T18:00:29.788284746Z ERROR: No running kube-apiserver pods found
2026-09-17T18:00:29.788589467Z No resources found
2026-09-17T18:00:29.801840911Z Wrote inspect data to must-gather.
2026-09-17T18:00:29.906225722Z No resources found
2026-09-17T18:00:29.928776083Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T18:00:29.966847204Z INFO: Worker host service log collection to complete.
2026-09-17T18:00:29.978080990Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-17T18:00:30.086318478Z Wrote inspect data to must-gather.
2026-09-17T18:00:30.257838282Z Wrote inspect data to must-gather.
2026-09-17T18:00:30.364942590Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-17T18:00:30.409363564Z INFO: 'previous-logs' folder not found on ip-10-0-132-90.ec2.internal, skipping...
2026-09-17T18:00:30.484753986Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-17T18:00:30.490023791Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-09-17T18:00:30.490679632Z Wrote inspect data to must-gather.
2026-09-17T18:00:30.524075301Z Wrote inspect data to must-gather.
2026-09-17T18:00:30.541293545Z Wrote inspect data to must-gather.
2026-09-17T18:00:30.557630412Z Wrote inspect data to must-gather.
2026-09-17T18:00:30.724500056Z Gathering data for ns/openshift-monitoring...
2026-09-17T18:00:30.732628218Z error: the server doesn't have a resource type "machineconfigs"
2026-09-17T18:00:30.777839106Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-17T18:00:30.927639009Z INFO: 'previous-logs' folder not found on ip-10-0-134-52.ec2.internal, skipping...
2026-09-17T18:00:30.941060882Z Wrote inspect data to must-gather.
2026-09-17T18:00:30.979676217Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-17T18:00:31.066884870Z tar: Removing leading `/' from member names
2026-09-17T18:00:31.074998869Z INFO: HAProxy config collection complete.
2026-09-17T18:00:31.076342251Z INFO: Getting rules from prometheus-k8s-0
2026-09-17T18:00:31.230175402Z INFO: 'previous-logs' folder not found on ip-10-0-134-96.ec2.internal, skipping...
2026-09-17T18:00:31.230279976Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-17T18:00:31.230365434Z INFO: Machine Config Daemon termination log collection complete.
2026-09-17T18:00:31.260782051Z Wrote inspect data to must-gather.
2026-09-17T18:00:31.270938525Z Wrote inspect data to must-gather.
2026-09-17T18:00:31.428155439Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-17T18:00:31.542550020Z Wrote inspect data to must-gather.
2026-09-17T18:00:31.574297454Z INFO: Getting status/config from prometheus-k8s-0
2026-09-17T18:00:31.666282812Z Wrote inspect data to must-gather.
2026-09-17T18:00:31.889025923Z INFO: INTERCONNECT MODE
2026-09-17T18:00:31.889025923Z INFO: Gathering ovn-kubernetes DBs
2026-09-17T18:00:31.932294715Z Wrote inspect data to must-gather.
2026-09-17T18:00:32.009935924Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-17T18:00:32.014234908Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-17T18:00:32.085726763Z INFO: Gathering OVN_Northbound from ovnkube-node-45dp8...
2026-09-17T18:00:32.085937367Z INFO: Gathering OVN_Northbound from ovnkube-node-ncbx8...
2026-09-17T18:00:32.086169003Z INFO: Gathering OVN_Northbound from ovnkube-node-zbx6h...
2026-09-17T18:00:32.086455461Z INFO: Gathering OVN_Southbound from ovnkube-node-45dp8...
2026-09-17T18:00:32.086704019Z INFO: Gathering OVN_Southbound from ovnkube-node-ncbx8...
2026-09-17T18:00:32.087010272Z INFO: Gathering OVN_Southbound from ovnkube-node-zbx6h...
2026-09-17T18:00:32.206330256Z Wrote inspect data to must-gather.
2026-09-17T18:00:32.325601582Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-17T18:00:32.409794348Z tar: Removing leading `/' from member names
2026-09-17T18:00:32.428683541Z tar: Removing leading `/' from member names
2026-09-17T18:00:32.435310787Z tar: Removing leading `/' from member names
2026-09-17T18:00:32.440281341Z tar: Removing leading `/' from member names
2026-09-17T18:00:32.440386849Z tar: Removing leading `/' from member names
2026-09-17T18:00:32.441585113Z tar: Removing leading `/' from member names
2026-09-17T18:00:32.559545549Z Wrote inspect data to must-gather.
2026-09-17T18:00:32.626039746Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-17T18:00:32.660088662Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-17T18:00:32.774460882Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-17T18:00:32.811421422Z Wrote inspect data to must-gather.
2026-09-17T18:00:32.949721385Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-17T18:00:32.974394530Z INFO: Waiting for network log collection to complete ...
2026-09-17T18:00:32.974505843Z INFO: Waiting for ovnk database copies to complete ...
2026-09-17T18:00:32.975009327Z INFO: Copying ovnk databases complete.
2026-09-17T18:00:32.978905802Z 3.7M	must-gather/network_logs/ovnk_database_store
2026-09-17T18:00:32.980980985Z ovnk_database_store/
2026-09-17T18:00:32.980980985Z ovnk_database_store/ovnkube-node-45dp8_sbdb
2026-09-17T18:00:33.005563171Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:00:33.018258932Z ovnk_database_store/ovnkube-node-zbx6h_sbdb
2026-09-17T18:00:33.040466553Z ovnk_database_store/ovnkube-node-ncbx8_nbdb
2026-09-17T18:00:33.050672417Z ovnk_database_store/ovnkube-node-ncbx8_sbdb
2026-09-17T18:00:33.060760378Z Wrote inspect data to must-gather.
2026-09-17T18:00:33.073720978Z ovnk_database_store/ovnkube-node-45dp8_nbdb
2026-09-17T18:00:33.082216738Z ovnk_database_store/ovnkube-node-zbx6h_nbdb
2026-09-17T18:00:33.096246669Z INFO: Network log collection complete.
2026-09-17T18:00:33.239316157Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:00:33.241296966Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-17T18:00:33.272560135Z W0917 18:00:33.272510    1501 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-17T18:00:33.278263309Z Wrote inspect data to must-gather.
2026-09-17T18:00:33.316692893Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T18:00:33.359481166Z INFO: Getting status from alertmanager-main-0
2026-09-17T18:00:33.454044828Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:00:33.533624635Z Gathering data for ns/openshift-network-console...
2026-09-17T18:00:33.578022133Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-17T18:00:33.580369795Z INFO: Fallback to identify the container image from release info
2026-09-17T18:00:33.678679533Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:00:33.681536870Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-17T18:00:33.883989963Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:00:34.047394593Z Gathering data for ns/openshift-console-operator...
2026-09-17T18:00:34.082935482Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:00:34.085231313Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-17T18:00:34.313742706Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:00:34.397198350Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0720c8b078cc0cebb01b4c5cde21180aee7c2bb1a0d322b21890c0d3707406ea
2026-09-17T18:00:34.476478340Z Gathering data for ns/openshift-console...
2026-09-17T18:00:34.528606981Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:00:34.530874286Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-09-17T18:00:34.540444971Z daemonset.apps/perf-node-gather-daemonset created
2026-09-17T18:00:34.687510385Z Waiting for performance profile collector pods to become ready: 1
2026-09-17T18:00:34.716336443Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:00:34.918540854Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:00:34.921273884Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-09-17T18:00:35.095114119Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-17T18:00:35.118219459Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:00:35.320829331Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:00:35.323996290Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-17T18:00:35.497090205Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:00:35.500188582Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-09-17T18:00:35.667244092Z Gathering data for ns/openshift-dns-operator...
2026-09-17T18:00:35.693634717Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:00:35.836521117Z Waiting for performance profile collector pods to become ready: 2
2026-09-17T18:00:36.019747747Z Gathering data for ns/openshift-dns...
2026-09-17T18:00:36.346865431Z Done executing Istio gather script
2026-09-17T18:00:36.789647567Z Gathering data for ns/openshift-image-registry...
2026-09-17T18:00:36.961820174Z Waiting for performance profile collector pods to become ready: 3
2026-09-17T18:00:37.300461420Z Gathering data for ns/openshift-ingress-operator...
2026-09-17T18:00:37.651481957Z Gathering data for ns/openshift-ingress...
2026-09-17T18:00:38.000309068Z Gathering data for ns/openshift-ingress-canary...
2026-09-17T18:00:38.072260445Z Waiting for performance profile collector pods to become ready: 4
2026-09-17T18:00:38.500950725Z Gathering data for ns/openshift-insights...
2026-09-17T18:00:39.193935378Z Waiting for performance profile collector pods to become ready: 5
2026-09-17T18:00:40.313875473Z Waiting for performance profile collector pods to become ready: 6
2026-09-17T18:00:40.504686263Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-09-17T18:00:41.000484390Z Gathering data for ns/kserve...
2026-09-17T18:00:41.431367898Z Waiting for performance profile collector pods to become ready: 7
2026-09-17T18:00:42.557055885Z Waiting for performance profile collector pods to become ready: 8
2026-09-17T18:00:42.808256107Z Gathering data for ns/openshift-config...
2026-09-17T18:00:43.080700200Z Gathering data for ns/openshift-config-managed...
2026-09-17T18:00:43.557261425Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-17T18:00:43.692842195Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-17T18:00:43.935242010Z Gathering data for ns/openshift-kube-apiserver...
2026-09-17T18:00:43.941978859Z Collecting performance related data for node ip-10-0-134-96.ec2.internal
2026-09-17T18:00:43.949638592Z Collecting performance related data for node ip-10-0-134-52.ec2.internal
2026-09-17T18:00:43.960853677Z Collecting performance related data for node ip-10-0-132-90.ec2.internal
2026-09-17T18:00:44.275307274Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-17T18:00:44.674068810Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-17T18:00:45.080356682Z Gathering data for ns/openshift-kube-scheduler...
2026-09-17T18:00:45.463471644Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-17T18:00:45.510558488Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-09-17T18:00:45.809836436Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-17T18:00:46.188580339Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-17T18:00:46.570130280Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-17T18:00:47.285264852Z Gathering data for ns/openshift-multus...
2026-09-17T18:00:49.010280058Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-17T18:00:50.452588189Z Gathering data for ns/openshift-host-network...
2026-09-17T18:00:50.517431890Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-09-17T18:00:50.833175802Z Gathering data for ns/openshift-network-diagnostics...
2026-09-17T18:00:51.442772469Z Gathering data for ns/openshift-network-node-identity...
2026-09-17T18:00:51.922906595Z Gathering data for ns/openshift-network-operator...
2026-09-17T18:00:51.935396938Z Collecting kubelet logs for node ip-10-0-134-96.ec2.internal
2026-09-17T18:00:51.937543375Z Collecting kubelet logs for node ip-10-0-134-52.ec2.internal
2026-09-17T18:00:51.939563396Z Collecting kubelet logs for node ip-10-0-132-90.ec2.internal
2026-09-17T18:00:52.325677203Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-dx4r5 namespace
2026-09-17T18:00:52.331346660Z INFO: Node performance data collection complete.
2026-09-17T18:00:52.393203244Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-17T18:00:52.692843685Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-17T18:00:53.213389084Z Gathering data for ns/openshift-apiserver-operator...
2026-09-17T18:00:53.548077653Z Gathering data for ns/openshift-apiserver...
2026-09-17T18:00:53.950922227Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-17T18:00:54.243063472Z Gathering data for ns/openshift-controller-manager...
2026-09-17T18:00:54.543173071Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-17T18:00:55.265125520Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-17T18:00:55.521931197Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-09-17T18:00:55.615024078Z Gathering data for ns/openshift-service-ca-operator...
2026-09-17T18:00:55.933758360Z Gathering data for ns/openshift-service-ca...
2026-09-17T18:00:56.329650271Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-17T18:00:57.243973823Z Wrote inspect data to must-gather.
2026-09-17T18:00:57.244012184Z error: inspection completed with the errors occurred while gathering data:
2026-09-17T18:00:57.244012184Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-17T18:00:57.370375496Z error: the server doesn't have a resource type "clusters"
2026-09-17T18:00:57.531803419Z Caches written to disk