2026-09-17T18:31:19.521520665Z [disk usage checker] Started
2026-09-17T18:31:19.524743897Z [disk usage checker] Volume usage percentage: current = 37 ; allowed = 70
2026-09-17T18:31:19.802867567Z Gathering data for ns/openshift-cluster-version...
2026-09-17T18:31:19.947358023Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-17T18:31:20.049626208Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T18:31:20.134306242Z Gathering data for ns/default...
2026-09-17T18:31:20.466218769Z Gathering data for ns/openshift...
2026-09-17T18:31:20.884696112Z Gathering data for ns/kube-system...
2026-09-17T18:31:21.665230326Z Gathering data for ns/openshift-etcd...
2026-09-17T18:31:21.990108228Z Wrote inspect data to must-gather.
2026-09-17T18:31:21.990152673Z error: inspection completed with the errors occurred while gathering data:
2026-09-17T18:31:21.990152673Z     namespaces "assisted-installer" not found
2026-09-17T18:31:22.429212543Z Waiting on subprocesses to finish execution.
2026-09-17T18:31:22.432454247Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-17T18:31:22.434969628Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-17T18:31:22.445965101Z INFO: Gathering HAProxy config files
2026-09-17T18:31:22.447618986Z Executing Istio gather script
2026-09-17T18:31:22.451447810Z INFO: Collecting host service logs for crio
2026-09-17T18:31:22.451824035Z INFO: Collecting host service logs for kubelet
2026-09-17T18:31:22.452162967Z INFO: Collecting host service logs for rpm-ostreed
2026-09-17T18:31:22.452514499Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-17T18:31:22.452855536Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-17T18:31:22.453198685Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-17T18:31:22.453608153Z INFO: Collecting host service logs for NetworkManager
2026-09-17T18:31:22.455850428Z INFO: Collecting host service logs for openvswitch
2026-09-17T18:31:22.456227504Z INFO: Collecting host service logs for ovs-configuration
2026-09-17T18:31:22.456601247Z INFO: Collecting host service logs for ovsdb-server
2026-09-17T18:31:22.456956303Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-17T18:31:22.457390810Z INFO: Waiting for worker host service log collection to complete ...
2026-09-17T18:31:22.481532244Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-17T18:31:22.505386077Z INFO: Waiting for node performance related collection to complete ...
2026-09-17T18:31:23.741367772Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-17T18:31:23.907238740Z Wrote inspect data to must-gather.
2026-09-17T18:31:23.913529343Z Wrote inspect data to must-gather.
2026-09-17T18:31:23.940861218Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-17T18:31:23.994976265Z No resources found
2026-09-17T18:31:24.007167101Z error: the server doesn't have a resource type "performanceprofile"
2026-09-17T18:31:24.132377690Z INFO: Collecting Insights Archives from insights-operator-6f9c644dd9-dtl88
2026-09-17T18:31:24.132430077Z insights-runtime-extractor-54hqg
2026-09-17T18:31:24.132451191Z insights-runtime-extractor-hjpz9
2026-09-17T18:31:24.133836781Z INFO: "metallb-operator" not detected. Skipping.
2026-09-17T18:31:24.134159926Z insights-runtime-extractor-l5lqq
2026-09-17T18:31:24.139290539Z INFO: Worker host service log collection to complete.
2026-09-17T18:31:24.140555798Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-17T18:31:24.140555798Z INFO: on-disk MachineConfig config collection complete.
2026-09-17T18:31:24.183540502Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-17T18:31:24.186178513Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-17T18:31:24.186609657Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-17T18:31:24.284420945Z No resources found in openshift-etcd namespace.
2026-09-17T18:31:24.366537384Z ERROR: No running kube-apiserver pods found
2026-09-17T18:31:24.528570802Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-17T18:31:24.528570802Z See 'oc rsync -h' for help and examples
2026-09-17T18:31:24.571276055Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-17T18:31:24.602708242Z [disk usage checker] Volume usage percentage: current = 37 ; allowed = 70
2026-09-17T18:31:24.750212037Z INFO: 'previous-logs' folder not found on ip-10-0-133-139.ec2.internal, skipping...
2026-09-17T18:31:24.759619466Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-17T18:31:24.940721122Z Gathering data for ns/openshift-monitoring...
2026-09-17T18:31:25.092816624Z Wrote inspect data to must-gather.
2026-09-17T18:31:25.167424780Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T18:31:25.216867902Z No resources found
2026-09-17T18:31:25.269554384Z Wrote inspect data to must-gather.
2026-09-17T18:31:25.339843602Z Wrote inspect data to must-gather.
2026-09-17T18:31:25.374219159Z tar: Removing leading `/' from member names
2026-09-17T18:31:25.383894732Z INFO: HAProxy config collection complete.
2026-09-17T18:31:25.418194091Z Wrote inspect data to must-gather.
2026-09-17T18:31:25.457011174Z Wrote inspect data to must-gather.
2026-09-17T18:31:25.482314053Z error: the server doesn't have a resource type "machineconfigs"
2026-09-17T18:31:25.527254827Z INFO: 'previous-logs' folder not found on ip-10-0-133-71.ec2.internal, skipping...
2026-09-17T18:31:25.696785733Z Wrote inspect data to must-gather.
2026-09-17T18:31:25.707688530Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-17T18:31:25.713309060Z INFO: Getting rules from prometheus-k8s-0
2026-09-17T18:31:25.826400776Z INFO: 'previous-logs' folder not found on ip-10-0-138-116.ec2.internal, skipping...
2026-09-17T18:31:25.826515352Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-17T18:31:25.826576366Z INFO: Machine Config Daemon termination log collection complete.
2026-09-17T18:31:25.976527005Z Wrote inspect data to must-gather.
2026-09-17T18:31:25.979969047Z Wrote inspect data to must-gather.
2026-09-17T18:31:26.192943423Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-17T18:31:26.202212534Z INFO: Getting status/config from prometheus-k8s-0
2026-09-17T18:31:26.278634004Z Wrote inspect data to must-gather.
2026-09-17T18:31:26.296928203Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-17T18:31:26.429446161Z Wrote inspect data to must-gather.
2026-09-17T18:31:26.613117645Z INFO: INTERCONNECT MODE
2026-09-17T18:31:26.613220591Z INFO: Gathering ovn-kubernetes DBs
2026-09-17T18:31:26.682563365Z Wrote inspect data to must-gather.
2026-09-17T18:31:26.754303969Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-17T18:31:26.773581738Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-17T18:31:26.808318067Z INFO: Gathering OVN_Northbound from ovnkube-node-5l8m4...
2026-09-17T18:31:26.808559210Z INFO: Gathering OVN_Northbound from ovnkube-node-8hxld...
2026-09-17T18:31:26.808812091Z INFO: Gathering OVN_Northbound from ovnkube-node-rrf79...
2026-09-17T18:31:26.809071814Z INFO: Gathering OVN_Southbound from ovnkube-node-5l8m4...
2026-09-17T18:31:26.809389448Z INFO: Gathering OVN_Southbound from ovnkube-node-8hxld...
2026-09-17T18:31:26.809667809Z INFO: Gathering OVN_Southbound from ovnkube-node-rrf79...
2026-09-17T18:31:27.035672452Z Wrote inspect data to must-gather.
2026-09-17T18:31:27.168630057Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-17T18:31:27.189355965Z tar: Removing leading `/' from member names
2026-09-17T18:31:27.203576625Z tar: Removing leading `/' from member names
2026-09-17T18:31:27.210068561Z tar: Removing leading `/' from member names
2026-09-17T18:31:27.225002030Z tar: Removing leading `/' from member names
2026-09-17T18:31:27.230624217Z tar: Removing leading `/' from member names
2026-09-17T18:31:27.256733437Z tar: Removing leading `/' from member names
2026-09-17T18:31:27.365999253Z Wrote inspect data to must-gather.
2026-09-17T18:31:27.505358875Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-17T18:31:27.508083282Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-17T18:31:27.627476147Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T18:31:27.728436897Z Wrote inspect data to must-gather.
2026-09-17T18:31:27.744175490Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:31:27.824807630Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-17T18:31:27.835276206Z INFO: Waiting for network log collection to complete ...
2026-09-17T18:31:27.835534820Z INFO: Waiting for ovnk database copies to complete ...
2026-09-17T18:31:27.835839433Z INFO: Copying ovnk databases complete.
2026-09-17T18:31:27.837685887Z 4.8M	must-gather/network_logs/ovnk_database_store
2026-09-17T18:31:27.840229767Z ovnk_database_store/
2026-09-17T18:31:27.840300700Z ovnk_database_store/ovnkube-node-5l8m4_nbdb
2026-09-17T18:31:27.855323718Z ovnk_database_store/ovnkube-node-8hxld_sbdb
2026-09-17T18:31:27.871253671Z Gathering data for ns/openshift-network-console...
2026-09-17T18:31:27.883684013Z ovnk_database_store/ovnkube-node-5l8m4_sbdb
2026-09-17T18:31:27.914737477Z ovnk_database_store/ovnkube-node-8hxld_nbdb
2026-09-17T18:31:27.925097372Z ovnk_database_store/ovnkube-node-rrf79_sbdb
2026-09-17T18:31:27.958318814Z ovnk_database_store/ovnkube-node-rrf79_nbdb
2026-09-17T18:31:27.975258935Z INFO: Network log collection complete.
2026-09-17T18:31:28.009602660Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:31:28.014044848Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-17T18:31:28.094452248Z Wrote inspect data to must-gather.
2026-09-17T18:31:28.238071369Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:31:28.330374561Z INFO: Getting status from alertmanager-main-0
2026-09-17T18:31:28.445858778Z Wrote inspect data to must-gather.
2026-09-17T18:31:28.450843114Z Gathering data for ns/openshift-console-operator...
2026-09-17T18:31:28.498927871Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:31:28.504647537Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-17T18:31:28.732632799Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:31:28.747100588Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-17T18:31:28.751278727Z INFO: Fallback to identify the container image from release info
2026-09-17T18:31:28.933385715Z Gathering data for ns/openshift-console...
2026-09-17T18:31:28.954034817Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:31:28.957027826Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-17T18:31:29.176712399Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:31:29.338282060Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0720c8b078cc0cebb01b4c5cde21180aee7c2bb1a0d322b21890c0d3707406ea
2026-09-17T18:31:29.426462443Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:31:29.431163333Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-09-17T18:31:29.506083192Z daemonset.apps/perf-node-gather-daemonset created
2026-09-17T18:31:29.607549825Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-17T18:31:29.640179651Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:31:29.640529231Z [disk usage checker] Volume usage percentage: current = 37 ; allowed = 70
2026-09-17T18:31:29.673577544Z Waiting for performance profile collector pods to become ready: 1
2026-09-17T18:31:29.844123185Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:31:29.848696270Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-09-17T18:31:30.045196502Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:31:30.207278527Z Gathering data for ns/openshift-dns-operator...
2026-09-17T18:31:30.269235251Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:31:30.273924162Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-17T18:31:30.433954289Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:31:30.556271990Z Gathering data for ns/openshift-dns...
2026-09-17T18:31:30.627204254Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:31:30.797103831Z Waiting for performance profile collector pods to become ready: 2
2026-09-17T18:31:31.247943187Z Gathering data for ns/openshift-image-registry...
2026-09-17T18:31:31.324243102Z Done executing Istio gather script
2026-09-17T18:31:31.767630620Z Gathering data for ns/openshift-ingress-operator...
2026-09-17T18:31:31.921876578Z Waiting for performance profile collector pods to become ready: 3
2026-09-17T18:31:32.125573742Z Gathering data for ns/openshift-ingress...
2026-09-17T18:31:32.473163887Z Gathering data for ns/openshift-ingress-canary...
2026-09-17T18:31:32.871911094Z Gathering data for ns/openshift-insights...
2026-09-17T18:31:33.035992615Z Waiting for performance profile collector pods to become ready: 4
2026-09-17T18:31:34.178080811Z Waiting for performance profile collector pods to become ready: 5
2026-09-17T18:31:34.645127142Z [disk usage checker] Volume usage percentage: current = 37 ; allowed = 70
2026-09-17T18:31:35.300200079Z Waiting for performance profile collector pods to become ready: 6
2026-09-17T18:31:35.513423523Z Gathering data for ns/kserve...
2026-09-17T18:31:36.425650177Z Waiting for performance profile collector pods to become ready: 7
2026-09-17T18:31:37.545025800Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-17T18:31:37.807125609Z Gathering data for ns/openshift-config...
2026-09-17T18:31:37.813538561Z Collecting performance related data for node ip-10-0-133-139.ec2.internal
2026-09-17T18:31:37.822693123Z Collecting performance related data for node ip-10-0-133-71.ec2.internal
2026-09-17T18:31:37.831692498Z Collecting performance related data for node ip-10-0-138-116.ec2.internal
2026-09-17T18:31:38.165771839Z Gathering data for ns/openshift-config-managed...
2026-09-17T18:31:38.775497024Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-17T18:31:39.148563938Z Gathering data for ns/openshift-kube-apiserver...
2026-09-17T18:31:39.564547755Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-17T18:31:39.649800521Z [disk usage checker] Volume usage percentage: current = 37 ; allowed = 70
2026-09-17T18:31:39.885590322Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-17T18:31:40.238277714Z Gathering data for ns/openshift-kube-scheduler...
2026-09-17T18:31:40.602668533Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-17T18:31:41.020652538Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-17T18:31:41.401577359Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-17T18:31:41.743935424Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-17T18:31:42.281218801Z Gathering data for ns/openshift-multus...
2026-09-17T18:31:43.938828078Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-17T18:31:44.654550225Z [disk usage checker] Volume usage percentage: current = 37 ; allowed = 70
2026-09-17T18:31:45.315536970Z Gathering data for ns/openshift-host-network...
2026-09-17T18:31:45.672175859Z Gathering data for ns/openshift-network-diagnostics...
2026-09-17T18:31:46.236556665Z Gathering data for ns/openshift-network-node-identity...
2026-09-17T18:31:46.528320748Z Collecting kubelet logs for node ip-10-0-133-139.ec2.internal
2026-09-17T18:31:46.530588571Z Collecting kubelet logs for node ip-10-0-133-71.ec2.internal
2026-09-17T18:31:46.533378828Z Collecting kubelet logs for node ip-10-0-138-116.ec2.internal
2026-09-17T18:31:46.694725436Z Gathering data for ns/openshift-network-operator...
2026-09-17T18:31:46.971228466Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-5s492 namespace
2026-09-17T18:31:46.978248898Z INFO: Node performance data collection complete.
2026-09-17T18:31:47.137934830Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-17T18:31:47.471832678Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-17T18:31:47.994053999Z Gathering data for ns/openshift-apiserver-operator...
2026-09-17T18:31:48.319402108Z Gathering data for ns/openshift-apiserver...
2026-09-17T18:31:48.759518184Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-17T18:31:49.040433078Z Gathering data for ns/openshift-controller-manager...
2026-09-17T18:31:49.339410922Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-17T18:31:49.659893856Z [disk usage checker] Volume usage percentage: current = 37 ; allowed = 70
2026-09-17T18:31:50.009803670Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-17T18:31:50.315491575Z Gathering data for ns/openshift-service-ca-operator...
2026-09-17T18:31:50.665296379Z Gathering data for ns/openshift-service-ca...
2026-09-17T18:31:51.077713056Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-17T18:31:52.065291133Z Wrote inspect data to must-gather.
2026-09-17T18:31:52.065352961Z error: inspection completed with the errors occurred while gathering data:
2026-09-17T18:31:52.065352961Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-17T18:31:52.194455584Z error: the server doesn't have a resource type "clusters"
2026-09-17T18:31:52.353033795Z Caches written to disk