2026-09-17T18:33:45.310028127Z [disk usage checker] Started
2026-09-17T18:33:45.313431009Z [disk usage checker] Volume usage percentage: current = 35 ; allowed = 70
2026-09-17T18:33:45.601316758Z Gathering data for ns/openshift-cluster-version...
2026-09-17T18:33:45.750168103Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-17T18:33:45.847822777Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T18:33:45.930367833Z Gathering data for ns/default...
2026-09-17T18:33:46.273108430Z Gathering data for ns/openshift...
2026-09-17T18:33:46.694189177Z Gathering data for ns/kube-system...
2026-09-17T18:33:47.458666333Z Gathering data for ns/openshift-etcd...
2026-09-17T18:33:47.821033780Z Wrote inspect data to must-gather.
2026-09-17T18:33:47.821069482Z error: inspection completed with the errors occurred while gathering data:
2026-09-17T18:33:47.821069482Z     namespaces "assisted-installer" not found
2026-09-17T18:33:48.189448791Z Waiting on subprocesses to finish execution.
2026-09-17T18:33:48.197264575Z Executing Istio gather script
2026-09-17T18:33:48.198215197Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-17T18:33:48.201199114Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-17T18:33:48.204449419Z INFO: Gathering HAProxy config files
2026-09-17T18:33:48.206291611Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-17T18:33:48.272772473Z INFO: Collecting host service logs for crio
2026-09-17T18:33:48.273109166Z INFO: Collecting host service logs for kubelet
2026-09-17T18:33:48.273402208Z INFO: Collecting host service logs for rpm-ostreed
2026-09-17T18:33:48.273679378Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-17T18:33:48.274051866Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-17T18:33:48.274343706Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-17T18:33:48.274631621Z INFO: Collecting host service logs for NetworkManager
2026-09-17T18:33:48.274905789Z INFO: Collecting host service logs for openvswitch
2026-09-17T18:33:48.275200329Z INFO: Collecting host service logs for ovs-configuration
2026-09-17T18:33:48.275481756Z INFO: Collecting host service logs for ovsdb-server
2026-09-17T18:33:48.275740808Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-17T18:33:48.276692692Z INFO: Waiting for worker host service log collection to complete ...
2026-09-17T18:33:48.326294633Z INFO: Waiting for node performance related collection to complete ...
2026-09-17T18:33:48.669718226Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-17T18:33:48.792098033Z INFO: Collecting Insights Archives from insights-operator-6f9c644dd9-4pk97
2026-09-17T18:33:48.792150987Z insights-runtime-extractor-pvmjd
2026-09-17T18:33:48.792170861Z insights-runtime-extractor-pxzlk
2026-09-17T18:33:48.792189640Z insights-runtime-extractor-qbxbs
2026-09-17T18:33:48.815703770Z INFO: "metallb-operator" not detected. Skipping.
2026-09-17T18:33:48.990305698Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-17T18:33:48.990340843Z INFO: on-disk MachineConfig config collection complete.
2026-09-17T18:33:48.992330681Z Wrote inspect data to must-gather.
2026-09-17T18:33:49.003549252Z No resources found in openshift-etcd namespace.
2026-09-17T18:33:49.532176978Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-17T18:33:49.582313348Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-17T18:33:49.794320979Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-17T18:33:49.794320979Z See 'oc rsync -h' for help and examples
2026-09-17T18:33:49.818888687Z No resources found
2026-09-17T18:33:50.100124396Z ERROR: No running kube-apiserver pods found
2026-09-17T18:33:50.168329920Z INFO: Worker host service log collection to complete.
2026-09-17T18:33:50.172658273Z error: the server doesn't have a resource type "performanceprofile"
2026-09-17T18:33:50.230041306Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-17T18:33:50.239525499Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-17T18:33:50.348083942Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-17T18:33:50.354240804Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T18:33:50.383328082Z Gathering data for ns/openshift-monitoring...
2026-09-17T18:33:50.427301283Z [disk usage checker] Volume usage percentage: current = 35 ; allowed = 70
2026-09-17T18:33:50.590546875Z Wrote inspect data to must-gather.
2026-09-17T18:33:50.642440549Z No resources found
2026-09-17T18:33:50.754392263Z INFO: 'previous-logs' folder not found on ip-10-0-132-143.ec2.internal, skipping...
2026-09-17T18:33:50.757944857Z Wrote inspect data to must-gather.
2026-09-17T18:33:50.771672392Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-17T18:33:50.871631304Z Wrote inspect data to must-gather.
2026-09-17T18:33:50.913160395Z Wrote inspect data to must-gather.
2026-09-17T18:33:50.985155716Z Wrote inspect data to must-gather.
2026-09-17T18:33:51.012026501Z Wrote inspect data to must-gather.
2026-09-17T18:33:51.145731961Z tar: Removing leading `/' from member names
2026-09-17T18:33:51.148385174Z error: the server doesn't have a resource type "machineconfigs"
2026-09-17T18:33:51.155839892Z INFO: HAProxy config collection complete.
2026-09-17T18:33:51.161592933Z INFO: 'previous-logs' folder not found on ip-10-0-134-17.ec2.internal, skipping...
2026-09-17T18:33:51.206963182Z INFO: Getting rules from prometheus-k8s-0
2026-09-17T18:33:51.315479865Z Wrote inspect data to must-gather.
2026-09-17T18:33:51.374775105Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-17T18:33:51.466106892Z INFO: 'previous-logs' folder not found on ip-10-0-135-202.ec2.internal, skipping...
2026-09-17T18:33:51.466210124Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-17T18:33:51.466259986Z INFO: Machine Config Daemon termination log collection complete.
2026-09-17T18:33:51.612164581Z Wrote inspect data to must-gather.
2026-09-17T18:33:51.613656181Z Wrote inspect data to must-gather.
2026-09-17T18:33:51.710623873Z INFO: Getting status/config from prometheus-k8s-0
2026-09-17T18:33:51.790818785Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-17T18:33:51.899648860Z Wrote inspect data to must-gather.
2026-09-17T18:33:51.957443390Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-17T18:33:52.025378649Z Wrote inspect data to must-gather.
2026-09-17T18:33:52.210318091Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-17T18:33:52.228067436Z INFO: INTERCONNECT MODE
2026-09-17T18:33:52.228098961Z INFO: Gathering ovn-kubernetes DBs
2026-09-17T18:33:52.298466904Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-17T18:33:52.299698371Z Wrote inspect data to must-gather.
2026-09-17T18:33:52.423573520Z INFO: Gathering OVN_Northbound from ovnkube-node-jmbb8...
2026-09-17T18:33:52.423782640Z INFO: Gathering OVN_Northbound from ovnkube-node-l5r9g...
2026-09-17T18:33:52.424008971Z INFO: Gathering OVN_Northbound from ovnkube-node-qkrtg...
2026-09-17T18:33:52.424293940Z INFO: Gathering OVN_Southbound from ovnkube-node-jmbb8...
2026-09-17T18:33:52.424579886Z INFO: Gathering OVN_Southbound from ovnkube-node-l5r9g...
2026-09-17T18:33:52.424869298Z INFO: Gathering OVN_Southbound from ovnkube-node-qkrtg...
2026-09-17T18:33:52.534995775Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-17T18:33:52.592183120Z Wrote inspect data to must-gather.
2026-09-17T18:33:52.750530466Z tar: Removing leading `/' from member names
2026-09-17T18:33:52.751931481Z tar: Removing leading `/' from member names
2026-09-17T18:33:52.771323735Z tar: Removing leading `/' from member names
2026-09-17T18:33:52.799708097Z tar: Removing leading `/' from member names
2026-09-17T18:33:52.803575408Z tar: Removing leading `/' from member names
2026-09-17T18:33:52.859084972Z tar: Removing leading `/' from member names
2026-09-17T18:33:52.937648886Z Wrote inspect data to must-gather.
2026-09-17T18:33:52.939828166Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-17T18:33:53.006379210Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-17T18:33:53.187543882Z W0917 18:33:53.187499    1409 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-17T18:33:53.195697986Z Wrote inspect data to must-gather.
2026-09-17T18:33:53.206748242Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T18:33:53.242343607Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:33:53.242984179Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-17T18:33:53.377898537Z INFO: Waiting for network log collection to complete ...
2026-09-17T18:33:53.378049044Z INFO: Waiting for ovnk database copies to complete ...
2026-09-17T18:33:53.378569726Z INFO: Copying ovnk databases complete.
2026-09-17T18:33:53.380285165Z 4.8M	must-gather/network_logs/ovnk_database_store
2026-09-17T18:33:53.382620182Z ovnk_database_store/
2026-09-17T18:33:53.382620182Z ovnk_database_store/ovnkube-node-l5r9g_nbdb
2026-09-17T18:33:53.395574948Z ovnk_database_store/ovnkube-node-jmbb8_nbdb
2026-09-17T18:33:53.409177144Z ovnk_database_store/ovnkube-node-l5r9g_sbdb
2026-09-17T18:33:53.427728482Z Gathering data for ns/openshift-network-console...
2026-09-17T18:33:53.437043097Z ovnk_database_store/ovnkube-node-jmbb8_sbdb
2026-09-17T18:33:53.468000798Z ovnk_database_store/ovnkube-node-qkrtg_sbdb
2026-09-17T18:33:53.499882638Z ovnk_database_store/ovnkube-node-qkrtg_nbdb
2026-09-17T18:33:53.515537787Z INFO: Network log collection complete.
2026-09-17T18:33:53.532515882Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:33:53.536175346Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-17T18:33:53.554396930Z Wrote inspect data to must-gather.
2026-09-17T18:33:53.693156673Z INFO: Getting status from alertmanager-main-0
2026-09-17T18:33:53.783299081Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:33:53.889407668Z Wrote inspect data to must-gather.
2026-09-17T18:33:53.943882702Z Gathering data for ns/openshift-console-operator...
2026-09-17T18:33:54.019813954Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:33:54.023971180Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-17T18:33:54.196344797Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-17T18:33:54.200831936Z INFO: Fallback to identify the container image from release info
2026-09-17T18:33:54.239223235Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:33:54.368376907Z Gathering data for ns/openshift-console...
2026-09-17T18:33:54.435161618Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:33:54.438324973Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-17T18:33:54.655351999Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:33:54.758574680Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0720c8b078cc0cebb01b4c5cde21180aee7c2bb1a0d322b21890c0d3707406ea
2026-09-17T18:33:54.879111802Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:33:54.884255031Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-09-17T18:33:54.916736469Z daemonset.apps/perf-node-gather-daemonset created
2026-09-17T18:33:54.937626809Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-17T18:33:55.076040303Z Waiting for performance profile collector pods to become ready: 1
2026-09-17T18:33:55.085838386Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:33:55.271268534Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:33:55.278481081Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-09-17T18:33:55.440637104Z [disk usage checker] Volume usage percentage: current = 35 ; allowed = 70
2026-09-17T18:33:55.477271785Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:33:55.553393255Z Gathering data for ns/openshift-dns-operator...
2026-09-17T18:33:55.686529098Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:33:55.689735334Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-17T18:33:55.859172233Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:33:55.918057924Z Gathering data for ns/openshift-dns...
2026-09-17T18:33:56.043555938Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:33:56.222272589Z Waiting for performance profile collector pods to become ready: 2
2026-09-17T18:33:56.536848375Z Gathering data for ns/openshift-image-registry...
2026-09-17T18:33:56.764954363Z Done executing Istio gather script
2026-09-17T18:33:57.016290914Z Gathering data for ns/openshift-ingress-operator...
2026-09-17T18:33:57.349016450Z Gathering data for ns/openshift-ingress...
2026-09-17T18:33:57.363883609Z Waiting for performance profile collector pods to become ready: 3
2026-09-17T18:33:57.682852408Z Gathering data for ns/openshift-ingress-canary...
2026-09-17T18:33:58.097241674Z Gathering data for ns/openshift-insights...
2026-09-17T18:33:58.488082668Z Waiting for performance profile collector pods to become ready: 4
2026-09-17T18:33:59.619901098Z Waiting for performance profile collector pods to become ready: 5
2026-09-17T18:34:00.445351791Z [disk usage checker] Volume usage percentage: current = 35 ; allowed = 70
2026-09-17T18:34:00.557963813Z Gathering data for ns/kserve...
2026-09-17T18:34:00.749016713Z Waiting for performance profile collector pods to become ready: 6
2026-09-17T18:34:01.881258602Z Waiting for performance profile collector pods to become ready: 7
2026-09-17T18:34:02.916332663Z Gathering data for ns/openshift-config...
2026-09-17T18:34:03.019986758Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-17T18:34:03.197609209Z Gathering data for ns/openshift-config-managed...
2026-09-17T18:34:03.303426004Z Collecting performance related data for node ip-10-0-135-202.ec2.internal
2026-09-17T18:34:03.310058177Z Collecting performance related data for node ip-10-0-132-143.ec2.internal
2026-09-17T18:34:03.316758662Z Collecting performance related data for node ip-10-0-134-17.ec2.internal
2026-09-17T18:34:03.653669395Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-17T18:34:04.000305218Z Gathering data for ns/openshift-kube-apiserver...
2026-09-17T18:34:04.359513432Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-17T18:34:04.748748863Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-17T18:34:05.086154288Z Gathering data for ns/openshift-kube-scheduler...
2026-09-17T18:34:05.347076728Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-17T18:34:05.449794272Z [disk usage checker] Volume usage percentage: current = 35 ; allowed = 70
2026-09-17T18:34:05.674003078Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-17T18:34:06.055678049Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-17T18:34:06.405235496Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-17T18:34:06.954218789Z Gathering data for ns/openshift-multus...
2026-09-17T18:34:08.663273868Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-17T18:34:10.167747586Z Gathering data for ns/openshift-host-network...
2026-09-17T18:34:10.454366461Z [disk usage checker] Volume usage percentage: current = 35 ; allowed = 70
2026-09-17T18:34:10.491814645Z Gathering data for ns/openshift-network-diagnostics...
2026-09-17T18:34:11.114195363Z Gathering data for ns/openshift-network-node-identity...
2026-09-17T18:34:11.569758313Z Collecting kubelet logs for node ip-10-0-135-202.ec2.internal
2026-09-17T18:34:11.572028047Z Collecting kubelet logs for node ip-10-0-132-143.ec2.internal
2026-09-17T18:34:11.573875468Z Collecting kubelet logs for node ip-10-0-134-17.ec2.internal
2026-09-17T18:34:11.647108478Z Gathering data for ns/openshift-network-operator...
2026-09-17T18:34:12.002407759Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-m4nv9 namespace
2026-09-17T18:34:12.008731291Z INFO: Node performance data collection complete.
2026-09-17T18:34:12.085382255Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-17T18:34:12.452580875Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-17T18:34:12.996378985Z Gathering data for ns/openshift-apiserver-operator...
2026-09-17T18:34:13.377190748Z Gathering data for ns/openshift-apiserver...
2026-09-17T18:34:13.815567029Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-17T18:34:14.155519617Z Gathering data for ns/openshift-controller-manager...
2026-09-17T18:34:14.479470157Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-17T18:34:15.303644822Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-17T18:34:15.458944037Z [disk usage checker] Volume usage percentage: current = 35 ; allowed = 70
2026-09-17T18:34:15.667499719Z Gathering data for ns/openshift-service-ca-operator...
2026-09-17T18:34:16.033969033Z Gathering data for ns/openshift-service-ca...
2026-09-17T18:34:16.430045081Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-17T18:34:17.287734858Z Wrote inspect data to must-gather.
2026-09-17T18:34:17.287780371Z error: inspection completed with the errors occurred while gathering data:
2026-09-17T18:34:17.287780371Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-17T18:34:17.416265415Z error: the server doesn't have a resource type "clusters"
2026-09-17T18:34:17.596604842Z Caches written to disk