2026-11-04T09:54:00.502524186Z [disk usage checker] Started
2026-11-04T09:54:00.506240169Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:54:01.399844734Z Gathering data for ns/openshift-cluster-version...
2026-11-04T09:54:02.274919794Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable no sooner than v6.0+
2026-11-04T09:54:02.826630677Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-11-04T09:54:03.319760905Z Gathering data for ns/default...
2026-11-04T09:54:05.199205274Z Gathering data for ns/openshift...
2026-11-04T09:54:05.510780953Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:54:07.120835863Z Gathering data for ns/kube-system...
2026-11-04T09:54:07.219921370Z Waiting on subprocesses to finish execution.
2026-11-04T09:54:07.226506852Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-11-04T09:54:07.229475310Z INFO: Gathering HAProxy config files
2026-11-04T09:54:07.235874641Z Executing Istio gather script
2026-11-04T09:54:07.238456250Z INFO: Gathering machine config daemon's old logs from all nodes
2026-11-04T09:54:07.240721830Z INFO: Collecting host service logs for crio
2026-11-04T09:54:07.240798563Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-11-04T09:54:07.240997791Z INFO: Collecting host service logs for kubelet
2026-11-04T09:54:07.241365730Z INFO: Collecting host service logs for rpm-ostreed
2026-11-04T09:54:07.241722681Z INFO: Collecting host service logs for ostree-finalize-staged
2026-11-04T09:54:07.242040332Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-11-04T09:54:07.242397710Z INFO: Collecting host service logs for machine-config-daemon-host
2026-11-04T09:54:07.242739699Z INFO: Collecting host service logs for NetworkManager
2026-11-04T09:54:07.243074602Z INFO: Collecting host service logs for openvswitch
2026-11-04T09:54:07.243510702Z INFO: Collecting host service logs for ovs-configuration
2026-11-04T09:54:07.243865742Z INFO: Collecting host service logs for ovsdb-server
2026-11-04T09:54:07.244225928Z INFO: Collecting host service logs for ovs-vswitchd
2026-11-04T09:54:07.244629589Z INFO: Waiting for worker host service log collection to complete ...
2026-11-04T09:54:07.711765364Z INFO: Waiting for node performance related collection to complete ...
2026-11-04T09:54:08.192628625Z No resources found in openshift-etcd namespace.
2026-11-04T09:54:08.223915254Z INFO: "metallb-operator" not detected. Skipping.
2026-11-04T09:54:08.263408191Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-11-04T09:54:08.538044317Z Wrote inspect data to must-gather.
2026-11-04T09:54:08.539746950Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-11-04T09:54:08.539828786Z INFO: on-disk MachineConfig config collection complete.
2026-11-04T09:54:08.546380075Z INFO: Found 1 replicas - prometheus-k8s-0
2026-11-04T09:54:08.603160052Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-11-04T09:54:08.636576830Z INFO: "nfd" not detected. Skipping.
2026-11-04T09:54:08.675806939Z No resources found
2026-11-04T09:54:08.736660421Z INFO: Collecting Insights Archives from insights-operator-7c4986d449-tv282
2026-11-04T09:54:08.736660421Z insights-runtime-extractor-dtzr6
2026-11-04T09:54:08.736660421Z insights-runtime-extractor-gk7vs
2026-11-04T09:54:08.736660421Z insights-runtime-extractor-rbjzf
2026-11-04T09:54:08.739077966Z INFO: Worker host service log collection to complete.
2026-11-04T09:54:08.805511046Z INFO: "sriov-network-operator" not detected. Skipping.
2026-11-04T09:54:08.808845260Z error: the server doesn't have a resource type "performanceprofile"
2026-11-04T09:54:08.831529091Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-11-04T09:54:08.831529091Z See 'oc rsync -h' for help and examples
2026-11-04T09:54:08.856208703Z ERROR: No running kube-apiserver pods found
2026-11-04T09:54:09.008614339Z INFO: Waiting for HAProxy config collection to complete ...
2026-11-04T09:54:09.016524882Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-11-04T09:54:09.486328353Z Wrote inspect data to must-gather.
2026-11-04T09:54:09.611790586Z Wrote inspect data to must-gather.
2026-11-04T09:54:09.638712831Z INFO: 'previous-logs' folder not found on ip-10-0-128-178.us-west-1.compute.internal, skipping...
2026-11-04T09:54:09.677825367Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-11-04T09:54:09.873807394Z No resources found
2026-11-04T09:54:09.986999411Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-11-04T09:54:10.021273728Z Wrote inspect data to must-gather.
2026-11-04T09:54:10.060459536Z Wrote inspect data to must-gather.
2026-11-04T09:54:10.106630249Z error: the server doesn't have a resource type "machineconfigs"
2026-11-04T09:54:10.109657474Z INFO: 'previous-logs' folder not found on ip-10-0-139-0.us-west-1.compute.internal, skipping...
2026-11-04T09:54:10.115562519Z Wrote inspect data to must-gather.
2026-11-04T09:54:10.155888520Z Wrote inspect data to must-gather.
2026-11-04T09:54:10.259590929Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-11-04T09:54:10.293901909Z tar: Removing leading `/' from member names
2026-11-04T09:54:10.397429860Z INFO: HAProxy config collection complete.
2026-11-04T09:54:10.491491367Z error: the server doesn't have a resource type "machineconfigpools"
2026-11-04T09:54:10.509327357Z INFO: 'previous-logs' folder not found on ip-10-0-141-14.us-west-1.compute.internal, skipping...
2026-11-04T09:54:10.509376813Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-11-04T09:54:10.509376813Z INFO: Machine Config Daemon termination log collection complete.
2026-11-04T09:54:10.515759245Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:54:10.686638684Z Wrote inspect data to must-gather.
2026-11-04T09:54:10.824150518Z INFO: Getting rules from prometheus-k8s-0
2026-11-04T09:54:10.941456041Z Wrote inspect data to must-gather.
2026-11-04T09:54:11.270490353Z error: the server doesn't have a resource type "kubeletconfigs"
2026-11-04T09:54:11.286105351Z Inspecting resource crd/backendtlspolicies.gateway.networking.k8s.io
2026-11-04T09:54:11.328622030Z Wrote inspect data to must-gather.
2026-11-04T09:54:11.798900718Z Wrote inspect data to must-gather.
2026-11-04T09:54:11.853927783Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:54:11.864203387Z Wrote inspect data to must-gather.
2026-11-04T09:54:12.057523219Z Gathering data for ns/openshift-cloud-credential-operator...
2026-11-04T09:54:12.224742486Z Wrote inspect data to must-gather.
2026-11-04T09:54:12.297242973Z INFO: Getting status/config from prometheus-k8s-0
2026-11-04T09:54:12.532858556Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:54:12.535713738Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-11-04T09:54:12.564381388Z INFO: LEGACY MODE
2026-11-04T09:54:12.564425198Z INFO: Gathering ovn-kubernetes master data
2026-11-04T09:54:12.663373680Z Wrote inspect data to must-gather.
2026-11-04T09:54:12.814073854Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable no sooner than v6.0+
2026-11-04T09:54:12.986412388Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:54:13.068967690Z Wrote inspect data to must-gather.
2026-11-04T09:54:13.432732832Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-11-04T09:54:13.586864189Z Wrote inspect data to must-gather.
2026-11-04T09:54:13.606363509Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:54:13.608707243Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-11-04T09:54:13.706303018Z INFO: Getting status/flags from prometheus-k8s-0
2026-11-04T09:54:14.157117445Z Wrote inspect data to must-gather.
2026-11-04T09:54:14.199516712Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:54:14.582871699Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-11-04T09:54:14.598144505Z INFO: Waiting for network log collection to complete ...
2026-11-04T09:54:14.598204198Z INFO: Waiting for ovnk database copies to complete ...
2026-11-04T09:54:14.598584192Z INFO: Copying ovnk databases complete.
2026-11-04T09:54:14.600344090Z du: cannot access 'must-gather/network_logs/ovnk_database_store': No such file or directory
2026-11-04T09:54:14.601009544Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 802 is not a child of this shell
2026-11-04T09:54:14.601009544Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 803 is not a child of this shell
2026-11-04T09:54:14.601009544Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 805 is not a child of this shell
2026-11-04T09:54:14.601009544Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 807 is not a child of this shell
2026-11-04T09:54:14.601037889Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 808 is not a child of this shell
2026-11-04T09:54:14.601037889Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 809 is not a child of this shell
2026-11-04T09:54:14.601037889Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 810 is not a child of this shell
2026-11-04T09:54:14.601037889Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 812 is not a child of this shell
2026-11-04T09:54:14.601037889Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 820 is not a child of this shell
2026-11-04T09:54:14.601037889Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 821 is not a child of this shell
2026-11-04T09:54:14.601037889Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 823 is not a child of this shell
2026-11-04T09:54:14.601037889Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 831 is not a child of this shell
2026-11-04T09:54:14.601037889Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 838 is not a child of this shell
2026-11-04T09:54:14.605126406Z tar: ovnk_database_store: Cannot stat: No such file or directory
2026-11-04T09:54:14.609357417Z tar: Exiting with failure status due to previous errors
2026-11-04T09:54:14.609967044Z INFO: Network log collection complete.
2026-11-04T09:54:14.611052652Z Wrote inspect data to must-gather.
2026-11-04T09:54:14.893297680Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:54:14.896949914Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-11-04T09:54:15.016609394Z Gathering data for ns/openshift-monitoring...
2026-11-04T09:54:15.096595704Z Gathering data for ns/openshift-etcd...
2026-11-04T09:54:15.425351223Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:54:15.436703248Z INFO: Hosted cluster detected, get NTO container image from the TuneD pod
2026-11-04T09:54:15.480142591Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-11-04T09:54:15.522007589Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:54:15.809680628Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:36f60e56e0594610b09d7a64639223d0a6a314315bca9313914f95c902a1597c
2026-11-04T09:54:16.051960762Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:54:16.054894405Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-11-04T09:54:16.166212936Z daemonset.apps/perf-node-gather-daemonset created
2026-11-04T09:54:16.419400812Z Waiting for performance profile collector pods to become ready: 1
2026-11-04T09:54:16.557519918Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-11-04T09:54:16.582124795Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:54:16.924968727Z Wrote inspect data to must-gather.
2026-11-04T09:54:16.925004183Z error: inspection completed with the errors occurred while gathering data:
2026-11-04T09:54:16.925004183Z     namespaces "assisted-installer" not found
2026-11-04T09:54:17.267492411Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:54:17.270156509Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-11-04T09:54:17.655056497Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:54:17.682094569Z Waiting for performance profile collector pods to become ready: 2
2026-11-04T09:54:17.809157964Z INFO: Getting status from alertmanager-main-0
2026-11-04T09:54:18.319683511Z Wrote inspect data to /must-gather/istio.
2026-11-04T09:54:18.932449287Z Waiting for performance profile collector pods to become ready: 3
2026-11-04T09:54:20.179670696Z Done executing Istio gather script
2026-11-04T09:54:20.191679461Z Waiting for performance profile collector pods to become ready: 4
2026-11-04T09:54:20.526980483Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:54:21.442049768Z Waiting for performance profile collector pods to become ready: 5
2026-11-04T09:54:22.694581654Z Waiting for performance profile collector pods to become ready: 6
2026-11-04T09:54:23.946782067Z Waiting for performance profile collector pods to become ready: 7
2026-11-04T09:54:25.196749549Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-11-04T09:54:25.531599980Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:54:25.760677650Z Collecting performance related data for node ip-10-0-141-14.us-west-1.compute.internal
2026-11-04T09:54:25.768660723Z Collecting performance related data for node ip-10-0-128-178.us-west-1.compute.internal
2026-11-04T09:54:25.775461930Z Collecting performance related data for node ip-10-0-139-0.us-west-1.compute.internal
2026-11-04T09:54:30.536297192Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:54:35.540881192Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:54:36.107602400Z Gathering data for ns/openshift-network-console...
2026-11-04T09:54:38.768374562Z Gathering data for ns/openshift-console-operator...
2026-11-04T09:54:39.332804277Z Collecting kubelet logs for node ip-10-0-141-14.us-west-1.compute.internal
2026-11-04T09:54:39.334643389Z Collecting kubelet logs for node ip-10-0-128-178.us-west-1.compute.internal
2026-11-04T09:54:39.336427633Z Collecting kubelet logs for node ip-10-0-139-0.us-west-1.compute.internal
2026-11-04T09:54:40.167665168Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-9tvf5 namespace
2026-11-04T09:54:40.222949308Z INFO: Node performance data collection complete.
2026-11-04T09:54:40.545478137Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:54:41.171939640Z Gathering data for ns/openshift-console...
2026-11-04T09:54:44.291743354Z Gathering data for ns/openshift-cluster-storage-operator...
2026-11-04T09:54:45.549945747Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:54:47.483412291Z Gathering data for ns/openshift-dns-operator...
2026-11-04T09:54:49.561080074Z Gathering data for ns/openshift-dns...
2026-11-04T09:54:50.554842028Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:54:55.559533219Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:54:55.602029131Z Gathering data for ns/openshift-image-registry...
2026-11-04T09:54:59.268629541Z Gathering data for ns/openshift-ingress-operator...
2026-11-04T09:55:00.564114083Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:55:01.255213902Z Gathering data for ns/openshift-ingress...
2026-11-04T09:55:03.688028958Z Gathering data for ns/openshift-ingress-canary...
2026-11-04T09:55:05.568751708Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:55:06.939687468Z Gathering data for ns/openshift-insights...
2026-11-04T09:55:10.573450310Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:55:15.473733529Z Gathering data for ns/openshift-jobset-operator...
2026-11-04T09:55:15.577967791Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:55:18.565890217Z Gathering data for ns/openshift-config...
2026-11-04T09:55:20.415382010Z Gathering data for ns/openshift-config-managed...
2026-11-04T09:55:20.582571847Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:55:22.444688294Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-11-04T09:55:24.359740879Z Gathering data for ns/openshift-kube-apiserver...
2026-11-04T09:55:25.587113852Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:55:26.206735376Z Gathering data for ns/openshift-kube-controller-manager...
2026-11-04T09:55:28.170688370Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-11-04T09:55:30.178324089Z Gathering data for ns/openshift-kube-scheduler...
2026-11-04T09:55:30.591631630Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:55:32.059477904Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-11-04T09:55:34.026854132Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-11-04T09:55:35.595986069Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:55:36.727097834Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-11-04T09:55:38.993737040Z Gathering data for ns/openshift-user-workload-monitoring...
2026-11-04T09:55:40.600499768Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:55:41.693039736Z Gathering data for ns/openshift-multus...
2026-11-04T09:55:45.604970306Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:55:50.609839578Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:55:55.614492154Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:55:58.303591936Z Gathering data for ns/openshift-ovn-kubernetes...
2026-11-04T09:56:00.619067206Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:56:05.623609812Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:56:10.628187720Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:56:11.876228835Z Gathering data for ns/openshift-host-network...
2026-11-04T09:56:14.018437003Z Gathering data for ns/openshift-network-diagnostics...
2026-11-04T09:56:15.632747329Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:56:18.645292742Z Gathering data for ns/openshift-network-node-identity...
2026-11-04T09:56:20.637378033Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:56:21.703744795Z Gathering data for ns/openshift-network-operator...
2026-11-04T09:56:24.910323855Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-11-04T09:56:25.641991539Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:56:26.754192935Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-11-04T09:56:30.333737968Z Gathering data for ns/openshift-apiserver-operator...
2026-11-04T09:56:30.646587375Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:56:32.208054967Z Gathering data for ns/openshift-apiserver...
2026-11-04T09:56:34.754751260Z Gathering data for ns/openshift-controller-manager-operator...
2026-11-04T09:56:35.651364338Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:56:36.584072407Z Gathering data for ns/openshift-controller-manager...
2026-11-04T09:56:38.503288092Z Gathering data for ns/openshift-cluster-samples-operator...
2026-11-04T09:56:40.655918966Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:56:41.667065594Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-11-04T09:56:43.574213244Z Gathering data for ns/openshift-service-ca-operator...
2026-11-04T09:56:45.660500891Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:56:45.859698266Z Gathering data for ns/openshift-service-ca...
2026-11-04T09:56:48.498625565Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-11-04T09:56:50.665133794Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-11-04T09:56:54.590815133Z Wrote inspect data to must-gather.
2026-11-04T09:56:54.590855956Z error: inspection completed with the errors occurred while gathering data:
2026-11-04T09:56:54.590855956Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-11-04T09:56:55.015817058Z error: the server doesn't have a resource type "clusters"
2026-11-04T09:56:55.056347926Z Caches written to disk