2026-07-07T14:00:14.486834204Z [disk usage checker] Started
2026-07-07T14:00:14.490249762Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-07T14:00:14.679107606Z Gathering data for ns/openshift-cluster-version...
2026-07-07T14:00:14.811616597Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-07T14:00:14.895928991Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-07T14:00:14.980292834Z Gathering data for ns/default...
2026-07-07T14:00:15.262078518Z Gathering data for ns/openshift...
2026-07-07T14:00:15.663422729Z Gathering data for ns/kube-system...
2026-07-07T14:00:16.388752960Z Gathering data for ns/openshift-etcd...
2026-07-07T14:00:16.691132987Z Wrote inspect data to must-gather.
2026-07-07T14:00:16.691167044Z error: inspection completed with the errors occurred while gathering data:
2026-07-07T14:00:16.691167044Z     namespaces "assisted-installer" not found
2026-07-07T14:00:16.952523341Z Waiting on subprocesses to finish execution.
2026-07-07T14:00:16.956453576Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-07T14:00:16.961605393Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-07T14:00:16.965573351Z Executing Istio gather script
2026-07-07T14:00:16.967729344Z INFO: Gathering HAProxy config files
2026-07-07T14:00:16.976602948Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-07T14:00:17.040947516Z INFO: Collecting host service logs for crio
2026-07-07T14:00:17.041317512Z INFO: Collecting host service logs for kubelet
2026-07-07T14:00:17.041646957Z INFO: Collecting host service logs for rpm-ostreed
2026-07-07T14:00:17.041944613Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-07T14:00:17.042254275Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-07T14:00:17.042560740Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-07T14:00:17.042856431Z INFO: Collecting host service logs for NetworkManager
2026-07-07T14:00:17.043163658Z INFO: Collecting host service logs for openvswitch
2026-07-07T14:00:17.043455874Z INFO: Collecting host service logs for ovs-configuration
2026-07-07T14:00:17.043757940Z INFO: Collecting host service logs for ovsdb-server
2026-07-07T14:00:17.044048467Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-07T14:00:17.044424571Z INFO: Waiting for worker host service log collection to complete ...
2026-07-07T14:00:17.071864790Z INFO: Waiting for node performance related collection to complete ...
2026-07-07T14:00:17.533681418Z INFO: "metallb-operator" not detected. Skipping.
2026-07-07T14:00:17.738729665Z No resources found in openshift-etcd namespace.
2026-07-07T14:00:17.791505151Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-07T14:00:17.826029868Z error: the server doesn't have a resource type "performanceprofile"
2026-07-07T14:00:17.990806685Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-07T14:00:17.990884836Z INFO: on-disk MachineConfig config collection complete.
2026-07-07T14:00:17.991651776Z Wrote inspect data to must-gather.
2026-07-07T14:00:18.039886879Z No resources found
2026-07-07T14:00:18.138621829Z INFO: Collecting Insights Archives from insights-operator-5c847dcfc4-xk9lh
2026-07-07T14:00:18.138676569Z insights-runtime-extractor-d8gv5
2026-07-07T14:00:18.138710687Z insights-runtime-extractor-hh7ct
2026-07-07T14:00:18.138730976Z insights-runtime-extractor-x4mbw
2026-07-07T14:00:18.179090469Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-07T14:00:18.214672627Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-07T14:00:18.415233259Z ERROR: No running kube-apiserver pods found
2026-07-07T14:00:18.447127699Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-07T14:00:18.628170096Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-07T14:00:18.715160068Z INFO: Worker host service log collection to complete.
2026-07-07T14:00:18.817947936Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-07T14:00:18.837172117Z Wrote inspect data to must-gather.
2026-07-07T14:00:18.910621605Z Wrote inspect data to must-gather.
2026-07-07T14:00:18.944984075Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-07T14:00:18.944984075Z See 'oc rsync -h' for help and examples
2026-07-07T14:00:19.154284461Z Wrote inspect data to must-gather.
2026-07-07T14:00:19.170510020Z Gathering data for ns/openshift-monitoring...
2026-07-07T14:00:19.204683795Z error: the server doesn't have a resource type "machineconfigs"
2026-07-07T14:00:19.261648086Z No resources found
2026-07-07T14:00:19.272706575Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-07T14:00:19.278981681Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-07T14:00:19.302618386Z Wrote inspect data to must-gather.
2026-07-07T14:00:19.319388606Z Wrote inspect data to must-gather.
2026-07-07T14:00:19.374740772Z INFO: 'previous-logs' folder not found on ip-10-0-137-20.ec2.internal, skipping...
2026-07-07T14:00:19.380408906Z Wrote inspect data to must-gather.
2026-07-07T14:00:19.450215288Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-07T14:00:19.497149819Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-07T14:00:19.587621280Z tar: Removing leading `/' from member names
2026-07-07T14:00:19.594464341Z INFO: HAProxy config collection complete.
2026-07-07T14:00:19.687536438Z Wrote inspect data to must-gather.
2026-07-07T14:00:19.691910311Z INFO: 'previous-logs' folder not found on ip-10-0-140-101.ec2.internal, skipping...
2026-07-07T14:00:19.715911171Z Wrote inspect data to must-gather.
2026-07-07T14:00:19.765969871Z INFO: Getting rules from prometheus-k8s-0
2026-07-07T14:00:19.788504928Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-07T14:00:19.878933748Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-07T14:00:19.963837774Z INFO: 'previous-logs' folder not found on ip-10-0-143-229.ec2.internal, skipping...
2026-07-07T14:00:19.963881395Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-07T14:00:19.963910321Z INFO: Machine Config Daemon termination log collection complete.
2026-07-07T14:00:19.973841280Z Wrote inspect data to must-gather.
2026-07-07T14:00:20.093702961Z Wrote inspect data to must-gather.
2026-07-07T14:00:20.170862296Z INFO: Getting status/config from prometheus-k8s-0
2026-07-07T14:00:20.211129137Z Wrote inspect data to must-gather.
2026-07-07T14:00:20.263985408Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-07T14:00:20.315952287Z Wrote inspect data to must-gather.
2026-07-07T14:00:20.448870849Z Wrote inspect data to /must-gather/istio.
2026-07-07T14:00:20.495114710Z INFO: INTERCONNECT MODE
2026-07-07T14:00:20.495145081Z INFO: Gathering ovn-kubernetes DBs
2026-07-07T14:00:20.528498795Z Wrote inspect data to must-gather.
2026-07-07T14:00:20.569981691Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-07T14:00:20.658877772Z Wrote inspect data to /must-gather/istio.
2026-07-07T14:00:20.661530031Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-07T14:00:20.665223747Z INFO: Gathering OVN_Northbound from ovnkube-node-8bt94...
2026-07-07T14:00:20.665476497Z INFO: Gathering OVN_Northbound from ovnkube-node-cw57h...
2026-07-07T14:00:20.665722013Z INFO: Gathering OVN_Northbound from ovnkube-node-m2jfs...
2026-07-07T14:00:20.666006405Z INFO: Gathering OVN_Southbound from ovnkube-node-8bt94...
2026-07-07T14:00:20.666320316Z INFO: Gathering OVN_Southbound from ovnkube-node-cw57h...
2026-07-07T14:00:20.666590557Z INFO: Gathering OVN_Southbound from ovnkube-node-m2jfs...
2026-07-07T14:00:20.823863771Z Wrote inspect data to must-gather.
2026-07-07T14:00:20.862525840Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-07T14:00:20.962477370Z tar: Removing leading `/' from member names
2026-07-07T14:00:20.984960873Z tar: Removing leading `/' from member names
2026-07-07T14:00:21.007395666Z tar: Removing leading `/' from member names
2026-07-07T14:00:21.036277997Z tar: Removing leading `/' from member names
2026-07-07T14:00:21.037188224Z Wrote inspect data to /must-gather/istio.
2026-07-07T14:00:21.051047393Z tar: Removing leading `/' from member names
2026-07-07T14:00:21.053727420Z tar: Removing leading `/' from member names
2026-07-07T14:00:21.092264735Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-07T14:00:21.108987221Z Wrote inspect data to must-gather.
2026-07-07T14:00:21.164510086Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-07T14:00:21.274558411Z Wrote inspect data to /must-gather/istio.
2026-07-07T14:00:21.277009881Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-07T14:00:21.355410146Z Wrote inspect data to must-gather.
2026-07-07T14:00:21.442291228Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-07T14:00:21.503012129Z Wrote inspect data to /must-gather/istio.
2026-07-07T14:00:21.517671211Z INFO: Waiting for network log collection to complete ...
2026-07-07T14:00:21.517723320Z INFO: Waiting for ovnk database copies to complete ...
2026-07-07T14:00:21.518214075Z INFO: Copying ovnk databases complete.
2026-07-07T14:00:21.520291164Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-07-07T14:00:21.522952643Z ovnk_database_store/
2026-07-07T14:00:21.522952643Z ovnk_database_store/ovnkube-node-cw57h_nbdb
2026-07-07T14:00:21.530046955Z ovnk_database_store/ovnkube-node-8bt94_nbdb
2026-07-07T14:00:21.537895811Z ovnk_database_store/ovnkube-node-8bt94_sbdb
2026-07-07T14:00:21.555881335Z ovnk_database_store/ovnkube-node-cw57h_sbdb
2026-07-07T14:00:21.571521614Z ovnk_database_store/ovnkube-node-m2jfs_sbdb
2026-07-07T14:00:21.582428212Z Wrote inspect data to must-gather.
2026-07-07T14:00:21.590485544Z ovnk_database_store/ovnkube-node-m2jfs_nbdb
2026-07-07T14:00:21.601887512Z INFO: Network log collection complete.
2026-07-07T14:00:21.727717909Z Wrote inspect data to /must-gather/istio.
2026-07-07T14:00:21.729892649Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-07T14:00:21.777924864Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-07T14:00:21.851854537Z INFO: Getting status from alertmanager-main-0
2026-07-07T14:00:21.856209881Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-07T14:00:21.858175964Z INFO: Fallback to identify the container image from release info
2026-07-07T14:00:21.973830217Z Wrote inspect data to /must-gather/istio.
2026-07-07T14:00:21.981913907Z Gathering data for ns/openshift-network-console...
2026-07-07T14:00:22.155669448Z Wrote inspect data to /must-gather/istio.
2026-07-07T14:00:22.158030229Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-07T14:00:22.321398220Z Wrote inspect data to /must-gather/istio.
2026-07-07T14:00:22.386960105Z Gathering data for ns/openshift-console-operator...
2026-07-07T14:00:22.505076947Z Wrote inspect data to /must-gather/istio.
2026-07-07T14:00:22.761123945Z Gathering data for ns/openshift-console...
2026-07-07T14:00:23.106915537Z Done executing Istio gather script
2026-07-07T14:00:23.239506911Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-07T14:00:23.691681671Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5be8cdcf34f35a89a70b6bfa49e66897641871237570b3e09766f83a0a473e6a
2026-07-07T14:00:23.717379220Z Gathering data for ns/openshift-dns-operator...
2026-07-07T14:00:23.815007403Z daemonset.apps/perf-node-gather-daemonset created
2026-07-07T14:00:23.917982742Z Waiting for performance profile collector pods to become ready: 1
2026-07-07T14:00:24.055522235Z Gathering data for ns/openshift-dns...
2026-07-07T14:00:24.503402033Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-07T14:00:24.668466704Z Gathering data for ns/openshift-image-registry...
2026-07-07T14:00:25.030370942Z Waiting for performance profile collector pods to become ready: 2
2026-07-07T14:00:25.148955023Z Gathering data for ns/openshift-ingress-operator...
2026-07-07T14:00:25.479360643Z Gathering data for ns/openshift-ingress...
2026-07-07T14:00:25.788572944Z Gathering data for ns/openshift-ingress-canary...
2026-07-07T14:00:26.145803877Z Waiting for performance profile collector pods to become ready: 3
2026-07-07T14:00:26.178626421Z Gathering data for ns/openshift-insights...
2026-07-07T14:00:27.248002326Z Waiting for performance profile collector pods to become ready: 4
2026-07-07T14:00:28.026252403Z Gathering data for ns/openshift-config...
2026-07-07T14:00:28.291823104Z Gathering data for ns/openshift-config-managed...
2026-07-07T14:00:28.373049546Z Waiting for performance profile collector pods to become ready: 5
2026-07-07T14:00:28.705802303Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-07T14:00:29.007538790Z Gathering data for ns/openshift-kube-apiserver...
2026-07-07T14:00:29.279223904Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-07T14:00:29.469132854Z Waiting for performance profile collector pods to become ready: 6
2026-07-07T14:00:29.508075243Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-07T14:00:29.601102107Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-07T14:00:29.859290740Z Gathering data for ns/openshift-kube-scheduler...
2026-07-07T14:00:30.104180283Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-07T14:00:30.424311698Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-07T14:00:30.583145457Z Waiting for performance profile collector pods to become ready: 7
2026-07-07T14:00:30.748100998Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-07T14:00:31.056372414Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-07T14:00:31.519209145Z Gathering data for ns/openshift-multus...
2026-07-07T14:00:31.680659410Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-07T14:00:31.884342155Z Collecting performance related data for node ip-10-0-143-229.ec2.internal
2026-07-07T14:00:31.891091923Z Collecting performance related data for node ip-10-0-140-101.ec2.internal
2026-07-07T14:00:31.898574503Z Collecting performance related data for node ip-10-0-137-20.ec2.internal
2026-07-07T14:00:33.187439227Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-07T14:00:34.514526127Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-07T14:00:34.553727479Z Gathering data for ns/openshift-host-network...
2026-07-07T14:00:34.872950088Z Gathering data for ns/openshift-network-diagnostics...
2026-07-07T14:00:35.398761238Z Gathering data for ns/openshift-network-node-identity...
2026-07-07T14:00:35.799424264Z Gathering data for ns/openshift-network-operator...
2026-07-07T14:00:36.178879937Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-07T14:00:36.438095003Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-07T14:00:36.930670227Z Gathering data for ns/openshift-apiserver-operator...
2026-07-07T14:00:37.231357973Z Gathering data for ns/openshift-apiserver...
2026-07-07T14:00:37.635281941Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-07T14:00:37.895937783Z Gathering data for ns/openshift-controller-manager...
2026-07-07T14:00:38.168305604Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-07T14:00:38.843181535Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-07T14:00:39.125910273Z Gathering data for ns/openshift-service-ca-operator...
2026-07-07T14:00:39.438418549Z Gathering data for ns/openshift-service-ca...
2026-07-07T14:00:39.487849135Z Collecting kubelet logs for node ip-10-0-143-229.ec2.internal
2026-07-07T14:00:39.490157978Z Collecting kubelet logs for node ip-10-0-140-101.ec2.internal
2026-07-07T14:00:39.492487385Z Collecting kubelet logs for node ip-10-0-137-20.ec2.internal
2026-07-07T14:00:39.522497666Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-07T14:00:39.840369591Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-07T14:00:39.845995043Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-4hwmv namespace
2026-07-07T14:00:39.851070779Z INFO: Node performance data collection complete.
2026-07-07T14:00:40.721537562Z Wrote inspect data to must-gather.
2026-07-07T14:00:40.721589358Z error: inspection completed with the errors occurred while gathering data:
2026-07-07T14:00:40.721589358Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-07T14:00:40.847485628Z error: the server doesn't have a resource type "clusters"
2026-07-07T14:00:40.973581731Z Caches written to disk