2026-09-16T19:48:59.705957331Z [disk usage checker] Started
2026-09-16T19:48:59.709178250Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-16T19:48:59.917344975Z Gathering data for ns/openshift-cluster-version...
2026-09-16T19:49:00.066825783Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-16T19:49:00.154218329Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T19:49:00.271035172Z Gathering data for ns/default...
2026-09-16T19:49:00.598094632Z Gathering data for ns/openshift...
2026-09-16T19:49:01.053201847Z Gathering data for ns/kube-system...
2026-09-16T19:49:01.823279227Z Gathering data for ns/openshift-etcd...
2026-09-16T19:49:02.162772071Z Wrote inspect data to must-gather.
2026-09-16T19:49:02.162802568Z error: inspection completed with the errors occurred while gathering data:
2026-09-16T19:49:02.162802568Z     namespaces "assisted-installer" not found
2026-09-16T19:49:02.423608947Z Waiting on subprocesses to finish execution.
2026-09-16T19:49:02.430120144Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-16T19:49:02.434440935Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-16T19:49:02.437028939Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-16T19:49:02.437302519Z Executing Istio gather script
2026-09-16T19:49:02.441591724Z INFO: Gathering HAProxy config files
2026-09-16T19:49:02.469943515Z INFO: Waiting for node performance related collection to complete ...
2026-09-16T19:49:02.477476375Z INFO: Collecting host service logs for crio
2026-09-16T19:49:02.477730523Z INFO: Collecting host service logs for kubelet
2026-09-16T19:49:02.478006941Z INFO: Collecting host service logs for rpm-ostreed
2026-09-16T19:49:02.478268253Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-16T19:49:02.478556322Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-16T19:49:02.478820214Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-16T19:49:02.479075292Z INFO: Collecting host service logs for NetworkManager
2026-09-16T19:49:02.479328919Z INFO: Collecting host service logs for openvswitch
2026-09-16T19:49:02.479621870Z INFO: Collecting host service logs for ovs-configuration
2026-09-16T19:49:02.479891352Z INFO: Collecting host service logs for ovsdb-server
2026-09-16T19:49:02.480159861Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-16T19:49:02.480500194Z INFO: Waiting for worker host service log collection to complete ...
2026-09-16T19:49:02.877983560Z INFO: Collecting Insights Archives from insights-operator-5844b856db-ms6lz
2026-09-16T19:49:02.878053487Z insights-runtime-extractor-4k7s2
2026-09-16T19:49:02.878074760Z insights-runtime-extractor-lprs5
2026-09-16T19:49:02.878093965Z insights-runtime-extractor-v7bkm
2026-09-16T19:49:02.995886767Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-16T19:49:03.331802572Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-16T19:49:03.376522756Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-16T19:49:03.414897317Z No resources found in openshift-etcd namespace.
2026-09-16T19:49:03.482702631Z Wrote inspect data to must-gather.
2026-09-16T19:49:03.500978625Z error: the server doesn't have a resource type "performanceprofile"
2026-09-16T19:49:03.549833354Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-16T19:49:03.549833354Z See 'oc rsync -h' for help and examples
2026-09-16T19:49:03.588505856Z INFO: "metallb-operator" not detected. Skipping.
2026-09-16T19:49:03.617105794Z No resources found
2026-09-16T19:49:03.924517016Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-16T19:49:03.951418458Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-16T19:49:03.951498038Z INFO: on-disk MachineConfig config collection complete.
2026-09-16T19:49:03.992325607Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-16T19:49:04.015927281Z INFO: Worker host service log collection to complete.
2026-09-16T19:49:04.060759412Z ERROR: No running kube-apiserver pods found
2026-09-16T19:49:04.309737225Z Wrote inspect data to must-gather.
2026-09-16T19:49:04.525566759Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-16T19:49:04.551814700Z No resources found
2026-09-16T19:49:04.635697669Z tar: Removing leading `/' from member names
2026-09-16T19:49:04.641452377Z INFO: HAProxy config collection complete.
2026-09-16T19:49:04.697713638Z Wrote inspect data to must-gather.
2026-09-16T19:49:04.769473397Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T19:49:04.803303683Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-16T19:49:04.856309044Z Wrote inspect data to must-gather.
2026-09-16T19:49:04.883696633Z Wrote inspect data to must-gather.
2026-09-16T19:49:04.945688164Z Gathering data for ns/openshift-monitoring...
2026-09-16T19:49:04.949266511Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-16T19:49:05.046733880Z Wrote inspect data to must-gather.
2026-09-16T19:49:05.074353017Z INFO: 'previous-logs' folder not found on ip-10-0-136-67.ec2.internal, skipping...
2026-09-16T19:49:05.144282844Z error: the server doesn't have a resource type "machineconfigs"
2026-09-16T19:49:05.158643609Z Wrote inspect data to must-gather.
2026-09-16T19:49:05.335150681Z INFO: Getting rules from prometheus-k8s-0
2026-09-16T19:49:05.371174000Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-16T19:49:05.399802856Z Wrote inspect data to must-gather.
2026-09-16T19:49:05.420661540Z INFO: 'previous-logs' folder not found on ip-10-0-140-94.ec2.internal, skipping...
2026-09-16T19:49:05.645043761Z Wrote inspect data to must-gather.
2026-09-16T19:49:05.701031371Z Wrote inspect data to must-gather.
2026-09-16T19:49:05.716652487Z INFO: 'previous-logs' folder not found on ip-10-0-141-147.ec2.internal, skipping...
2026-09-16T19:49:05.716680533Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-16T19:49:05.716701445Z INFO: Machine Config Daemon termination log collection complete.
2026-09-16T19:49:05.822287987Z INFO: Getting status/config from prometheus-k8s-0
2026-09-16T19:49:05.827954321Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-16T19:49:05.983536283Z Wrote inspect data to must-gather.
2026-09-16T19:49:06.077867419Z Wrote inspect data to must-gather.
2026-09-16T19:49:06.197820684Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-16T19:49:06.266860214Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-16T19:49:06.312804331Z Wrote inspect data to must-gather.
2026-09-16T19:49:06.312836086Z INFO: INTERCONNECT MODE
2026-09-16T19:49:06.312899186Z INFO: Gathering ovn-kubernetes DBs
2026-09-16T19:49:06.483974955Z INFO: Gathering OVN_Northbound from ovnkube-node-5tr9z...
2026-09-16T19:49:06.484248371Z INFO: Gathering OVN_Northbound from ovnkube-node-cp5kh...
2026-09-16T19:49:06.484509710Z INFO: Gathering OVN_Northbound from ovnkube-node-slx6p...
2026-09-16T19:49:06.484790699Z INFO: Gathering OVN_Southbound from ovnkube-node-5tr9z...
2026-09-16T19:49:06.485040237Z INFO: Gathering OVN_Southbound from ovnkube-node-cp5kh...
2026-09-16T19:49:06.485280719Z INFO: Gathering OVN_Southbound from ovnkube-node-slx6p...
2026-09-16T19:49:06.532433862Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-16T19:49:06.598488637Z Wrote inspect data to must-gather.
2026-09-16T19:49:06.763258180Z tar: Removing leading `/' from member names
2026-09-16T19:49:06.767262437Z tar: Removing leading `/' from member names
2026-09-16T19:49:06.768839402Z tar: Removing leading `/' from member names
2026-09-16T19:49:06.880960132Z tar: Removing leading `/' from member names
2026-09-16T19:49:06.887014085Z tar: Removing leading `/' from member names
2026-09-16T19:49:06.917040115Z tar: Removing leading `/' from member names
2026-09-16T19:49:06.954573184Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-16T19:49:06.961824526Z Wrote inspect data to must-gather.
2026-09-16T19:49:06.987973933Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-16T19:49:07.149652040Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-16T19:49:07.216225620Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-16T19:49:07.219020494Z Wrote inspect data to must-gather.
2026-09-16T19:49:07.370896835Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:49:07.382073946Z INFO: Waiting for network log collection to complete ...
2026-09-16T19:49:07.382121773Z INFO: Waiting for ovnk database copies to complete ...
2026-09-16T19:49:07.382623589Z INFO: Copying ovnk databases complete.
2026-09-16T19:49:07.384142221Z 2.9M	must-gather/network_logs/ovnk_database_store
2026-09-16T19:49:07.386284355Z ovnk_database_store/
2026-09-16T19:49:07.386284355Z ovnk_database_store/ovnkube-node-5tr9z_nbdb
2026-09-16T19:49:07.394740974Z ovnk_database_store/ovnkube-node-slx6p_sbdb
2026-09-16T19:49:07.412309069Z ovnk_database_store/ovnkube-node-cp5kh_nbdb
2026-09-16T19:49:07.420391685Z ovnk_database_store/ovnkube-node-cp5kh_sbdb
2026-09-16T19:49:07.439797449Z ovnk_database_store/ovnkube-node-5tr9z_sbdb
2026-09-16T19:49:07.457791195Z ovnk_database_store/ovnkube-node-slx6p_nbdb
2026-09-16T19:49:07.465419968Z Wrote inspect data to must-gather.
2026-09-16T19:49:07.467470516Z INFO: Network log collection complete.
2026-09-16T19:49:07.582702796Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:49:07.586414139Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-16T19:49:07.642625630Z INFO: Getting status from alertmanager-main-0
2026-09-16T19:49:07.705195433Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T19:49:07.798702165Z Wrote inspect data to must-gather.
2026-09-16T19:49:07.813722682Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:49:07.910440654Z Gathering data for ns/openshift-network-console...
2026-09-16T19:49:08.035117730Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:49:08.037942965Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-16T19:49:08.084630979Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-16T19:49:08.086833036Z INFO: Fallback to identify the container image from release info
2026-09-16T19:49:08.231632216Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:49:08.400508866Z Gathering data for ns/openshift-console-operator...
2026-09-16T19:49:08.423739012Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:49:08.426215928Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-16T19:49:08.629916697Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:49:08.717571051Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a
2026-09-16T19:49:08.842616170Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:49:08.845770968Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-09-16T19:49:08.875700004Z daemonset.apps/perf-node-gather-daemonset created
2026-09-16T19:49:08.934872160Z Gathering data for ns/openshift-console...
2026-09-16T19:49:09.000092344Z Waiting for performance profile collector pods to become ready: 1
2026-09-16T19:49:09.033207911Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:49:09.221492978Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:49:09.223852964Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-09-16T19:49:09.418983905Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:49:09.555698257Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-16T19:49:09.625724409Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:49:09.628402830Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-16T19:49:09.785099720Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:49:09.812494583Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-16T19:49:09.996371797Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:49:10.148903787Z Waiting for performance profile collector pods to become ready: 2
2026-09-16T19:49:10.177656628Z Gathering data for ns/openshift-dns-operator...
2026-09-16T19:49:10.529075585Z Gathering data for ns/openshift-dns...
2026-09-16T19:49:10.697814339Z Done executing Istio gather script
2026-09-16T19:49:11.276277963Z Waiting for performance profile collector pods to become ready: 3
2026-09-16T19:49:11.320346239Z Gathering data for ns/openshift-image-registry...
2026-09-16T19:49:11.879604779Z Gathering data for ns/openshift-ingress-operator...
2026-09-16T19:49:12.309056778Z Gathering data for ns/openshift-ingress...
2026-09-16T19:49:12.400635428Z Waiting for performance profile collector pods to become ready: 4
2026-09-16T19:49:12.735051962Z Gathering data for ns/openshift-ingress-canary...
2026-09-16T19:49:13.232449435Z Gathering data for ns/openshift-insights...
2026-09-16T19:49:13.513198547Z Waiting for performance profile collector pods to become ready: 5
2026-09-16T19:49:14.634696637Z Waiting for performance profile collector pods to become ready: 6
2026-09-16T19:49:14.817481167Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-16T19:49:15.682012898Z Gathering data for ns/kserve...
2026-09-16T19:49:15.743608962Z Waiting for performance profile collector pods to become ready: 7
2026-09-16T19:49:16.869865117Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-16T19:49:17.048796951Z Gathering data for ns/openshift-config...
2026-09-16T19:49:17.135636256Z Collecting performance related data for node ip-10-0-136-67.ec2.internal
2026-09-16T19:49:17.142212983Z Collecting performance related data for node ip-10-0-140-94.ec2.internal
2026-09-16T19:49:17.149147615Z Collecting performance related data for node ip-10-0-141-147.ec2.internal
2026-09-16T19:49:17.393903440Z Gathering data for ns/openshift-config-managed...
2026-09-16T19:49:17.902221738Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-16T19:49:18.291874401Z Gathering data for ns/openshift-kube-apiserver...
2026-09-16T19:49:18.636994136Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-16T19:49:19.009519648Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-16T19:49:19.363525122Z Gathering data for ns/openshift-kube-scheduler...
2026-09-16T19:49:19.698601460Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-16T19:49:19.821866670Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-16T19:49:20.099259875Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-16T19:49:20.442299837Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-16T19:49:20.768207691Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-16T19:49:21.317464139Z Gathering data for ns/openshift-multus...
2026-09-16T19:49:22.826504870Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-16T19:49:24.123467852Z Gathering data for ns/openshift-host-network...
2026-09-16T19:49:24.456846484Z Gathering data for ns/openshift-network-diagnostics...
2026-09-16T19:49:24.826444542Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-16T19:49:25.022727316Z Gathering data for ns/openshift-network-node-identity...
2026-09-16T19:49:25.447898412Z Gathering data for ns/openshift-network-operator...
2026-09-16T19:49:25.450826293Z Collecting kubelet logs for node ip-10-0-136-67.ec2.internal
2026-09-16T19:49:25.452580354Z Collecting kubelet logs for node ip-10-0-140-94.ec2.internal
2026-09-16T19:49:25.454280424Z Collecting kubelet logs for node ip-10-0-141-147.ec2.internal
2026-09-16T19:49:25.796963320Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-fdmt5 namespace
2026-09-16T19:49:25.801798597Z INFO: Node performance data collection complete.
2026-09-16T19:49:25.902524814Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-16T19:49:26.263989364Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-16T19:49:26.763808024Z Gathering data for ns/openshift-apiserver-operator...
2026-09-16T19:49:27.112939605Z Gathering data for ns/openshift-apiserver...
2026-09-16T19:49:27.494684026Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-16T19:49:27.808984330Z Gathering data for ns/openshift-controller-manager...
2026-09-16T19:49:28.095771139Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-16T19:49:28.773572587Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-16T19:49:29.100695214Z Gathering data for ns/openshift-service-ca-operator...
2026-09-16T19:49:29.411678994Z Gathering data for ns/openshift-service-ca...
2026-09-16T19:49:29.835352772Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-16T19:49:29.847332045Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-16T19:49:30.756891946Z Wrote inspect data to must-gather.
2026-09-16T19:49:30.756937255Z error: inspection completed with the errors occurred while gathering data:
2026-09-16T19:49:30.756937255Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-16T19:49:30.882411494Z error: the server doesn't have a resource type "clusters"
2026-09-16T19:49:31.008752760Z Caches written to disk