2026-09-17T03:44:53.984508897Z [disk usage checker] Started
2026-09-17T03:44:53.987921436Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-17T03:44:54.191211879Z Gathering data for ns/openshift-cluster-version...
2026-09-17T03:44:54.322595076Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-17T03:44:54.404221694Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T03:44:54.483729074Z Gathering data for ns/default...
2026-09-17T03:44:54.769800088Z Gathering data for ns/openshift...
2026-09-17T03:44:55.179785386Z Gathering data for ns/kube-system...
2026-09-17T03:44:55.861770155Z Gathering data for ns/openshift-etcd...
2026-09-17T03:44:56.149071608Z Wrote inspect data to must-gather.
2026-09-17T03:44:56.149100515Z error: inspection completed with the errors occurred while gathering data:
2026-09-17T03:44:56.149100515Z     namespaces "assisted-installer" not found
2026-09-17T03:44:56.517242782Z Waiting on subprocesses to finish execution.
2026-09-17T03:44:56.524051664Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-17T03:44:56.525545159Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-17T03:44:56.528150593Z INFO: Gathering HAProxy config files
2026-09-17T03:44:56.531858563Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-17T03:44:56.533345946Z INFO: Collecting host service logs for crio
2026-09-17T03:44:56.533641554Z INFO: Collecting host service logs for kubelet
2026-09-17T03:44:56.533947364Z INFO: Collecting host service logs for rpm-ostreed
2026-09-17T03:44:56.534258533Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-17T03:44:56.534567389Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-17T03:44:56.534971435Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-17T03:44:56.535315474Z INFO: Collecting host service logs for NetworkManager
2026-09-17T03:44:56.535659179Z INFO: Collecting host service logs for openvswitch
2026-09-17T03:44:56.535998817Z INFO: Collecting host service logs for ovs-configuration
2026-09-17T03:44:56.536346948Z INFO: Collecting host service logs for ovsdb-server
2026-09-17T03:44:56.536698418Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-17T03:44:56.537097429Z INFO: Waiting for worker host service log collection to complete ...
2026-09-17T03:44:56.564429938Z Executing Istio gather script
2026-09-17T03:44:56.642796511Z INFO: Waiting for node performance related collection to complete ...
2026-09-17T03:44:57.016190965Z INFO: "metallb-operator" not detected. Skipping.
2026-09-17T03:44:57.113752957Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-17T03:44:57.244211540Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-17T03:44:57.307913921Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-17T03:44:57.307998986Z INFO: on-disk MachineConfig config collection complete.
2026-09-17T03:44:57.324478980Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-17T03:44:57.420093401Z error: the server doesn't have a resource type "performanceprofile"
2026-09-17T03:44:57.552695071Z No resources found in openshift-etcd namespace.
2026-09-17T03:44:57.554796696Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-17T03:44:57.680807644Z INFO: Collecting Insights Archives from insights-operator-5844b856db-ptjvk
2026-09-17T03:44:57.680874288Z insights-runtime-extractor-pbcm5
2026-09-17T03:44:57.680896742Z insights-runtime-extractor-pxhgp
2026-09-17T03:44:57.680918135Z insights-runtime-extractor-ztkwj
2026-09-17T03:44:57.762264555Z No resources found
2026-09-17T03:44:57.775201516Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-17T03:44:57.880129499Z INFO: Worker host service log collection to complete.
2026-09-17T03:44:58.018453357Z INFO: HAProxy config collection complete.
2026-09-17T03:44:58.303981157Z Wrote inspect data to must-gather.
2026-09-17T03:44:58.425137155Z Wrote inspect data to must-gather.
2026-09-17T03:44:58.444555427Z Wrote inspect data to must-gather.
2026-09-17T03:44:58.477457610Z Wrote inspect data to must-gather.
2026-09-17T03:44:58.549690258Z ERROR: No running kube-apiserver pods found
2026-09-17T03:44:58.611641887Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-17T03:44:58.611641887Z See 'oc rsync -h' for help and examples
2026-09-17T03:44:58.667130532Z INFO: 'previous-logs' folder not found on ip-10-0-133-213.ec2.internal, skipping...
2026-09-17T03:44:58.731269791Z Wrote inspect data to must-gather.
2026-09-17T03:44:58.819647242Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-17T03:44:58.893469226Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-17T03:44:58.913443381Z Wrote inspect data to must-gather.
2026-09-17T03:44:58.918223434Z No resources found
2026-09-17T03:44:58.953272810Z Wrote inspect data to must-gather.
2026-09-17T03:44:58.977869348Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T03:44:59.004004960Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-17T03:44:59.035401190Z error: the server doesn't have a resource type "machineconfigs"
2026-09-17T03:44:59.192847145Z Gathering data for ns/openshift-monitoring...
2026-09-17T03:44:59.195061743Z Wrote inspect data to must-gather.
2026-09-17T03:44:59.237404188Z INFO: 'previous-logs' folder not found on ip-10-0-139-110.ec2.internal, skipping...
2026-09-17T03:44:59.247629569Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-17T03:44:59.380395862Z INFO: Getting rules from prometheus-k8s-0
2026-09-17T03:44:59.510245618Z Wrote inspect data to must-gather.
2026-09-17T03:44:59.531708026Z Wrote inspect data to must-gather.
2026-09-17T03:44:59.555910061Z INFO: 'previous-logs' folder not found on ip-10-0-143-180.ec2.internal, skipping...
2026-09-17T03:44:59.555946171Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-17T03:44:59.555956661Z INFO: Machine Config Daemon termination log collection complete.
2026-09-17T03:44:59.680467272Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-17T03:44:59.691026072Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-17T03:44:59.767093311Z Wrote inspect data to must-gather.
2026-09-17T03:44:59.793724073Z INFO: Getting status/config from prometheus-k8s-0
2026-09-17T03:44:59.913504028Z Wrote inspect data to must-gather.
2026-09-17T03:45:00.042725349Z INFO: INTERCONNECT MODE
2026-09-17T03:45:00.042753640Z INFO: Gathering ovn-kubernetes DBs
2026-09-17T03:45:00.125455834Z Wrote inspect data to must-gather.
2026-09-17T03:45:00.185712138Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-17T03:45:00.202792247Z INFO: Gathering OVN_Northbound from ovnkube-node-c8wtl...
2026-09-17T03:45:00.203038414Z INFO: Gathering OVN_Northbound from ovnkube-node-n4lf4...
2026-09-17T03:45:00.203333482Z INFO: Gathering OVN_Northbound from ovnkube-node-qtgbm...
2026-09-17T03:45:00.203677528Z INFO: Gathering OVN_Southbound from ovnkube-node-c8wtl...
2026-09-17T03:45:00.204024814Z INFO: Gathering OVN_Southbound from ovnkube-node-n4lf4...
2026-09-17T03:45:00.204367295Z INFO: Gathering OVN_Southbound from ovnkube-node-qtgbm...
2026-09-17T03:45:00.386771007Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-17T03:45:00.445184740Z Wrote inspect data to must-gather.
2026-09-17T03:45:00.507176860Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-17T03:45:00.570792696Z tar: Removing leading `/' from member names
2026-09-17T03:45:00.573979663Z tar: Removing leading `/' from member names
2026-09-17T03:45:00.574068350Z tar: Removing leading `/' from member names
2026-09-17T03:45:00.574709288Z tar: Removing leading `/' from member names
2026-09-17T03:45:00.582755913Z tar: Removing leading `/' from member names
2026-09-17T03:45:00.590959553Z tar: Removing leading `/' from member names
2026-09-17T03:45:00.665082558Z Wrote inspect data to /must-gather/istio.
2026-09-17T03:45:00.715533427Z Wrote inspect data to must-gather.
2026-09-17T03:45:00.771735991Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-17T03:45:00.817372112Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-17T03:45:00.918930606Z Wrote inspect data to /must-gather/istio.
2026-09-17T03:45:00.926038520Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-17T03:45:00.966094528Z Wrote inspect data to must-gather.
2026-09-17T03:45:01.124845559Z INFO: Waiting for network log collection to complete ...
2026-09-17T03:45:01.124896518Z INFO: Waiting for ovnk database copies to complete ...
2026-09-17T03:45:01.125399196Z INFO: Copying ovnk databases complete.
2026-09-17T03:45:01.127165207Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-09-17T03:45:01.129641917Z ovnk_database_store/
2026-09-17T03:45:01.129669402Z ovnk_database_store/ovnkube-node-c8wtl_nbdb
2026-09-17T03:45:01.137945634Z ovnk_database_store/ovnkube-node-c8wtl_sbdb
2026-09-17T03:45:01.144509103Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-17T03:45:01.156269884Z Wrote inspect data to /must-gather/istio.
2026-09-17T03:45:01.156464586Z ovnk_database_store/ovnkube-node-n4lf4_nbdb
2026-09-17T03:45:01.161972355Z ovnk_database_store/ovnkube-node-qtgbm_nbdb
2026-09-17T03:45:01.170168494Z ovnk_database_store/ovnkube-node-qtgbm_sbdb
2026-09-17T03:45:01.191395832Z ovnk_database_store/ovnkube-node-n4lf4_sbdb
2026-09-17T03:45:01.208029790Z Wrote inspect data to must-gather.
2026-09-17T03:45:01.210780840Z INFO: Network log collection complete.
2026-09-17T03:45:01.373328339Z Wrote inspect data to /must-gather/istio.
2026-09-17T03:45:01.378716286Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-17T03:45:01.426576854Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T03:45:01.522196201Z Wrote inspect data to must-gather.
2026-09-17T03:45:01.532070008Z INFO: Getting status from alertmanager-main-0
2026-09-17T03:45:01.590464613Z Wrote inspect data to /must-gather/istio.
2026-09-17T03:45:01.628510241Z Gathering data for ns/openshift-network-console...
2026-09-17T03:45:01.806185255Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-17T03:45:01.808291931Z INFO: Fallback to identify the container image from release info
2026-09-17T03:45:01.815430607Z Wrote inspect data to /must-gather/istio.
2026-09-17T03:45:01.817673722Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-17T03:45:02.043224330Z Wrote inspect data to /must-gather/istio.
2026-09-17T03:45:02.055167455Z Gathering data for ns/openshift-console-operator...
2026-09-17T03:45:02.228790083Z Wrote inspect data to /must-gather/istio.
2026-09-17T03:45:02.231171262Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-17T03:45:02.401988284Z Wrote inspect data to /must-gather/istio.
2026-09-17T03:45:02.436791760Z Gathering data for ns/openshift-console...
2026-09-17T03:45:02.525403492Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a
2026-09-17T03:45:02.611909528Z Wrote inspect data to /must-gather/istio.
2026-09-17T03:45:02.658377870Z daemonset.apps/perf-node-gather-daemonset created
2026-09-17T03:45:02.797601756Z Waiting for performance profile collector pods to become ready: 1
2026-09-17T03:45:02.959878184Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-17T03:45:03.311976921Z Done executing Istio gather script
2026-09-17T03:45:03.456998787Z Gathering data for ns/openshift-dns-operator...
2026-09-17T03:45:03.752499359Z Gathering data for ns/openshift-dns...
2026-09-17T03:45:03.904101804Z Waiting for performance profile collector pods to become ready: 2
2026-09-17T03:45:04.010591810Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-17T03:45:04.311653392Z Gathering data for ns/openshift-image-registry...
2026-09-17T03:45:04.788223117Z Gathering data for ns/openshift-ingress-operator...
2026-09-17T03:45:05.015490870Z Waiting for performance profile collector pods to become ready: 3
2026-09-17T03:45:05.141680327Z Gathering data for ns/openshift-ingress...
2026-09-17T03:45:05.445278857Z Gathering data for ns/openshift-ingress-canary...
2026-09-17T03:45:05.818948011Z Gathering data for ns/openshift-insights...
2026-09-17T03:45:06.139353758Z Waiting for performance profile collector pods to become ready: 4
2026-09-17T03:45:07.242312399Z Waiting for performance profile collector pods to become ready: 5
2026-09-17T03:45:07.609296902Z Gathering data for ns/openshift-jobset-operator...
2026-09-17T03:45:08.353116745Z Waiting for performance profile collector pods to become ready: 6
2026-09-17T03:45:08.385875204Z Gathering data for ns/openshift-config...
2026-09-17T03:45:08.669890097Z Gathering data for ns/openshift-config-managed...
2026-09-17T03:45:09.016193009Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-17T03:45:09.059103152Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-17T03:45:09.307122189Z Gathering data for ns/openshift-kube-apiserver...
2026-09-17T03:45:09.462577863Z Waiting for performance profile collector pods to become ready: 7
2026-09-17T03:45:09.551131547Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-17T03:45:09.857566770Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-17T03:45:10.140719399Z Gathering data for ns/openshift-kube-scheduler...
2026-09-17T03:45:10.434779495Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-17T03:45:10.573905213Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-17T03:45:10.732091827Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-17T03:45:10.834374779Z Collecting performance related data for node ip-10-0-143-180.ec2.internal
2026-09-17T03:45:10.845106071Z Collecting performance related data for node ip-10-0-139-110.ec2.internal
2026-09-17T03:45:10.856075426Z Collecting performance related data for node ip-10-0-133-213.ec2.internal
2026-09-17T03:45:11.089408105Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-17T03:45:11.474043203Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-17T03:45:12.088052318Z Gathering data for ns/openshift-multus...
2026-09-17T03:45:13.483971967Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-17T03:45:14.021280438Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-17T03:45:14.637798718Z Gathering data for ns/openshift-host-network...
2026-09-17T03:45:14.921494978Z Gathering data for ns/openshift-network-diagnostics...
2026-09-17T03:45:15.439246941Z Gathering data for ns/openshift-network-node-identity...
2026-09-17T03:45:15.859393510Z Gathering data for ns/openshift-network-operator...
2026-09-17T03:45:16.211678467Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-17T03:45:16.466455065Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-17T03:45:17.010061710Z Gathering data for ns/openshift-apiserver-operator...
2026-09-17T03:45:17.294199373Z Gathering data for ns/openshift-apiserver...
2026-09-17T03:45:17.668578699Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-17T03:45:17.907787873Z Gathering data for ns/openshift-controller-manager...
2026-09-17T03:45:18.171736335Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-17T03:45:18.821950326Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-17T03:45:19.027143030Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-17T03:45:19.089334022Z Gathering data for ns/openshift-service-ca-operator...
2026-09-17T03:45:19.225126762Z Collecting kubelet logs for node ip-10-0-143-180.ec2.internal
2026-09-17T03:45:19.226847691Z Collecting kubelet logs for node ip-10-0-139-110.ec2.internal
2026-09-17T03:45:19.228509739Z Collecting kubelet logs for node ip-10-0-133-213.ec2.internal
2026-09-17T03:45:19.429709335Z Gathering data for ns/openshift-service-ca...
2026-09-17T03:45:19.579951627Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-5n9xs namespace
2026-09-17T03:45:19.584760845Z INFO: Node performance data collection complete.
2026-09-17T03:45:19.830552086Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-17T03:45:20.691834072Z Wrote inspect data to must-gather.
2026-09-17T03:45:20.691882022Z error: inspection completed with the errors occurred while gathering data:
2026-09-17T03:45:20.691882022Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-17T03:45:20.813483344Z error: the server doesn't have a resource type "clusters"
2026-09-17T03:45:20.946844763Z Caches written to disk