2026-07-16T17:21:54.075810189Z [disk usage checker] Started
2026-07-16T17:21:54.079107715Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-07-16T17:21:54.280036997Z Gathering data for ns/openshift-cluster-version...
2026-07-16T17:21:54.436158829Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-16T17:21:54.545403996Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-16T17:21:54.633443540Z Gathering data for ns/default...
2026-07-16T17:21:54.995926637Z Gathering data for ns/openshift...
2026-07-16T17:21:55.457298825Z Gathering data for ns/kube-system...
2026-07-16T17:21:56.323149608Z Gathering data for ns/openshift-etcd...
2026-07-16T17:21:56.661490084Z Wrote inspect data to must-gather.
2026-07-16T17:21:56.661524607Z error: inspection completed with the errors occurred while gathering data:
2026-07-16T17:21:56.661524607Z     namespaces "assisted-installer" not found
2026-07-16T17:21:56.858543107Z Waiting on subprocesses to finish execution.
2026-07-16T17:21:56.869167877Z INFO: Gathering HAProxy config files
2026-07-16T17:21:56.870367229Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-16T17:21:56.874890042Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-16T17:21:56.875125664Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-16T17:21:56.877514761Z INFO: Collecting host service logs for crio
2026-07-16T17:21:56.877970954Z INFO: Collecting host service logs for kubelet
2026-07-16T17:21:56.878383428Z INFO: Collecting host service logs for rpm-ostreed
2026-07-16T17:21:56.878718896Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-16T17:21:56.879122003Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-16T17:21:56.879457176Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-16T17:21:56.879813123Z INFO: Collecting host service logs for NetworkManager
2026-07-16T17:21:56.880176560Z INFO: Collecting host service logs for openvswitch
2026-07-16T17:21:56.880518367Z INFO: Collecting host service logs for ovs-configuration
2026-07-16T17:21:56.880888047Z INFO: Collecting host service logs for ovsdb-server
2026-07-16T17:21:56.881227787Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-16T17:21:56.881782465Z INFO: Waiting for worker host service log collection to complete ...
2026-07-16T17:21:56.888712110Z Executing Istio gather script
2026-07-16T17:21:56.896859715Z INFO: Waiting for node performance related collection to complete ...
2026-07-16T17:21:57.362420424Z No resources found in openshift-etcd namespace.
2026-07-16T17:21:57.700171089Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-16T17:21:57.700253033Z INFO: on-disk MachineConfig config collection complete.
2026-07-16T17:21:57.959961307Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-16T17:21:57.997471564Z error: the server doesn't have a resource type "performanceprofile"
2026-07-16T17:21:58.065317327Z INFO: "metallb-operator" not detected. Skipping.
2026-07-16T17:21:58.120172580Z No resources found
2026-07-16T17:21:58.121155939Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-16T17:21:58.242304598Z Wrote inspect data to must-gather.
2026-07-16T17:21:58.260271959Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-16T17:21:58.334999507Z INFO: Collecting Insights Archives from insights-operator-68d9dd94cf-7pxs4
2026-07-16T17:21:58.334999507Z insights-runtime-extractor-cjkzc
2026-07-16T17:21:58.334999507Z insights-runtime-extractor-rvckl
2026-07-16T17:21:58.334999507Z insights-runtime-extractor-wzq66
2026-07-16T17:21:58.394085971Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-16T17:21:58.530067148Z ERROR: No running kube-apiserver pods found
2026-07-16T17:21:58.553882490Z INFO: Worker host service log collection to complete.
2026-07-16T17:21:58.629995399Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-16T17:21:58.776270666Z Wrote inspect data to must-gather.
2026-07-16T17:21:58.780178110Z Wrote inspect data to must-gather.
2026-07-16T17:21:59.000353202Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-16T17:21:59.151009858Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-07-16T17:21:59.218330901Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-16T17:21:59.278467850Z Wrote inspect data to must-gather.
2026-07-16T17:21:59.313570769Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-16T17:21:59.313570769Z See 'oc rsync -h' for help and examples
2026-07-16T17:21:59.350308497Z INFO: 'previous-logs' folder not found on ip-10-0-135-0.ec2.internal, skipping...
2026-07-16T17:21:59.460770886Z error: the server doesn't have a resource type "machineconfigs"
2026-07-16T17:21:59.526959799Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-16T17:21:59.588350083Z No resources found
2026-07-16T17:21:59.595092827Z Wrote inspect data to must-gather.
2026-07-16T17:21:59.662119833Z tar: Removing leading `/' from member names
2026-07-16T17:21:59.666173766Z INFO: HAProxy config collection complete.
2026-07-16T17:21:59.712234797Z Gathering data for ns/openshift-monitoring...
2026-07-16T17:21:59.831952437Z Wrote inspect data to must-gather.
2026-07-16T17:21:59.843409317Z Wrote inspect data to must-gather.
2026-07-16T17:21:59.873616965Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-16T17:21:59.959389034Z INFO: 'previous-logs' folder not found on ip-10-0-135-219.ec2.internal, skipping...
2026-07-16T17:21:59.998462882Z INFO: Getting rules from prometheus-k8s-0
2026-07-16T17:22:00.078024136Z Wrote inspect data to must-gather.
2026-07-16T17:22:00.172755678Z Wrote inspect data to must-gather.
2026-07-16T17:22:00.267149968Z INFO: 'previous-logs' folder not found on ip-10-0-140-136.ec2.internal, skipping...
2026-07-16T17:22:00.267183863Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-16T17:22:00.267217630Z INFO: Machine Config Daemon termination log collection complete.
2026-07-16T17:22:00.303556020Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-16T17:22:00.336059647Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-16T17:22:00.380732409Z Wrote inspect data to must-gather.
2026-07-16T17:22:00.514699130Z INFO: Getting status/config from prometheus-k8s-0
2026-07-16T17:22:00.565715588Z Wrote inspect data to must-gather.
2026-07-16T17:22:00.629641685Z Wrote inspect data to must-gather.
2026-07-16T17:22:00.812363426Z Wrote inspect data to must-gather.
2026-07-16T17:22:00.930184868Z INFO: INTERCONNECT MODE
2026-07-16T17:22:00.930221139Z INFO: Gathering ovn-kubernetes DBs
2026-07-16T17:22:00.953396167Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-16T17:22:01.068866919Z Wrote inspect data to must-gather.
2026-07-16T17:22:01.110926778Z INFO: Gathering OVN_Northbound from ovnkube-node-88ddw...
2026-07-16T17:22:01.111209268Z INFO: Gathering OVN_Northbound from ovnkube-node-gs2g2...
2026-07-16T17:22:01.111435461Z INFO: Gathering OVN_Northbound from ovnkube-node-wxs4c...
2026-07-16T17:22:01.111712844Z INFO: Gathering OVN_Southbound from ovnkube-node-88ddw...
2026-07-16T17:22:01.112023427Z INFO: Gathering OVN_Southbound from ovnkube-node-gs2g2...
2026-07-16T17:22:01.112608365Z INFO: Gathering OVN_Southbound from ovnkube-node-wxs4c...
2026-07-16T17:22:01.263546085Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-16T17:22:01.414816386Z tar: Removing leading `/' from member names
2026-07-16T17:22:01.423198074Z Wrote inspect data to must-gather.
2026-07-16T17:22:01.427056815Z tar: Removing leading `/' from member names
2026-07-16T17:22:01.444578903Z tar: Removing leading `/' from member names
2026-07-16T17:22:01.451757941Z tar: Removing leading `/' from member names
2026-07-16T17:22:01.459044402Z tar: Removing leading `/' from member names
2026-07-16T17:22:01.464637375Z tar: Removing leading `/' from member names
2026-07-16T17:22:01.587554933Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-16T17:22:01.657177643Z Wrote inspect data to must-gather.
2026-07-16T17:22:01.868580286Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-16T17:22:01.926574393Z Wrote inspect data to must-gather.
2026-07-16T17:22:01.932551606Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-16T17:22:01.968190906Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-16T17:22:02.009935963Z INFO: Waiting for network log collection to complete ...
2026-07-16T17:22:02.010053988Z INFO: Waiting for ovnk database copies to complete ...
2026-07-16T17:22:02.010565764Z INFO: Copying ovnk databases complete.
2026-07-16T17:22:02.012572034Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-07-16T17:22:02.015614628Z ovnk_database_store/
2026-07-16T17:22:02.015659839Z ovnk_database_store/ovnkube-node-wxs4c_sbdb
2026-07-16T17:22:02.035743442Z ovnk_database_store/ovnkube-node-88ddw_nbdb
2026-07-16T17:22:02.043071239Z ovnk_database_store/ovnkube-node-gs2g2_nbdb
2026-07-16T17:22:02.051490747Z ovnk_database_store/ovnkube-node-wxs4c_nbdb
2026-07-16T17:22:02.062005755Z ovnk_database_store/ovnkube-node-gs2g2_sbdb
2026-07-16T17:22:02.082000069Z ovnk_database_store/ovnkube-node-88ddw_sbdb
2026-07-16T17:22:02.096247013Z Wrote inspect data to /must-gather/istio.
2026-07-16T17:22:02.105094770Z INFO: Network log collection complete.
2026-07-16T17:22:02.166920435Z Wrote inspect data to must-gather.
2026-07-16T17:22:02.322399469Z Wrote inspect data to /must-gather/istio.
2026-07-16T17:22:02.327853679Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-16T17:22:02.402553408Z INFO: Getting status from alertmanager-main-0
2026-07-16T17:22:02.477783999Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-16T17:22:02.480109278Z INFO: Fallback to identify the container image from release info
2026-07-16T17:22:02.554829482Z Wrote inspect data to /must-gather/istio.
2026-07-16T17:22:02.602905985Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-16T17:22:02.759428410Z Wrote inspect data to /must-gather/istio.
2026-07-16T17:22:02.761855579Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-16T17:22:02.783214078Z Gathering data for ns/openshift-network-console...
2026-07-16T17:22:02.954042215Z Wrote inspect data to /must-gather/istio.
2026-07-16T17:22:03.007184617Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d8d5dc4ea635b13544e6f92103089219e732369ae30557b7beee2e8a11bf500d
2026-07-16T17:22:03.150494862Z daemonset.apps/perf-node-gather-daemonset created
2026-07-16T17:22:03.172127286Z Wrote inspect data to /must-gather/istio.
2026-07-16T17:22:03.176101809Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-16T17:22:03.303251170Z Waiting for performance profile collector pods to become ready: 1
2026-07-16T17:22:03.346661995Z Gathering data for ns/openshift-console-operator...
2026-07-16T17:22:03.406468237Z Wrote inspect data to /must-gather/istio.
2026-07-16T17:22:03.617783129Z Wrote inspect data to /must-gather/istio.
2026-07-16T17:22:03.622232585Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-07-16T17:22:03.817580272Z Wrote inspect data to /must-gather/istio.
2026-07-16T17:22:03.874670935Z Gathering data for ns/openshift-console...
2026-07-16T17:22:04.029241930Z Wrote inspect data to /must-gather/istio.
2026-07-16T17:22:04.031612591Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-07-16T17:22:04.156462211Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-07-16T17:22:04.189503629Z Wrote inspect data to /must-gather/istio.
2026-07-16T17:22:04.399571091Z Wrote inspect data to /must-gather/istio.
2026-07-16T17:22:04.403380645Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-16T17:22:04.458319348Z Waiting for performance profile collector pods to become ready: 2
2026-07-16T17:22:04.501260645Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-16T17:22:04.594443740Z Wrote inspect data to /must-gather/istio.
2026-07-16T17:22:04.786549431Z Wrote inspect data to /must-gather/istio.
2026-07-16T17:22:05.119552107Z Gathering data for ns/openshift-dns-operator...
2026-07-16T17:22:05.511009043Z Done executing Istio gather script
2026-07-16T17:22:05.528176371Z Gathering data for ns/openshift-dns...
2026-07-16T17:22:05.585227779Z Waiting for performance profile collector pods to become ready: 3
2026-07-16T17:22:06.341121626Z Gathering data for ns/openshift-image-registry...
2026-07-16T17:22:06.694951436Z Waiting for performance profile collector pods to become ready: 4
2026-07-16T17:22:06.927022696Z Gathering data for ns/openshift-ingress-operator...
2026-07-16T17:22:07.398232352Z Gathering data for ns/openshift-ingress...
2026-07-16T17:22:07.823044892Z Gathering data for ns/openshift-ingress-canary...
2026-07-16T17:22:07.830230571Z Waiting for performance profile collector pods to become ready: 5
2026-07-16T17:22:08.312196426Z Gathering data for ns/openshift-insights...
2026-07-16T17:22:08.965956410Z Waiting for performance profile collector pods to become ready: 6
2026-07-16T17:22:09.161065441Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-07-16T17:22:10.100554356Z Waiting for performance profile collector pods to become ready: 7
2026-07-16T17:22:10.882633076Z Gathering data for ns/kserve...
2026-07-16T17:22:11.216742281Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-16T17:22:11.502269961Z Collecting performance related data for node ip-10-0-135-219.ec2.internal
2026-07-16T17:22:11.509391108Z Collecting performance related data for node ip-10-0-135-0.ec2.internal
2026-07-16T17:22:11.517368198Z Collecting performance related data for node ip-10-0-140-136.ec2.internal
2026-07-16T17:22:12.514111692Z Gathering data for ns/openshift-config...
2026-07-16T17:22:13.203853840Z Gathering data for ns/openshift-config-managed...
2026-07-16T17:22:13.749300442Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-16T17:22:14.167079278Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-07-16T17:22:14.238485206Z Gathering data for ns/openshift-kube-apiserver...
2026-07-16T17:22:14.631962832Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-16T17:22:15.007031423Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-16T17:22:15.411035491Z Gathering data for ns/openshift-kube-scheduler...
2026-07-16T17:22:15.772421916Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-16T17:22:16.155187918Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-16T17:22:16.575861168Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-16T17:22:16.994005776Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-16T17:22:17.721551134Z Gathering data for ns/openshift-multus...
2026-07-16T17:22:19.172565185Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-07-16T17:22:19.558141351Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-16T17:22:19.732138054Z Collecting kubelet logs for node ip-10-0-135-219.ec2.internal
2026-07-16T17:22:19.733964397Z Collecting kubelet logs for node ip-10-0-135-0.ec2.internal
2026-07-16T17:22:19.736022118Z Collecting kubelet logs for node ip-10-0-140-136.ec2.internal
2026-07-16T17:22:20.072501055Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-md7tx namespace
2026-07-16T17:22:20.078293720Z INFO: Node performance data collection complete.
2026-07-16T17:22:21.210713005Z Gathering data for ns/openshift-host-network...
2026-07-16T17:22:21.570727835Z Gathering data for ns/openshift-network-diagnostics...
2026-07-16T17:22:22.178109671Z Gathering data for ns/openshift-network-node-identity...
2026-07-16T17:22:22.740270486Z Gathering data for ns/openshift-network-operator...
2026-07-16T17:22:23.228586102Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-16T17:22:23.594846474Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-16T17:22:24.150525321Z Gathering data for ns/openshift-apiserver-operator...
2026-07-16T17:22:24.179867162Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-07-16T17:22:24.545393479Z Gathering data for ns/openshift-apiserver...
2026-07-16T17:22:25.005342486Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-16T17:22:25.343814241Z Gathering data for ns/openshift-controller-manager...
2026-07-16T17:22:25.737096065Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-16T17:22:26.466794295Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-16T17:22:26.838686323Z Gathering data for ns/openshift-service-ca-operator...
2026-07-16T17:22:27.214161434Z Gathering data for ns/openshift-service-ca...
2026-07-16T17:22:27.663930090Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-16T17:22:28.687069762Z Wrote inspect data to must-gather.
2026-07-16T17:22:28.687120953Z error: inspection completed with the errors occurred while gathering data:
2026-07-16T17:22:28.687120953Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-16T17:22:28.819524593Z error: the server doesn't have a resource type "clusters"
2026-07-16T17:22:28.984910715Z Caches written to disk