2026-07-07T17:45:06.042717074Z [disk usage checker] Started 2026-07-07T17:45:06.046099277Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-07T17:45:06.228560023Z Gathering data for ns/openshift-cluster-version... 2026-07-07T17:45:06.358934311Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-07T17:45:06.446357209Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-07T17:45:06.528068632Z Gathering data for ns/default... 2026-07-07T17:45:06.814588388Z Gathering data for ns/openshift... 2026-07-07T17:45:07.197848541Z Gathering data for ns/kube-system... 2026-07-07T17:45:07.895683230Z Gathering data for ns/openshift-etcd... 2026-07-07T17:45:08.189056084Z Wrote inspect data to must-gather. 2026-07-07T17:45:08.189090876Z error: inspection completed with the errors occurred while gathering data: 2026-07-07T17:45:08.189090876Z namespaces "assisted-installer" not found 2026-07-07T17:45:08.518085301Z Waiting on subprocesses to finish execution. 2026-07-07T17:45:08.523694800Z Executing Istio gather script 2026-07-07T17:45:08.523886219Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-07-07T17:45:08.531746113Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-07-07T17:45:08.543072994Z INFO: Gathering machine config daemon's old logs from all nodes 2026-07-07T17:45:08.551303728Z INFO: Gathering HAProxy config files 2026-07-07T17:45:08.561991275Z INFO: Collecting host service logs for crio 2026-07-07T17:45:08.562326080Z INFO: Collecting host service logs for kubelet 2026-07-07T17:45:08.562660966Z INFO: Collecting host service logs for rpm-ostreed 2026-07-07T17:45:08.563006017Z INFO: Collecting host service logs for ostree-finalize-staged 2026-07-07T17:45:08.563337770Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-07-07T17:45:08.563651791Z INFO: Collecting host service logs for machine-config-daemon-host 2026-07-07T17:45:08.564000105Z INFO: Collecting host service logs for NetworkManager 2026-07-07T17:45:08.564332450Z INFO: Collecting host service logs for openvswitch 2026-07-07T17:45:08.564652707Z INFO: Collecting host service logs for ovs-configuration 2026-07-07T17:45:08.565026177Z INFO: Collecting host service logs for ovsdb-server 2026-07-07T17:45:08.565359728Z INFO: Collecting host service logs for ovs-vswitchd 2026-07-07T17:45:08.565888643Z INFO: Waiting for worker host service log collection to complete ... 2026-07-07T17:45:08.595959266Z INFO: Waiting for node performance related collection to complete ... 2026-07-07T17:45:08.790566565Z INFO: Collecting Insights Archives from insights-operator-5c847dcfc4-lhg67 2026-07-07T17:45:08.790633468Z insights-runtime-extractor-2d5hd 2026-07-07T17:45:08.790656623Z insights-runtime-extractor-skqv4 2026-07-07T17:45:08.790676552Z insights-runtime-extractor-sqhxh 2026-07-07T17:45:09.240339121Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-07-07T17:45:09.254485214Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-07-07T17:45:09.254573346Z INFO: on-disk MachineConfig config collection complete. 2026-07-07T17:45:09.421220904Z INFO: "metallb-operator" not detected. Skipping. 2026-07-07T17:45:09.836667290Z Wrote inspect data to must-gather. 2026-07-07T17:45:09.939198152Z INFO: "sriov-network-operator" not detected. Skipping. 2026-07-07T17:45:09.951829094Z ERROR: No running kube-apiserver pods found 2026-07-07T17:45:09.963782518Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-07-07T17:45:09.966749306Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-07-07T17:45:09.966749306Z See 'oc rsync -h' for help and examples 2026-07-07T17:45:10.031395121Z error: the server doesn't have a resource type "performanceprofile" 2026-07-07T17:45:10.156875227Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-07T17:45:10.242482210Z No resources found in openshift-etcd namespace. 2026-07-07T17:45:10.291496977Z Wrote inspect data to must-gather. 2026-07-07T17:45:10.301112638Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-07-07T17:45:10.453462149Z Gathering data for ns/openshift-monitoring... 2026-07-07T17:45:10.486056080Z Wrote inspect data to must-gather. 2026-07-07T17:45:10.548729340Z INFO: Found 1 replicas - prometheus-k8s-0 2026-07-07T17:45:10.554498671Z No resources found 2026-07-07T17:45:10.591862206Z INFO: Waiting for HAProxy config collection to complete ... 2026-07-07T17:45:10.639628702Z Wrote inspect data to must-gather. 2026-07-07T17:45:10.690447876Z Wrote inspect data to must-gather. 2026-07-07T17:45:10.692226522Z INFO: Worker host service log collection to complete. 2026-07-07T17:45:10.751646946Z No resources found 2026-07-07T17:45:10.906127178Z tar: Removing leading `/' from member names 2026-07-07T17:45:10.910541722Z INFO: HAProxy config collection complete. 2026-07-07T17:45:10.928377668Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-07-07T17:45:10.942051561Z Wrote inspect data to must-gather. 2026-07-07T17:45:10.966341673Z INFO: 'previous-logs' folder not found on ip-10-0-132-144.ec2.internal, skipping... 2026-07-07T17:45:10.975880244Z Wrote inspect data to must-gather. 2026-07-07T17:45:10.979891025Z Wrote inspect data to must-gather. 2026-07-07T17:45:11.055186172Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-07T17:45:11.128659357Z error: the server doesn't have a resource type "machineconfigs" 2026-07-07T17:45:11.259160477Z INFO: 'previous-logs' folder not found on ip-10-0-134-158.ec2.internal, skipping... 2026-07-07T17:45:11.266941703Z Wrote inspect data to must-gather. 2026-07-07T17:45:11.319704770Z error: the server doesn't have a resource type "machineconfigpools" 2026-07-07T17:45:11.418517981Z INFO: Getting rules from prometheus-k8s-0 2026-07-07T17:45:11.575222890Z Wrote inspect data to must-gather. 2026-07-07T17:45:11.584623119Z Wrote inspect data to must-gather. 2026-07-07T17:45:11.619283465Z INFO: 'previous-logs' folder not found on ip-10-0-137-233.ec2.internal, skipping... 2026-07-07T17:45:11.619318294Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-07-07T17:45:11.619367094Z INFO: Machine Config Daemon termination log collection complete. 2026-07-07T17:45:11.678737366Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-07-07T17:45:11.725649410Z error: the server doesn't have a resource type "kubeletconfigs" 2026-07-07T17:45:11.826494874Z INFO: Getting status/config from prometheus-k8s-0 2026-07-07T17:45:11.856461547Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-07-07T17:45:11.868997557Z INFO: INTERCONNECT MODE 2026-07-07T17:45:11.869031765Z INFO: Gathering ovn-kubernetes DBs 2026-07-07T17:45:11.954705390Z Wrote inspect data to must-gather. 2026-07-07T17:45:12.045162977Z INFO: Gathering OVN_Northbound from ovnkube-node-flhvx... 2026-07-07T17:45:12.045527515Z INFO: Gathering OVN_Northbound from ovnkube-node-l9cl2... 2026-07-07T17:45:12.045881915Z INFO: Gathering OVN_Northbound from ovnkube-node-n6tkx... 2026-07-07T17:45:12.046221860Z INFO: Gathering OVN_Southbound from ovnkube-node-flhvx... 2026-07-07T17:45:12.046489706Z INFO: Gathering OVN_Southbound from ovnkube-node-l9cl2... 2026-07-07T17:45:12.046736248Z INFO: Gathering OVN_Southbound from ovnkube-node-n6tkx... 2026-07-07T17:45:12.056513918Z Wrote inspect data to /must-gather/istio. 2026-07-07T17:45:12.182004986Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-07T17:45:12.321671022Z Wrote inspect data to must-gather. 2026-07-07T17:45:12.372047270Z tar: Removing leading `/' from member names 2026-07-07T17:45:12.406698558Z tar: Removing leading `/' from member names 2026-07-07T17:45:12.410098969Z tar: Removing leading `/' from member names 2026-07-07T17:45:12.414851520Z tar: Removing leading `/' from member names 2026-07-07T17:45:12.420573964Z tar: Removing leading `/' from member names 2026-07-07T17:45:12.421419439Z tar: Removing leading `/' from member names 2026-07-07T17:45:12.434500732Z INFO: Getting status/flags from prometheus-k8s-0 2026-07-07T17:45:12.476901118Z Wrote inspect data to /must-gather/istio. 2026-07-07T17:45:12.480665232Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-07-07T17:45:12.582623191Z Wrote inspect data to must-gather. 2026-07-07T17:45:12.719921238Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-07-07T17:45:12.725622492Z Wrote inspect data to /must-gather/istio. 2026-07-07T17:45:12.815566843Z W0707 17:45:12.815514 1421 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-07-07T17:45:12.823017125Z Wrote inspect data to must-gather. 2026-07-07T17:45:12.867305845Z INFO: Waiting for network log collection to complete ... 2026-07-07T17:45:12.867345898Z INFO: Waiting for ovnk database copies to complete ... 2026-07-07T17:45:12.867750949Z INFO: Copying ovnk databases complete. 2026-07-07T17:45:12.869328940Z 2.6M must-gather/network_logs/ovnk_database_store 2026-07-07T17:45:12.871455228Z ovnk_database_store/ 2026-07-07T17:45:12.871455228Z ovnk_database_store/ovnkube-node-l9cl2_nbdb 2026-07-07T17:45:12.878845611Z ovnk_database_store/ovnkube-node-flhvx_nbdb 2026-07-07T17:45:12.886840648Z ovnk_database_store/ovnkube-node-n6tkx_sbdb 2026-07-07T17:45:12.904209027Z ovnk_database_store/ovnkube-node-n6tkx_nbdb 2026-07-07T17:45:12.911509351Z ovnk_database_store/ovnkube-node-flhvx_sbdb 2026-07-07T17:45:12.928188193Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-07T17:45:12.928783738Z ovnk_database_store/ovnkube-node-l9cl2_sbdb 2026-07-07T17:45:12.951212001Z INFO: Network log collection complete. 2026-07-07T17:45:12.959071982Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-07-07T17:45:12.964564239Z Wrote inspect data to /must-gather/istio. 2026-07-07T17:45:12.968263502Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-07-07T17:45:13.126985221Z Gathering data for ns/openshift-network-console... 2026-07-07T17:45:13.148996532Z Wrote inspect data to must-gather. 2026-07-07T17:45:13.168040176Z Wrote inspect data to /must-gather/istio. 2026-07-07T17:45:13.200488671Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-07-07T17:45:13.377383742Z Wrote inspect data to /must-gather/istio. 2026-07-07T17:45:13.382813686Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-07-07T17:45:13.460882460Z Wrote inspect data to must-gather. 2026-07-07T17:45:13.592159647Z INFO: Getting status from alertmanager-main-0 2026-07-07T17:45:13.610994940Z Gathering data for ns/openshift-console-operator... 2026-07-07T17:45:13.621063399Z Wrote inspect data to /must-gather/istio. 2026-07-07T17:45:13.785851699Z Wrote inspect data to must-gather. 2026-07-07T17:45:13.836934106Z Wrote inspect data to /must-gather/istio. 2026-07-07T17:45:13.839271498Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-07-07T17:45:14.015639215Z Wrote inspect data to /must-gather/istio. 2026-07-07T17:45:14.036264998Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-07-07T17:45:14.039392869Z INFO: Fallback to identify the container image from release info 2026-07-07T17:45:14.041189329Z Gathering data for ns/openshift-console... 2026-07-07T17:45:14.203701327Z Wrote inspect data to /must-gather/istio. 2026-07-07T17:45:14.571011076Z Gathering data for ns/openshift-cluster-storage-operator... 2026-07-07T17:45:14.589089324Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5be8cdcf34f35a89a70b6bfa49e66897641871237570b3e09766f83a0a473e6a 2026-07-07T17:45:14.718179999Z daemonset.apps/perf-node-gather-daemonset created 2026-07-07T17:45:14.848493463Z Waiting for performance profile collector pods to become ready: 1 2026-07-07T17:45:14.870399187Z Done executing Istio gather script 2026-07-07T17:45:15.073144278Z Gathering data for ns/openshift-dns-operator... 2026-07-07T17:45:15.364154835Z Gathering data for ns/openshift-dns... 2026-07-07T17:45:15.960716692Z Gathering data for ns/openshift-image-registry... 2026-07-07T17:45:15.967112917Z Waiting for performance profile collector pods to become ready: 2 2026-07-07T17:45:16.059875613Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-07T17:45:16.397680418Z Gathering data for ns/openshift-ingress-operator... 2026-07-07T17:45:16.704596464Z Gathering data for ns/openshift-ingress... 2026-07-07T17:45:17.012747839Z Gathering data for ns/openshift-ingress-canary... 2026-07-07T17:45:17.072357509Z Waiting for performance profile collector pods to become ready: 3 2026-07-07T17:45:17.398063888Z Gathering data for ns/openshift-insights... 2026-07-07T17:45:18.183725813Z Waiting for performance profile collector pods to become ready: 4 2026-07-07T17:45:19.185618703Z Gathering data for ns/openshift-config... 2026-07-07T17:45:19.300814832Z Waiting for performance profile collector pods to become ready: 5 2026-07-07T17:45:19.447630367Z Gathering data for ns/openshift-config-managed... 2026-07-07T17:45:19.830083811Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-07-07T17:45:20.079355391Z Gathering data for ns/openshift-kube-apiserver... 2026-07-07T17:45:20.335059119Z Gathering data for ns/openshift-kube-controller-manager... 2026-07-07T17:45:20.419597098Z Waiting for performance profile collector pods to become ready: 6 2026-07-07T17:45:20.608145694Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-07-07T17:45:20.871153527Z Gathering data for ns/openshift-kube-scheduler... 2026-07-07T17:45:21.064844346Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-07T17:45:21.114060360Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-07-07T17:45:21.390249804Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-07-07T17:45:21.535613659Z Waiting for performance profile collector pods to become ready: 7 2026-07-07T17:45:21.720574983Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-07-07T17:45:22.039566160Z Gathering data for ns/openshift-user-workload-monitoring... 2026-07-07T17:45:22.495413979Z Gathering data for ns/openshift-multus... 2026-07-07T17:45:22.632652367Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-07-07T17:45:22.837942033Z Collecting performance related data for node ip-10-0-132-144.ec2.internal 2026-07-07T17:45:22.844722336Z Collecting performance related data for node ip-10-0-134-158.ec2.internal 2026-07-07T17:45:22.851725068Z Collecting performance related data for node ip-10-0-137-233.ec2.internal 2026-07-07T17:45:24.029586850Z Gathering data for ns/openshift-ovn-kubernetes... 2026-07-07T17:45:25.317426094Z Gathering data for ns/openshift-host-network... 2026-07-07T17:45:25.675892156Z Gathering data for ns/openshift-network-diagnostics... 2026-07-07T17:45:26.070063066Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-07T17:45:26.180493140Z Gathering data for ns/openshift-network-node-identity... 2026-07-07T17:45:26.560692949Z Gathering data for ns/openshift-network-operator... 2026-07-07T17:45:26.934092087Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-07-07T17:45:27.208269182Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-07-07T17:45:27.705197220Z Gathering data for ns/openshift-apiserver-operator... 2026-07-07T17:45:28.024720305Z Gathering data for ns/openshift-apiserver... 2026-07-07T17:45:28.403640307Z Gathering data for ns/openshift-controller-manager-operator... 2026-07-07T17:45:28.671110951Z Gathering data for ns/openshift-controller-manager... 2026-07-07T17:45:28.925372905Z Gathering data for ns/openshift-cluster-samples-operator... 2026-07-07T17:45:29.548825843Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-07-07T17:45:29.900385283Z Gathering data for ns/openshift-service-ca-operator... 2026-07-07T17:45:30.255474840Z Gathering data for ns/openshift-service-ca... 2026-07-07T17:45:30.582876562Z Collecting kubelet logs for node ip-10-0-132-144.ec2.internal 2026-07-07T17:45:30.584504884Z Collecting kubelet logs for node ip-10-0-134-158.ec2.internal 2026-07-07T17:45:30.586042630Z Collecting kubelet logs for node ip-10-0-137-233.ec2.internal 2026-07-07T17:45:30.655046986Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-07-07T17:45:30.889950294Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-qjxvr namespace 2026-07-07T17:45:30.894775771Z INFO: Node performance data collection complete. 2026-07-07T17:45:31.074870754Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-07T17:45:31.560492288Z Wrote inspect data to must-gather. 2026-07-07T17:45:31.560537198Z error: inspection completed with the errors occurred while gathering data: 2026-07-07T17:45:31.560537198Z skipping gathering secrets/support due to error: secrets "support" not found 2026-07-07T17:45:31.682566157Z error: the server doesn't have a resource type "clusters" 2026-07-07T17:45:31.806088362Z Caches written to disk