2026-09-17T10:00:25.389554353Z [disk usage checker] Started 2026-09-17T10:00:25.392916069Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-09-17T10:00:25.583385562Z Gathering data for ns/openshift-cluster-version... 2026-09-17T10:00:25.712981160Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-17T10:00:25.802657973Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-17T10:00:25.888152479Z Gathering data for ns/default... 2026-09-17T10:00:26.170836894Z Gathering data for ns/openshift... 2026-09-17T10:00:26.596674175Z Gathering data for ns/kube-system... 2026-09-17T10:00:27.275434153Z Gathering data for ns/openshift-etcd... 2026-09-17T10:00:27.585617944Z Wrote inspect data to must-gather. 2026-09-17T10:00:27.585660538Z error: inspection completed with the errors occurred while gathering data: 2026-09-17T10:00:27.585660538Z namespaces "assisted-installer" not found 2026-09-17T10:00:28.033755490Z Waiting on subprocesses to finish execution. 2026-09-17T10:00:28.040307985Z INFO: Gathering HAProxy config files 2026-09-17T10:00:28.046028791Z INFO: Collecting host service logs for crio 2026-09-17T10:00:28.046407615Z INFO: Collecting host service logs for kubelet 2026-09-17T10:00:28.047233148Z INFO: Collecting host service logs for rpm-ostreed 2026-09-17T10:00:28.047585704Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-17T10:00:28.047960333Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-17T10:00:28.048301683Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-17T10:00:28.048652920Z INFO: Collecting host service logs for NetworkManager 2026-09-17T10:00:28.049010441Z INFO: Collecting host service logs for openvswitch 2026-09-17T10:00:28.049252421Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-17T10:00:28.049352223Z INFO: Collecting host service logs for ovs-configuration 2026-09-17T10:00:28.049680583Z INFO: Collecting host service logs for ovsdb-server 2026-09-17T10:00:28.050094535Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-17T10:00:28.050446535Z INFO: Waiting for worker host service log collection to complete ... 2026-09-17T10:00:28.062422181Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-17T10:00:28.083022509Z Executing Istio gather script 2026-09-17T10:00:28.319370617Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-17T10:00:28.458315043Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-17T10:00:28.485926809Z No resources found 2026-09-17T10:00:28.516470476Z INFO: Waiting for node performance related collection to complete ... 2026-09-17T10:00:29.052631131Z Wrote inspect data to must-gather. 2026-09-17T10:00:29.150548558Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-17T10:00:29.216370919Z INFO: "metallb-operator" not detected. Skipping. 2026-09-17T10:00:29.266175197Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-17T10:00:29.267171698Z INFO: on-disk MachineConfig config collection complete. 2026-09-17T10:00:29.301954070Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-17T10:00:29.349321913Z INFO: Collecting Insights Archives from insights-operator-5844b856db-vfj4n 2026-09-17T10:00:29.349381411Z insights-runtime-extractor-4jwxr 2026-09-17T10:00:29.349400266Z insights-runtime-extractor-h8fs2 2026-09-17T10:00:29.349430784Z insights-runtime-extractor-nq7fh 2026-09-17T10:00:29.351254769Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-17T10:00:29.396795081Z No resources found in openshift-etcd namespace. 2026-09-17T10:00:29.563230855Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-17T10:00:29.577734218Z ERROR: No running kube-apiserver pods found 2026-09-17T10:00:29.597467454Z INFO: Worker host service log collection to complete. 2026-09-17T10:00:29.636756533Z error: the server doesn't have a resource type "performanceprofile" 2026-09-17T10:00:29.935984857Z Wrote inspect data to must-gather. 2026-09-17T10:00:30.023460865Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-17T10:00:30.269859086Z No resources found 2026-09-17T10:00:30.390285674Z Wrote inspect data to must-gather. 2026-09-17T10:00:30.422921397Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-17T10:00:30.444065258Z Wrote inspect data to must-gather. 2026-09-17T10:00:30.458474187Z Wrote inspect data to must-gather. 2026-09-17T10:00:30.472177413Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-09-17T10:00:30.498185223Z Gathering data for ns/openshift-monitoring... 2026-09-17T10:00:30.513280885Z Wrote inspect data to must-gather. 2026-09-17T10:00:30.516583208Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-17T10:00:30.517444266Z Wrote inspect data to must-gather. 2026-09-17T10:00:30.525706398Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-17T10:00:30.525706398Z See 'oc rsync -h' for help and examples 2026-09-17T10:00:30.575299053Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-17T10:00:30.671443163Z tar: Removing leading `/' from member names 2026-09-17T10:00:30.681126865Z INFO: HAProxy config collection complete. 2026-09-17T10:00:30.717260232Z error: the server doesn't have a resource type "machineconfigs" 2026-09-17T10:00:30.749417708Z INFO: 'previous-logs' folder not found on ip-10-0-135-161.ec2.internal, skipping... 2026-09-17T10:00:30.837938002Z Wrote inspect data to must-gather. 2026-09-17T10:00:30.896956296Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-17T10:00:31.022588943Z INFO: Getting rules from prometheus-k8s-0 2026-09-17T10:00:31.040436875Z INFO: 'previous-logs' folder not found on ip-10-0-137-238.ec2.internal, skipping... 2026-09-17T10:00:31.116132559Z Wrote inspect data to must-gather. 2026-09-17T10:00:31.184249795Z Wrote inspect data to must-gather. 2026-09-17T10:00:31.312891705Z INFO: 'previous-logs' folder not found on ip-10-0-138-7.ec2.internal, skipping... 2026-09-17T10:00:31.312999884Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-17T10:00:31.313050484Z INFO: Machine Config Daemon termination log collection complete. 2026-09-17T10:00:31.364079226Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-17T10:00:31.409532306Z Wrote inspect data to must-gather. 2026-09-17T10:00:31.478225608Z INFO: Getting status/config from prometheus-k8s-0 2026-09-17T10:00:31.569592844Z Wrote inspect data to must-gather. 2026-09-17T10:00:31.690844677Z INFO: INTERCONNECT MODE 2026-09-17T10:00:31.690955004Z INFO: Gathering ovn-kubernetes DBs 2026-09-17T10:00:31.707026727Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-17T10:00:31.784162029Z Wrote inspect data to must-gather. 2026-09-17T10:00:31.852238863Z INFO: Gathering OVN_Northbound from ovnkube-node-lgdft... 2026-09-17T10:00:31.852534172Z INFO: Gathering OVN_Northbound from ovnkube-node-qwln7... 2026-09-17T10:00:31.852793468Z INFO: Gathering OVN_Northbound from ovnkube-node-xfx6s... 2026-09-17T10:00:31.853060644Z INFO: Gathering OVN_Southbound from ovnkube-node-lgdft... 2026-09-17T10:00:31.853308816Z INFO: Gathering OVN_Southbound from ovnkube-node-qwln7... 2026-09-17T10:00:31.853592934Z INFO: Gathering OVN_Southbound from ovnkube-node-xfx6s... 2026-09-17T10:00:31.918751386Z Wrote inspect data to /must-gather/istio. 2026-09-17T10:00:31.967146277Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-17T10:00:32.144168443Z Wrote inspect data to must-gather. 2026-09-17T10:00:32.209093346Z tar: Removing leading `/' from member names 2026-09-17T10:00:32.219943875Z tar: Removing leading `/' from member names 2026-09-17T10:00:32.238637481Z tar: Removing leading `/' from member names 2026-09-17T10:00:32.245474723Z tar: Removing leading `/' from member names 2026-09-17T10:00:32.266954624Z tar: Removing leading `/' from member names 2026-09-17T10:00:32.277571880Z tar: Removing leading `/' from member names 2026-09-17T10:00:32.305971804Z Wrote inspect data to /must-gather/istio. 2026-09-17T10:00:32.310259409Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-17T10:00:32.357361949Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-17T10:00:32.424667642Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-17T10:00:32.430571779Z Wrote inspect data to must-gather. 2026-09-17T10:00:32.544165710Z Wrote inspect data to /must-gather/istio. 2026-09-17T10:00:32.654115423Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-17T10:00:32.702086825Z Wrote inspect data to must-gather. 2026-09-17T10:00:32.758231285Z INFO: Waiting for network log collection to complete ... 2026-09-17T10:00:32.758332914Z INFO: Waiting for ovnk database copies to complete ... 2026-09-17T10:00:32.758748888Z INFO: Copying ovnk databases complete. 2026-09-17T10:00:32.760894235Z 3.6M must-gather/network_logs/ovnk_database_store 2026-09-17T10:00:32.765022526Z ovnk_database_store/ 2026-09-17T10:00:32.765022526Z ovnk_database_store/ovnkube-node-xfx6s_sbdb 2026-09-17T10:00:32.775710720Z Wrote inspect data to /must-gather/istio. 2026-09-17T10:00:32.778759823Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-17T10:00:32.791763044Z ovnk_database_store/ovnkube-node-lgdft_nbdb 2026-09-17T10:00:32.798276549Z ovnk_database_store/ovnkube-node-xfx6s_nbdb 2026-09-17T10:00:32.807104614Z ovnk_database_store/ovnkube-node-lgdft_sbdb 2026-09-17T10:00:32.830076998Z ovnk_database_store/ovnkube-node-qwln7_nbdb 2026-09-17T10:00:32.837332378Z ovnk_database_store/ovnkube-node-qwln7_sbdb 2026-09-17T10:00:32.862525077Z INFO: Network log collection complete. 2026-09-17T10:00:32.924212496Z Wrote inspect data to must-gather. 2026-09-17T10:00:32.955545061Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-17T10:00:32.993941934Z Wrote inspect data to /must-gather/istio. 2026-09-17T10:00:33.104615653Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-17T10:00:33.212485324Z Wrote inspect data to /must-gather/istio. 2026-09-17T10:00:33.215183811Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-17T10:00:33.244186939Z Wrote inspect data to must-gather. 2026-09-17T10:00:33.337469981Z Gathering data for ns/openshift-network-console... 2026-09-17T10:00:33.383454638Z INFO: Getting status from alertmanager-main-0 2026-09-17T10:00:33.481746245Z Wrote inspect data to /must-gather/istio. 2026-09-17T10:00:33.530557444Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-17T10:00:33.532634605Z INFO: Fallback to identify the container image from release info 2026-09-17T10:00:33.688437690Z Wrote inspect data to /must-gather/istio. 2026-09-17T10:00:33.691964709Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-17T10:00:33.835422955Z Gathering data for ns/openshift-console-operator... 2026-09-17T10:00:33.854785122Z Wrote inspect data to /must-gather/istio. 2026-09-17T10:00:34.034586773Z Wrote inspect data to /must-gather/istio. 2026-09-17T10:00:34.239355869Z Gathering data for ns/openshift-console... 2026-09-17T10:00:34.687574143Z Done executing Istio gather script 2026-09-17T10:00:34.750889842Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-17T10:00:34.918801940Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a 2026-09-17T10:00:35.039904704Z daemonset.apps/perf-node-gather-daemonset created 2026-09-17T10:00:35.175649438Z Waiting for performance profile collector pods to become ready: 1 2026-09-17T10:00:35.276757445Z Gathering data for ns/openshift-dns-operator... 2026-09-17T10:00:35.504658643Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-09-17T10:00:35.602125499Z Gathering data for ns/openshift-dns... 2026-09-17T10:00:36.301233347Z Waiting for performance profile collector pods to become ready: 2 2026-09-17T10:00:36.349077418Z Gathering data for ns/openshift-image-registry... 2026-09-17T10:00:36.830717399Z Gathering data for ns/openshift-ingress-operator... 2026-09-17T10:00:37.196344798Z Gathering data for ns/openshift-ingress... 2026-09-17T10:00:37.405281402Z Waiting for performance profile collector pods to become ready: 3 2026-09-17T10:00:37.630563648Z Gathering data for ns/openshift-ingress-canary... 2026-09-17T10:00:38.129673751Z Gathering data for ns/openshift-insights... 2026-09-17T10:00:38.528785630Z Waiting for performance profile collector pods to become ready: 4 2026-09-17T10:00:39.637689946Z Waiting for performance profile collector pods to become ready: 5 2026-09-17T10:00:40.316050436Z Gathering data for ns/openshift-config... 2026-09-17T10:00:40.510516726Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-09-17T10:00:40.562083374Z Gathering data for ns/openshift-config-managed... 2026-09-17T10:00:40.754376342Z Waiting for performance profile collector pods to become ready: 6 2026-09-17T10:00:40.978065243Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-17T10:00:41.226959602Z Gathering data for ns/openshift-kube-apiserver... 2026-09-17T10:00:41.479203994Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-17T10:00:41.755064937Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-17T10:00:41.880970805Z Waiting for performance profile collector pods to become ready: 7 2026-09-17T10:00:42.039865081Z Gathering data for ns/openshift-kube-scheduler... 2026-09-17T10:00:42.287984132Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-17T10:00:42.559355001Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-17T10:00:42.853950339Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-17T10:00:43.003796234Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-17T10:00:43.181710936Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-17T10:00:43.263855082Z Collecting performance related data for node ip-10-0-135-161.ec2.internal 2026-09-17T10:00:43.270878590Z Collecting performance related data for node ip-10-0-138-7.ec2.internal 2026-09-17T10:00:43.278238958Z Collecting performance related data for node ip-10-0-137-238.ec2.internal 2026-09-17T10:00:43.797575621Z Gathering data for ns/openshift-multus... 2026-09-17T10:00:45.320005078Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-17T10:00:45.516010824Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-09-17T10:00:46.575090745Z Gathering data for ns/openshift-host-network... 2026-09-17T10:00:46.889744553Z Gathering data for ns/openshift-network-diagnostics... 2026-09-17T10:00:47.449076733Z Gathering data for ns/openshift-network-node-identity... 2026-09-17T10:00:47.872341845Z Gathering data for ns/openshift-network-operator... 2026-09-17T10:00:48.262660881Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-17T10:00:48.524456059Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-17T10:00:49.001530884Z Gathering data for ns/openshift-apiserver-operator... 2026-09-17T10:00:49.272015459Z Gathering data for ns/openshift-apiserver... 2026-09-17T10:00:49.690871757Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-17T10:00:49.940685788Z Gathering data for ns/openshift-controller-manager... 2026-09-17T10:00:50.201815423Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-17T10:00:50.521706418Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-09-17T10:00:50.880569858Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-17T10:00:51.187911676Z Gathering data for ns/openshift-service-ca-operator... 2026-09-17T10:00:51.504298369Z Gathering data for ns/openshift-service-ca... 2026-09-17T10:00:51.800237934Z Collecting kubelet logs for node ip-10-0-135-161.ec2.internal 2026-09-17T10:00:51.802407763Z Collecting kubelet logs for node ip-10-0-138-7.ec2.internal 2026-09-17T10:00:51.804587311Z Collecting kubelet logs for node ip-10-0-137-238.ec2.internal 2026-09-17T10:00:51.915897863Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-17T10:00:52.139553555Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-5v8j2 namespace 2026-09-17T10:00:52.144926365Z INFO: Node performance data collection complete. 2026-09-17T10:00:52.859040261Z Wrote inspect data to must-gather. 2026-09-17T10:00:52.859086810Z error: inspection completed with the errors occurred while gathering data: 2026-09-17T10:00:52.859086810Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-17T10:00:52.987324228Z error: the server doesn't have a resource type "clusters" 2026-09-17T10:00:53.119753601Z Caches written to disk