2026-06-08T11:52:56.508376409Z [disk usage checker] Started 2026-06-08T11:52:56.512458082Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-06-08T11:52:56.706452322Z Gathering data for ns/openshift-cluster-version... 2026-06-08T11:52:56.833540014Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-06-08T11:52:56.914288868Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-06-08T11:52:56.993004094Z Gathering data for ns/default... 2026-06-08T11:52:57.271467432Z Gathering data for ns/openshift... 2026-06-08T11:52:57.651068342Z Gathering data for ns/kube-system... 2026-06-08T11:52:58.310346461Z Gathering data for ns/openshift-etcd... 2026-06-08T11:52:58.606103692Z Wrote inspect data to must-gather. 2026-06-08T11:52:58.606135529Z error: inspection completed with the errors occurred while gathering data: 2026-06-08T11:52:58.606135529Z namespaces "assisted-installer" not found 2026-06-08T11:52:59.113411961Z Waiting on subprocesses to finish execution. 2026-06-08T11:52:59.122475923Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-06-08T11:52:59.122779159Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-06-08T11:52:59.123025921Z INFO: Gathering machine config daemon's old logs from all nodes 2026-06-08T11:52:59.123203753Z INFO: Gathering HAProxy config files 2026-06-08T11:52:59.124436608Z Executing Istio gather script 2026-06-08T11:52:59.161478358Z INFO: Collecting host service logs for crio 2026-06-08T11:52:59.161767234Z INFO: Collecting host service logs for kubelet 2026-06-08T11:52:59.162124682Z INFO: Collecting host service logs for rpm-ostreed 2026-06-08T11:52:59.162485431Z INFO: Collecting host service logs for ostree-finalize-staged 2026-06-08T11:52:59.163016705Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-06-08T11:52:59.163456242Z INFO: Collecting host service logs for machine-config-daemon-host 2026-06-08T11:52:59.163801278Z INFO: Collecting host service logs for NetworkManager 2026-06-08T11:52:59.164148767Z INFO: Collecting host service logs for openvswitch 2026-06-08T11:52:59.164505023Z INFO: Collecting host service logs for ovs-configuration 2026-06-08T11:52:59.164864329Z INFO: Collecting host service logs for ovsdb-server 2026-06-08T11:52:59.165310484Z INFO: Collecting host service logs for ovs-vswitchd 2026-06-08T11:52:59.165766447Z INFO: Waiting for worker host service log collection to complete ... 2026-06-08T11:52:59.377552390Z INFO: Waiting for node performance related collection to complete ... 2026-06-08T11:52:59.659900080Z INFO: Collecting Insights Archives from insights-operator-896549976-8l62d 2026-06-08T11:52:59.659957052Z insights-runtime-extractor-h2fhx 2026-06-08T11:52:59.659976444Z insights-runtime-extractor-qnqgr 2026-06-08T11:52:59.659995218Z insights-runtime-extractor-tsfh6 2026-06-08T11:52:59.706337015Z INFO: "sriov-network-operator" not detected. Skipping. 2026-06-08T11:52:59.802433488Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-06-08T11:52:59.852870909Z No resources found 2026-06-08T11:52:59.867791187Z No resources found in openshift-etcd namespace. 2026-06-08T11:52:59.875449972Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-06-08T11:52:59.982440864Z Wrote inspect data to must-gather. 2026-06-08T11:53:00.216629396Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-06-08T11:53:00.216667835Z INFO: on-disk MachineConfig config collection complete. 2026-06-08T11:53:00.344419382Z INFO: Found 1 replicas - prometheus-k8s-0 2026-06-08T11:53:00.370815602Z INFO: "metallb-operator" not detected. Skipping. 2026-06-08T11:53:00.448544396Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-06-08T11:53:00.448544396Z See 'oc rsync -h' for help and examples 2026-06-08T11:53:00.775114166Z error: the server doesn't have a resource type "performanceprofile" 2026-06-08T11:53:00.839014180Z Wrote inspect data to must-gather. 2026-06-08T11:53:00.920223816Z INFO: Worker host service log collection to complete. 2026-06-08T11:53:00.959541381Z ERROR: No running kube-apiserver pods found 2026-06-08T11:53:00.974976049Z INFO: Waiting for HAProxy config collection to complete ... 2026-06-08T11:53:01.118208538Z No resources found 2026-06-08T11:53:01.204461580Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-06-08T11:53:01.206487640Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-06-08T11:53:01.307877170Z Wrote inspect data to must-gather. 2026-06-08T11:53:01.339191703Z Wrote inspect data to must-gather. 2026-06-08T11:53:01.390162382Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-06-08T11:53:01.443577410Z Gathering data for ns/openshift-monitoring... 2026-06-08T11:53:01.466637639Z Wrote inspect data to must-gather. 2026-06-08T11:53:01.498099392Z Wrote inspect data to must-gather. 2026-06-08T11:53:01.523297006Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-06-08T11:53:01.558917971Z tar: Removing leading `/' from member names 2026-06-08T11:53:01.564505547Z INFO: HAProxy config collection complete. 2026-06-08T11:53:01.571940881Z INFO: 'previous-logs' folder not found on ip-10-0-130-94.ec2.internal, skipping... 2026-06-08T11:53:01.573287691Z Wrote inspect data to must-gather. 2026-06-08T11:53:01.756092060Z error: the server doesn't have a resource type "machineconfigs" 2026-06-08T11:53:01.811657373Z Wrote inspect data to must-gather. 2026-06-08T11:53:01.837900850Z INFO: Getting rules from prometheus-k8s-0 2026-06-08T11:53:01.876417072Z INFO: 'previous-logs' folder not found on ip-10-0-135-242.ec2.internal, skipping... 2026-06-08T11:53:01.964613987Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-06-08T11:53:01.965297828Z error: the server doesn't have a resource type "machineconfigpools" 2026-06-08T11:53:02.105776569Z Wrote inspect data to must-gather. 2026-06-08T11:53:02.158200104Z INFO: 'previous-logs' folder not found on ip-10-0-141-163.ec2.internal, skipping... 2026-06-08T11:53:02.158240993Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-06-08T11:53:02.158271624Z INFO: Machine Config Daemon termination log collection complete. 2026-06-08T11:53:02.207797411Z Wrote inspect data to must-gather. 2026-06-08T11:53:02.342639268Z INFO: Getting status/config from prometheus-k8s-0 2026-06-08T11:53:02.352084346Z Wrote inspect data to must-gather. 2026-06-08T11:53:02.357503350Z error: the server doesn't have a resource type "kubeletconfigs" 2026-06-08T11:53:02.565339715Z Wrote inspect data to must-gather. 2026-06-08T11:53:02.613623142Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-06-08T11:53:02.626699737Z INFO: INTERCONNECT MODE 2026-06-08T11:53:02.626742925Z INFO: Gathering ovn-kubernetes DBs 2026-06-08T11:53:02.756425863Z INFO: Getting status/flags from prometheus-k8s-0 2026-06-08T11:53:02.783376202Z Wrote inspect data to must-gather. 2026-06-08T11:53:02.794201599Z INFO: Gathering OVN_Northbound from ovnkube-node-bj2hd... 2026-06-08T11:53:02.794419470Z INFO: Gathering OVN_Northbound from ovnkube-node-bx9jt... 2026-06-08T11:53:02.794671671Z INFO: Gathering OVN_Northbound from ovnkube-node-f8jnb... 2026-06-08T11:53:02.795239806Z INFO: Gathering OVN_Southbound from ovnkube-node-bj2hd... 2026-06-08T11:53:02.795431111Z INFO: Gathering OVN_Southbound from ovnkube-node-bx9jt... 2026-06-08T11:53:02.795725983Z INFO: Gathering OVN_Southbound from ovnkube-node-f8jnb... 2026-06-08T11:53:02.812806988Z Wrote inspect data to /must-gather/istio. 2026-06-08T11:53:03.060004664Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-06-08T11:53:03.098834567Z tar: Removing leading `/' from member names 2026-06-08T11:53:03.103503229Z tar: Removing leading `/' from member names 2026-06-08T11:53:03.113216065Z tar: Removing leading `/' from member names 2026-06-08T11:53:03.123801194Z tar: Removing leading `/' from member names 2026-06-08T11:53:03.153545751Z tar: Removing leading `/' from member names 2026-06-08T11:53:03.154876421Z tar: Removing leading `/' from member names 2026-06-08T11:53:03.172160353Z Wrote inspect data to must-gather. 2026-06-08T11:53:03.207299737Z Wrote inspect data to /must-gather/istio. 2026-06-08T11:53:03.210711130Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-06-08T11:53:03.233746141Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-06-08T11:53:03.353530071Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-06-08T11:53:03.422287263Z Wrote inspect data to must-gather. 2026-06-08T11:53:03.436412592Z Wrote inspect data to /must-gather/istio. 2026-06-08T11:53:03.621830430Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-06-08T11:53:03.669478964Z Wrote inspect data to must-gather. 2026-06-08T11:53:03.696609766Z INFO: Waiting for network log collection to complete ... 2026-06-08T11:53:03.696709007Z INFO: Waiting for ovnk database copies to complete ... 2026-06-08T11:53:03.697117729Z INFO: Copying ovnk databases complete. 2026-06-08T11:53:03.697643960Z Wrote inspect data to /must-gather/istio. 2026-06-08T11:53:03.699717233Z 3.8M must-gather/network_logs/ovnk_database_store 2026-06-08T11:53:03.701068619Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-06-08T11:53:03.703267289Z ovnk_database_store/ 2026-06-08T11:53:03.703327271Z ovnk_database_store/ovnkube-node-bj2hd_sbdb 2026-06-08T11:53:03.730763003Z ovnk_database_store/ovnkube-node-bx9jt_nbdb 2026-06-08T11:53:03.740205481Z ovnk_database_store/ovnkube-node-bj2hd_nbdb 2026-06-08T11:53:03.750384257Z ovnk_database_store/ovnkube-node-f8jnb_nbdb 2026-06-08T11:53:03.759186218Z ovnk_database_store/ovnkube-node-bx9jt_sbdb 2026-06-08T11:53:03.784067768Z ovnk_database_store/ovnkube-node-f8jnb_sbdb 2026-06-08T11:53:03.807431362Z INFO: Network log collection complete. 2026-06-08T11:53:03.889501045Z W0608 11:53:03.889443 1502 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-06-08T11:53:03.895426236Z Wrote inspect data to must-gather. 2026-06-08T11:53:03.909203789Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-06-08T11:53:03.911025404Z Wrote inspect data to /must-gather/istio. 2026-06-08T11:53:04.032507214Z INFO: Getting status from alertmanager-main-0 2026-06-08T11:53:04.103002359Z Gathering data for ns/openshift-network-console... 2026-06-08T11:53:04.125566199Z Wrote inspect data to /must-gather/istio. 2026-06-08T11:53:04.128895965Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-06-08T11:53:04.217895325Z Wrote inspect data to must-gather. 2026-06-08T11:53:04.352024362Z Wrote inspect data to /must-gather/istio. 2026-06-08T11:53:04.473286082Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-06-08T11:53:04.475312854Z INFO: Fallback to identify the container image from release info 2026-06-08T11:53:04.568442662Z Wrote inspect data to /must-gather/istio. 2026-06-08T11:53:04.569774002Z Gathering data for ns/openshift-console-operator... 2026-06-08T11:53:04.571462823Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-06-08T11:53:04.728947777Z Wrote inspect data to /must-gather/istio. 2026-06-08T11:53:04.919396030Z Wrote inspect data to /must-gather/istio. 2026-06-08T11:53:04.955297350Z Gathering data for ns/openshift-console... 2026-06-08T11:53:05.229079255Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e 2026-06-08T11:53:05.358711955Z daemonset.apps/perf-node-gather-daemonset created 2026-06-08T11:53:05.490640332Z Gathering data for ns/openshift-cluster-storage-operator... 2026-06-08T11:53:05.499218803Z Waiting for performance profile collector pods to become ready: 1 2026-06-08T11:53:05.601285554Z Done executing Istio gather script 2026-06-08T11:53:05.960479899Z Gathering data for ns/openshift-dns-operator... 2026-06-08T11:53:06.296471323Z Gathering data for ns/openshift-dns... 2026-06-08T11:53:06.540065399Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-06-08T11:53:06.623995020Z Waiting for performance profile collector pods to become ready: 2 2026-06-08T11:53:06.896157593Z Gathering data for ns/openshift-image-registry... 2026-06-08T11:53:07.326776058Z Gathering data for ns/openshift-ingress-operator... 2026-06-08T11:53:07.619536586Z Gathering data for ns/openshift-ingress... 2026-06-08T11:53:07.746788673Z Waiting for performance profile collector pods to become ready: 3 2026-06-08T11:53:07.938968442Z Gathering data for ns/openshift-ingress-canary... 2026-06-08T11:53:08.289578484Z Gathering data for ns/openshift-insights... 2026-06-08T11:53:08.868134892Z Waiting for performance profile collector pods to become ready: 4 2026-06-08T11:53:09.976338056Z Waiting for performance profile collector pods to become ready: 5 2026-06-08T11:53:10.284096266Z Gathering data for ns/openshift-config... 2026-06-08T11:53:10.531189518Z Gathering data for ns/openshift-config-managed... 2026-06-08T11:53:10.902384460Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-06-08T11:53:11.094321482Z Waiting for performance profile collector pods to become ready: 6 2026-06-08T11:53:11.145356171Z Gathering data for ns/openshift-kube-apiserver... 2026-06-08T11:53:11.377950160Z Gathering data for ns/openshift-kube-controller-manager... 2026-06-08T11:53:11.545196147Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-06-08T11:53:11.641440190Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-06-08T11:53:11.939787284Z Gathering data for ns/openshift-kube-scheduler... 2026-06-08T11:53:12.194249167Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-06-08T11:53:12.223634958Z Waiting for performance profile collector pods to become ready: 7 2026-06-08T11:53:12.468530211Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-06-08T11:53:12.799847621Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-06-08T11:53:13.120458261Z Gathering data for ns/openshift-user-workload-monitoring... 2026-06-08T11:53:13.344990629Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-06-08T11:53:13.611707589Z Collecting performance related data for node ip-10-0-141-163.ec2.internal 2026-06-08T11:53:13.620311929Z Collecting performance related data for node ip-10-0-135-242.ec2.internal 2026-06-08T11:53:13.623013896Z Gathering data for ns/openshift-multus... 2026-06-08T11:53:13.631245970Z Collecting performance related data for node ip-10-0-130-94.ec2.internal 2026-06-08T11:53:15.165968884Z Gathering data for ns/openshift-ovn-kubernetes... 2026-06-08T11:53:16.411525146Z Gathering data for ns/openshift-host-network... 2026-06-08T11:53:16.550258580Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-06-08T11:53:16.749564435Z Gathering data for ns/openshift-network-diagnostics... 2026-06-08T11:53:17.255452789Z Gathering data for ns/openshift-network-node-identity... 2026-06-08T11:53:17.640416405Z Gathering data for ns/openshift-network-operator... 2026-06-08T11:53:18.013749668Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-06-08T11:53:18.261714242Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-06-08T11:53:18.738514885Z Gathering data for ns/openshift-apiserver-operator... 2026-06-08T11:53:19.036892892Z Gathering data for ns/openshift-apiserver... 2026-06-08T11:53:19.413665508Z Gathering data for ns/openshift-controller-manager-operator... 2026-06-08T11:53:19.681514099Z Gathering data for ns/openshift-controller-manager... 2026-06-08T11:53:19.941624918Z Gathering data for ns/openshift-cluster-samples-operator... 2026-06-08T11:53:20.565123613Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-06-08T11:53:20.833845423Z Gathering data for ns/openshift-service-ca-operator... 2026-06-08T11:53:21.151253213Z Gathering data for ns/openshift-service-ca... 2026-06-08T11:53:21.501600212Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-06-08T11:53:21.555070035Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-06-08T11:53:21.698677722Z Collecting kubelet logs for node ip-10-0-141-163.ec2.internal 2026-06-08T11:53:21.700535884Z Collecting kubelet logs for node ip-10-0-135-242.ec2.internal 2026-06-08T11:53:21.702546880Z Collecting kubelet logs for node ip-10-0-130-94.ec2.internal 2026-06-08T11:53:22.058902495Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-tgp8n namespace 2026-06-08T11:53:22.064103952Z INFO: Node performance data collection complete. 2026-06-08T11:53:22.466313260Z Wrote inspect data to must-gather. 2026-06-08T11:53:22.466360313Z error: inspection completed with the errors occurred while gathering data: 2026-06-08T11:53:22.466360313Z skipping gathering secrets/support due to error: secrets "support" not found 2026-06-08T11:53:22.590619782Z error: the server doesn't have a resource type "clusters" 2026-06-08T11:53:22.718908373Z Caches written to disk