2026-07-28T18:41:35.156319179Z [disk usage checker] Started 2026-07-28T18:41:35.159720573Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-07-28T18:41:35.344728570Z Gathering data for ns/openshift-cluster-version... 2026-07-28T18:41:35.482276625Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-28T18:41:35.575315272Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-28T18:41:35.658499883Z Gathering data for ns/default... 2026-07-28T18:41:35.949475130Z Gathering data for ns/openshift... 2026-07-28T18:41:36.355319709Z Gathering data for ns/kube-system... 2026-07-28T18:41:37.133029824Z Gathering data for ns/openshift-etcd... 2026-07-28T18:41:37.448262243Z Wrote inspect data to must-gather. 2026-07-28T18:41:37.448297398Z error: inspection completed with the errors occurred while gathering data: 2026-07-28T18:41:37.448297398Z namespaces "assisted-installer" not found 2026-07-28T18:41:37.959427076Z Waiting on subprocesses to finish execution. 2026-07-28T18:41:37.967475310Z INFO: Gathering machine config daemon's old logs from all nodes 2026-07-28T18:41:37.969790119Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-07-28T18:41:37.972082815Z Executing Istio gather script 2026-07-28T18:41:37.973163241Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-07-28T18:41:37.981088195Z INFO: Collecting host service logs for crio 2026-07-28T18:41:37.981411427Z INFO: Collecting host service logs for kubelet 2026-07-28T18:41:37.981720827Z INFO: Collecting host service logs for rpm-ostreed 2026-07-28T18:41:37.982046591Z INFO: Collecting host service logs for ostree-finalize-staged 2026-07-28T18:41:37.982356867Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-07-28T18:41:37.982642079Z INFO: Collecting host service logs for machine-config-daemon-host 2026-07-28T18:41:37.982952002Z INFO: Collecting host service logs for NetworkManager 2026-07-28T18:41:37.983258390Z INFO: Collecting host service logs for openvswitch 2026-07-28T18:41:37.983558641Z INFO: Collecting host service logs for ovs-configuration 2026-07-28T18:41:37.983908406Z INFO: Collecting host service logs for ovsdb-server 2026-07-28T18:41:37.984192920Z INFO: Collecting host service logs for ovs-vswitchd 2026-07-28T18:41:37.984559458Z INFO: Waiting for worker host service log collection to complete ... 2026-07-28T18:41:38.009296031Z INFO: Gathering HAProxy config files 2026-07-28T18:41:38.119525614Z INFO: Waiting for node performance related collection to complete ... 2026-07-28T18:41:38.647580093Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-8kvgr 2026-07-28T18:41:38.647632137Z insights-runtime-extractor-dhjwq 2026-07-28T18:41:38.647652405Z insights-runtime-extractor-z65fp 2026-07-28T18:41:38.647672685Z insights-runtime-extractor-zlnz9 2026-07-28T18:41:38.813091426Z Wrote inspect data to must-gather. 2026-07-28T18:41:38.813899466Z INFO: Found 1 replicas - prometheus-k8s-0 2026-07-28T18:41:38.892789916Z No resources found 2026-07-28T18:41:38.998756192Z INFO: "sriov-network-operator" not detected. Skipping. 2026-07-28T18:41:39.007061524Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-07-28T18:41:39.087505954Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-07-28T18:41:39.087596295Z INFO: on-disk MachineConfig config collection complete. 2026-07-28T18:41:39.191533936Z No resources found in openshift-etcd namespace. 2026-07-28T18:41:39.203780492Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-07-28T18:41:39.212147805Z INFO: "metallb-operator" not detected. Skipping. 2026-07-28T18:41:39.306813951Z error: the server doesn't have a resource type "performanceprofile" 2026-07-28T18:41:39.349356256Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-07-28T18:41:39.349356256Z See 'oc rsync -h' for help and examples 2026-07-28T18:41:39.448950169Z ERROR: No running kube-apiserver pods found 2026-07-28T18:41:39.527383733Z INFO: Worker host service log collection to complete. 2026-07-28T18:41:39.545851306Z INFO: Waiting for HAProxy config collection to complete ... 2026-07-28T18:41:39.617035028Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-07-28T18:41:39.684242917Z Wrote inspect data to must-gather. 2026-07-28T18:41:39.896235222Z Wrote inspect data to must-gather. 2026-07-28T18:41:40.110231626Z tar: Removing leading `/' from member names 2026-07-28T18:41:40.117766486Z INFO: HAProxy config collection complete. 2026-07-28T18:41:40.233672768Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-07-28T18:41:40.280085565Z INFO: 'previous-logs' folder not found on ip-10-0-134-110.ec2.internal, skipping... 2026-07-28T18:41:40.392796244Z Wrote inspect data to must-gather. 2026-07-28T18:41:40.438234612Z Wrote inspect data to must-gather. 2026-07-28T18:41:40.480516649Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-28T18:41:40.495900477Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-07-28T18:41:40.611605915Z No resources found 2026-07-28T18:41:40.613005626Z Gathering data for ns/openshift-monitoring... 2026-07-28T18:41:40.677347990Z Wrote inspect data to must-gather. 2026-07-28T18:41:40.787521273Z INFO: Getting rules from prometheus-k8s-0 2026-07-28T18:41:40.799293077Z Wrote inspect data to must-gather. 2026-07-28T18:41:40.829951104Z error: the server doesn't have a resource type "machineconfigs" 2026-07-28T18:41:40.934400434Z Wrote inspect data to must-gather. 2026-07-28T18:41:41.008345520Z INFO: 'previous-logs' folder not found on ip-10-0-138-231.ec2.internal, skipping... 2026-07-28T18:41:41.031126550Z error: the server doesn't have a resource type "machineconfigpools" 2026-07-28T18:41:41.247037963Z Wrote inspect data to must-gather. 2026-07-28T18:41:41.302780150Z INFO: Getting status/config from prometheus-k8s-0 2026-07-28T18:41:41.328143381Z Wrote inspect data to must-gather. 2026-07-28T18:41:41.369259714Z INFO: 'previous-logs' folder not found on ip-10-0-139-15.ec2.internal, skipping... 2026-07-28T18:41:41.369305053Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-07-28T18:41:41.369305053Z INFO: Machine Config Daemon termination log collection complete. 2026-07-28T18:41:41.508293921Z error: the server doesn't have a resource type "kubeletconfigs" 2026-07-28T18:41:41.559541565Z Wrote inspect data to must-gather. 2026-07-28T18:41:41.756349410Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-07-28T18:41:41.781359637Z Wrote inspect data to must-gather. 2026-07-28T18:41:41.784610971Z INFO: Getting status/flags from prometheus-k8s-0 2026-07-28T18:41:41.941097952Z INFO: INTERCONNECT MODE 2026-07-28T18:41:41.941197218Z INFO: Gathering ovn-kubernetes DBs 2026-07-28T18:41:42.027684479Z Wrote inspect data to must-gather. 2026-07-28T18:41:42.044960877Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-07-28T18:41:42.135358273Z INFO: Gathering OVN_Northbound from ovnkube-node-9l7dh... 2026-07-28T18:41:42.135695767Z INFO: Gathering OVN_Northbound from ovnkube-node-lbhmb... 2026-07-28T18:41:42.135983055Z INFO: Gathering OVN_Northbound from ovnkube-node-swcbs... 2026-07-28T18:41:42.136279204Z INFO: Gathering OVN_Southbound from ovnkube-node-9l7dh... 2026-07-28T18:41:42.136543548Z INFO: Gathering OVN_Southbound from ovnkube-node-lbhmb... 2026-07-28T18:41:42.136795384Z INFO: Gathering OVN_Southbound from ovnkube-node-swcbs... 2026-07-28T18:41:42.367508164Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-07-28T18:41:42.395159452Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-28T18:41:42.398704679Z Wrote inspect data to must-gather. 2026-07-28T18:41:42.457946794Z tar: Removing leading `/' from member names 2026-07-28T18:41:42.508174852Z tar: Removing leading `/' from member names 2026-07-28T18:41:42.508356694Z tar: Removing leading `/' from member names 2026-07-28T18:41:42.539060837Z tar: Removing leading `/' from member names 2026-07-28T18:41:42.564566737Z tar: Removing leading `/' from member names 2026-07-28T18:41:42.570483853Z tar: Removing leading `/' from member names 2026-07-28T18:41:42.705386845Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-07-28T18:41:42.719157934Z Wrote inspect data to must-gather. 2026-07-28T18:41:42.785226072Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-07-28T18:41:43.002998983Z W0728 18:41:43.002586 1415 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-07-28T18:41:43.009921104Z Wrote inspect data to must-gather. 2026-07-28T18:41:43.025128243Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:41:43.134222457Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-28T18:41:43.153575678Z INFO: Waiting for network log collection to complete ... 2026-07-28T18:41:43.153611568Z INFO: Waiting for ovnk database copies to complete ... 2026-07-28T18:41:43.154017903Z INFO: Copying ovnk databases complete. 2026-07-28T18:41:43.158750516Z 4.5M must-gather/network_logs/ovnk_database_store 2026-07-28T18:41:43.161907709Z ovnk_database_store/ 2026-07-28T18:41:43.161962257Z ovnk_database_store/ovnkube-node-9l7dh_nbdb 2026-07-28T18:41:43.181731585Z ovnk_database_store/ovnkube-node-lbhmb_nbdb 2026-07-28T18:41:43.194013646Z ovnk_database_store/ovnkube-node-lbhmb_sbdb 2026-07-28T18:41:43.202835958Z INFO: Getting status from alertmanager-main-0 2026-07-28T18:41:43.230828664Z ovnk_database_store/ovnkube-node-swcbs_nbdb 2026-07-28T18:41:43.243638026Z ovnk_database_store/ovnkube-node-swcbs_sbdb 2026-07-28T18:41:43.272654485Z ovnk_database_store/ovnkube-node-9l7dh_sbdb 2026-07-28T18:41:43.296821473Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:41:43.300349304Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-07-28T18:41:43.309424357Z INFO: Network log collection complete. 2026-07-28T18:41:43.351990253Z Gathering data for ns/openshift-network-console... 2026-07-28T18:41:43.358176176Z Wrote inspect data to must-gather. 2026-07-28T18:41:43.522187461Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:41:43.647730173Z Wrote inspect data to must-gather. 2026-07-28T18:41:43.739585560Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:41:43.743149685Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-07-28T18:41:43.811446383Z Gathering data for ns/openshift-console-operator... 2026-07-28T18:41:43.982184459Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-07-28T18:41:43.982546348Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:41:43.986234178Z INFO: Fallback to identify the container image from release info 2026-07-28T18:41:44.211298676Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:41:44.214347741Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-07-28T18:41:44.324993843Z Gathering data for ns/openshift-console... 2026-07-28T18:41:44.444964685Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:41:44.565976928Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811 2026-07-28T18:41:44.647170366Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:41:44.649767936Z Inspecting resource crd/inferencepools.inference.networking.k8s.io 2026-07-28T18:41:44.705846761Z daemonset.apps/perf-node-gather-daemonset created 2026-07-28T18:41:44.853981272Z Waiting for performance profile collector pods to become ready: 1 2026-07-28T18:41:44.866968179Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:41:44.971432364Z Gathering data for ns/openshift-cluster-storage-operator... 2026-07-28T18:41:45.074925184Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:41:45.079022983Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io 2026-07-28T18:41:45.281924718Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:41:45.297485539Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-07-28T18:41:45.502324526Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:41:45.505833617Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-07-28T18:41:45.522133300Z Gathering data for ns/openshift-dns-operator... 2026-07-28T18:41:45.691333452Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:41:45.806741068Z Gathering data for ns/openshift-dns... 2026-07-28T18:41:45.885855047Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:41:45.984130927Z Waiting for performance profile collector pods to become ready: 2 2026-07-28T18:41:46.425136437Z Gathering data for ns/openshift-image-registry... 2026-07-28T18:41:46.599744328Z Done executing Istio gather script 2026-07-28T18:41:46.910674658Z Gathering data for ns/openshift-ingress-operator... 2026-07-28T18:41:47.104323066Z Waiting for performance profile collector pods to become ready: 3 2026-07-28T18:41:47.229353823Z Gathering data for ns/openshift-ingress... 2026-07-28T18:41:47.557456604Z Gathering data for ns/openshift-ingress-canary... 2026-07-28T18:41:47.924144397Z Gathering data for ns/openshift-insights... 2026-07-28T18:41:48.253530529Z Waiting for performance profile collector pods to become ready: 4 2026-07-28T18:41:49.371972863Z Waiting for performance profile collector pods to become ready: 5 2026-07-28T18:41:50.303385044Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-07-28T18:41:50.303997791Z Gathering data for ns/kserve... 2026-07-28T18:41:50.489334278Z Waiting for performance profile collector pods to become ready: 6 2026-07-28T18:41:51.618483851Z Waiting for performance profile collector pods to become ready: 7 2026-07-28T18:41:52.695816655Z Gathering data for ns/openshift-config... 2026-07-28T18:41:52.755251224Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-07-28T18:41:52.979456067Z Gathering data for ns/openshift-config-managed... 2026-07-28T18:41:53.049156355Z Collecting performance related data for node ip-10-0-138-231.ec2.internal 2026-07-28T18:41:53.061529487Z Collecting performance related data for node ip-10-0-139-15.ec2.internal 2026-07-28T18:41:53.067659543Z Collecting performance related data for node ip-10-0-134-110.ec2.internal 2026-07-28T18:41:53.451691539Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-07-28T18:41:53.790310807Z Gathering data for ns/openshift-kube-apiserver... 2026-07-28T18:41:54.100722549Z Gathering data for ns/openshift-kube-controller-manager... 2026-07-28T18:41:54.471773944Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-07-28T18:41:54.806006943Z Gathering data for ns/openshift-kube-scheduler... 2026-07-28T18:41:55.099466458Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-07-28T18:41:55.309053657Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-07-28T18:41:55.393567768Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-07-28T18:41:55.750748550Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-07-28T18:41:56.099793151Z Gathering data for ns/openshift-user-workload-monitoring... 2026-07-28T18:41:56.700343277Z Gathering data for ns/openshift-multus... 2026-07-28T18:41:58.185761255Z Gathering data for ns/openshift-ovn-kubernetes... 2026-07-28T18:41:59.481933439Z Gathering data for ns/openshift-host-network... 2026-07-28T18:41:59.759812506Z Gathering data for ns/openshift-network-diagnostics... 2026-07-28T18:42:00.282747172Z Gathering data for ns/openshift-network-node-identity... 2026-07-28T18:42:00.316688872Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-07-28T18:42:00.718340432Z Gathering data for ns/openshift-network-operator... 2026-07-28T18:42:01.083998094Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-07-28T18:42:01.344742506Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-07-28T18:42:01.672634031Z Collecting kubelet logs for node ip-10-0-138-231.ec2.internal 2026-07-28T18:42:01.674438465Z Collecting kubelet logs for node ip-10-0-139-15.ec2.internal 2026-07-28T18:42:01.676447956Z Collecting kubelet logs for node ip-10-0-134-110.ec2.internal 2026-07-28T18:42:01.868322968Z Gathering data for ns/openshift-apiserver-operator... 2026-07-28T18:42:02.117900868Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-fsvbn namespace 2026-07-28T18:42:02.123185672Z INFO: Node performance data collection complete. 2026-07-28T18:42:02.211358431Z Gathering data for ns/openshift-apiserver... 2026-07-28T18:42:02.628601352Z Gathering data for ns/openshift-controller-manager-operator... 2026-07-28T18:42:02.874432518Z Gathering data for ns/openshift-controller-manager... 2026-07-28T18:42:03.139648000Z Gathering data for ns/openshift-cluster-samples-operator... 2026-07-28T18:42:03.784888624Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-07-28T18:42:04.057499874Z Gathering data for ns/openshift-service-ca-operator... 2026-07-28T18:42:04.367564643Z Gathering data for ns/openshift-service-ca... 2026-07-28T18:42:04.717883057Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-07-28T18:42:05.322329298Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-07-28T18:42:05.591714862Z Wrote inspect data to must-gather. 2026-07-28T18:42:05.591764409Z error: inspection completed with the errors occurred while gathering data: 2026-07-28T18:42:05.591764409Z skipping gathering secrets/support due to error: secrets "support" not found 2026-07-28T18:42:05.722690256Z error: the server doesn't have a resource type "clusters" 2026-07-28T18:42:05.872973952Z Caches written to disk