2026-07-28T15:44:43.127134711Z [disk usage checker] Started 2026-07-28T15:44:43.130248458Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-28T15:44:43.312014383Z Gathering data for ns/openshift-cluster-version... 2026-07-28T15:44:43.433200245Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-28T15:44:43.510182239Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-28T15:44:43.581334033Z Gathering data for ns/default... 2026-07-28T15:44:43.851280977Z Gathering data for ns/openshift... 2026-07-28T15:44:44.228534636Z Gathering data for ns/kube-system... 2026-07-28T15:44:44.870225368Z Gathering data for ns/openshift-etcd... 2026-07-28T15:44:45.144402180Z Wrote inspect data to must-gather. 2026-07-28T15:44:45.144433143Z error: inspection completed with the errors occurred while gathering data: 2026-07-28T15:44:45.144433143Z namespaces "assisted-installer" not found 2026-07-28T15:44:45.505289935Z Waiting on subprocesses to finish execution. 2026-07-28T15:44:45.509958955Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-07-28T15:44:45.513555459Z Executing Istio gather script 2026-07-28T15:44:45.516211103Z INFO: Gathering HAProxy config files 2026-07-28T15:44:45.517793161Z INFO: Collecting host service logs for crio 2026-07-28T15:44:45.518114271Z INFO: Collecting host service logs for kubelet 2026-07-28T15:44:45.518463459Z INFO: Collecting host service logs for rpm-ostreed 2026-07-28T15:44:45.518794652Z INFO: Collecting host service logs for ostree-finalize-staged 2026-07-28T15:44:45.519141858Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-07-28T15:44:45.519478728Z INFO: Collecting host service logs for machine-config-daemon-host 2026-07-28T15:44:45.519813662Z INFO: Collecting host service logs for NetworkManager 2026-07-28T15:44:45.520164800Z INFO: Collecting host service logs for openvswitch 2026-07-28T15:44:45.521102351Z INFO: Collecting host service logs for ovs-configuration 2026-07-28T15:44:45.521457927Z INFO: Collecting host service logs for ovsdb-server 2026-07-28T15:44:45.521792623Z INFO: Collecting host service logs for ovs-vswitchd 2026-07-28T15:44:45.522202695Z INFO: Waiting for worker host service log collection to complete ... 2026-07-28T15:44:45.522226177Z INFO: Gathering machine config daemon's old logs from all nodes 2026-07-28T15:44:45.525153581Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-07-28T15:44:45.736795801Z INFO: Waiting for node performance related collection to complete ... 2026-07-28T15:44:45.864838924Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-fzgwv 2026-07-28T15:44:45.864838924Z insights-runtime-extractor-ckh5z 2026-07-28T15:44:45.864838924Z insights-runtime-extractor-pr7fd 2026-07-28T15:44:45.864838924Z insights-runtime-extractor-w5z4f 2026-07-28T15:44:46.045856024Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-07-28T15:44:46.094623910Z No resources found in openshift-etcd namespace. 2026-07-28T15:44:46.518260607Z INFO: "sriov-network-operator" not detected. Skipping. 2026-07-28T15:44:46.526089086Z INFO: "metallb-operator" not detected. Skipping. 2026-07-28T15:44:46.555775667Z No resources found 2026-07-28T15:44:46.589100124Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-07-28T15:44:46.589177755Z INFO: on-disk MachineConfig config collection complete. 2026-07-28T15:44:46.652925480Z Wrote inspect data to must-gather. 2026-07-28T15:44:46.753150231Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-07-28T15:44:46.815551597Z error: the server doesn't have a resource type "performanceprofile" 2026-07-28T15:44:46.854150804Z INFO: Found 1 replicas - prometheus-k8s-0 2026-07-28T15:44:46.934576230Z ERROR: No running kube-apiserver pods found 2026-07-28T15:44:46.950463310Z INFO: Worker host service log collection to complete. 2026-07-28T15:44:47.015629531Z Wrote inspect data to must-gather. 2026-07-28T15:44:47.034007788Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-07-28T15:44:47.034007788Z See 'oc rsync -h' for help and examples 2026-07-28T15:44:47.062908183Z INFO: Waiting for HAProxy config collection to complete ... 2026-07-28T15:44:47.316003916Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-07-28T15:44:47.339564449Z Wrote inspect data to must-gather. 2026-07-28T15:44:47.626539092Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-28T15:44:47.636760497Z tar: Removing leading `/' from member names 2026-07-28T15:44:47.637139407Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-07-28T15:44:47.645458552Z INFO: HAProxy config collection complete. 2026-07-28T15:44:47.876945237Z Wrote inspect data to must-gather. 2026-07-28T15:44:47.986317136Z Wrote inspect data to must-gather. 2026-07-28T15:44:47.991318989Z Gathering data for ns/openshift-monitoring... 2026-07-28T15:44:47.995526251Z INFO: 'previous-logs' folder not found on ip-10-0-135-193.ec2.internal, skipping... 2026-07-28T15:44:48.046922891Z Wrote inspect data to must-gather. 2026-07-28T15:44:48.053681127Z No resources found 2026-07-28T15:44:48.124799329Z Wrote inspect data to must-gather. 2026-07-28T15:44:48.137382308Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-28T15:44:48.146462009Z error: the server doesn't have a resource type "machineconfigs" 2026-07-28T15:44:48.160205564Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-07-28T15:44:48.322961549Z error: the server doesn't have a resource type "machineconfigpools" 2026-07-28T15:44:48.339777101Z INFO: Getting rules from prometheus-k8s-0 2026-07-28T15:44:48.352315476Z INFO: 'previous-logs' folder not found on ip-10-0-138-160.ec2.internal, skipping... 2026-07-28T15:44:48.368750827Z Wrote inspect data to must-gather. 2026-07-28T15:44:48.544689965Z Wrote inspect data to must-gather. 2026-07-28T15:44:48.611021059Z INFO: 'previous-logs' folder not found on ip-10-0-143-125.ec2.internal, skipping... 2026-07-28T15:44:48.611053118Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-07-28T15:44:48.611085745Z INFO: Machine Config Daemon termination log collection complete. 2026-07-28T15:44:48.624593184Z Wrote inspect data to must-gather. 2026-07-28T15:44:48.691661919Z error: the server doesn't have a resource type "kubeletconfigs" 2026-07-28T15:44:48.753865470Z INFO: Getting status/config from prometheus-k8s-0 2026-07-28T15:44:48.858293518Z Wrote inspect data to must-gather. 2026-07-28T15:44:48.889591536Z Wrote inspect data to must-gather. 2026-07-28T15:44:49.073319229Z Wrote inspect data to must-gather. 2026-07-28T15:44:49.092449481Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-07-28T15:44:49.126225319Z INFO: INTERCONNECT MODE 2026-07-28T15:44:49.126268029Z INFO: Gathering ovn-kubernetes DBs 2026-07-28T15:44:49.134173660Z INFO: Getting status/flags from prometheus-k8s-0 2026-07-28T15:44:49.276002288Z Wrote inspect data to /must-gather/istio. 2026-07-28T15:44:49.277507182Z Wrote inspect data to must-gather. 2026-07-28T15:44:49.279938564Z INFO: Gathering OVN_Northbound from ovnkube-node-5vlpq... 2026-07-28T15:44:49.280147146Z INFO: Gathering OVN_Northbound from ovnkube-node-zmgsn... 2026-07-28T15:44:49.280347351Z INFO: Gathering OVN_Northbound from ovnkube-node-zrkfd... 2026-07-28T15:44:49.280618972Z INFO: Gathering OVN_Southbound from ovnkube-node-5vlpq... 2026-07-28T15:44:49.280861120Z INFO: Gathering OVN_Southbound from ovnkube-node-zmgsn... 2026-07-28T15:44:49.281170170Z INFO: Gathering OVN_Southbound from ovnkube-node-zrkfd... 2026-07-28T15:44:49.430818960Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-07-28T15:44:49.571149830Z tar: Removing leading `/' from member names 2026-07-28T15:44:49.603142979Z tar: Removing leading `/' from member names 2026-07-28T15:44:49.606223287Z tar: Removing leading `/' from member names 2026-07-28T15:44:49.610698568Z tar: Removing leading `/' from member names 2026-07-28T15:44:49.640938611Z Wrote inspect data to /must-gather/istio. 2026-07-28T15:44:49.645035244Z tar: Removing leading `/' from member names 2026-07-28T15:44:49.645701553Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-07-28T15:44:49.648128388Z tar: Removing leading `/' from member names 2026-07-28T15:44:49.682498146Z Wrote inspect data to must-gather. 2026-07-28T15:44:49.743367647Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-28T15:44:49.825575267Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-07-28T15:44:49.883080013Z Wrote inspect data to /must-gather/istio. 2026-07-28T15:44:49.901129187Z Wrote inspect data to must-gather. 2026-07-28T15:44:50.100249097Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-07-28T15:44:50.132400257Z Wrote inspect data to /must-gather/istio. 2026-07-28T15:44:50.134740771Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-07-28T15:44:50.135341577Z INFO: Waiting for network log collection to complete ... 2026-07-28T15:44:50.135451853Z INFO: Waiting for ovnk database copies to complete ... 2026-07-28T15:44:50.136005043Z INFO: Copying ovnk databases complete. 2026-07-28T15:44:50.137509224Z 3.5M must-gather/network_logs/ovnk_database_store 2026-07-28T15:44:50.139643766Z ovnk_database_store/ 2026-07-28T15:44:50.139643766Z ovnk_database_store/ovnkube-node-zrkfd_sbdb 2026-07-28T15:44:50.145107598Z Wrote inspect data to must-gather. 2026-07-28T15:44:50.164877978Z ovnk_database_store/ovnkube-node-5vlpq_sbdb 2026-07-28T15:44:50.184446243Z ovnk_database_store/ovnkube-node-zmgsn_sbdb 2026-07-28T15:44:50.205437876Z ovnk_database_store/ovnkube-node-5vlpq_nbdb 2026-07-28T15:44:50.214344278Z ovnk_database_store/ovnkube-node-zrkfd_nbdb 2026-07-28T15:44:50.224511762Z ovnk_database_store/ovnkube-node-zmgsn_nbdb 2026-07-28T15:44:50.236355523Z INFO: Network log collection complete. 2026-07-28T15:44:50.325637966Z Wrote inspect data to /must-gather/istio. 2026-07-28T15:44:50.356456968Z W0728 15:44:50.356401 1529 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-07-28T15:44:50.363710497Z Wrote inspect data to must-gather. 2026-07-28T15:44:50.441275138Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-28T15:44:50.499628687Z INFO: Getting status from alertmanager-main-0 2026-07-28T15:44:50.532086859Z Wrote inspect data to /must-gather/istio. 2026-07-28T15:44:50.534182013Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-07-28T15:44:50.645557169Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-07-28T15:44:50.648182059Z INFO: Fallback to identify the container image from release info 2026-07-28T15:44:50.654500745Z Gathering data for ns/openshift-network-console... 2026-07-28T15:44:50.771423890Z Wrote inspect data to /must-gather/istio. 2026-07-28T15:44:50.939792555Z Wrote inspect data to /must-gather/istio. 2026-07-28T15:44:50.941948035Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-07-28T15:44:51.039692672Z Gathering data for ns/openshift-console-operator... 2026-07-28T15:44:51.091457838Z Wrote inspect data to /must-gather/istio. 2026-07-28T15:44:51.256047261Z Wrote inspect data to /must-gather/istio. 2026-07-28T15:44:51.303085651Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811 2026-07-28T15:44:51.392425911Z Gathering data for ns/openshift-console... 2026-07-28T15:44:51.433623212Z daemonset.apps/perf-node-gather-daemonset created 2026-07-28T15:44:51.568397541Z Waiting for performance profile collector pods to become ready: 1 2026-07-28T15:44:51.918167694Z Done executing Istio gather script 2026-07-28T15:44:51.928123396Z Gathering data for ns/openshift-cluster-storage-operator... 2026-07-28T15:44:52.347800486Z Gathering data for ns/openshift-dns-operator... 2026-07-28T15:44:52.636767019Z Gathering data for ns/openshift-dns... 2026-07-28T15:44:52.675228098Z Waiting for performance profile collector pods to become ready: 2 2026-07-28T15:44:53.142562972Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-28T15:44:53.172891142Z Gathering data for ns/openshift-image-registry... 2026-07-28T15:44:53.601612794Z Gathering data for ns/openshift-ingress-operator... 2026-07-28T15:44:53.774362884Z Waiting for performance profile collector pods to become ready: 3 2026-07-28T15:44:53.943060066Z Gathering data for ns/openshift-ingress... 2026-07-28T15:44:54.258592996Z Gathering data for ns/openshift-ingress-canary... 2026-07-28T15:44:54.621065926Z Gathering data for ns/openshift-insights... 2026-07-28T15:44:54.870136018Z Waiting for performance profile collector pods to become ready: 4 2026-07-28T15:44:55.974875622Z Waiting for performance profile collector pods to become ready: 5 2026-07-28T15:44:56.499880417Z Gathering data for ns/openshift-config... 2026-07-28T15:44:56.771562333Z Gathering data for ns/openshift-config-managed... 2026-07-28T15:44:57.084937370Z Waiting for performance profile collector pods to become ready: 6 2026-07-28T15:44:57.156278625Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-07-28T15:44:57.374615726Z Gathering data for ns/openshift-kube-apiserver... 2026-07-28T15:44:57.592091221Z Gathering data for ns/openshift-kube-controller-manager... 2026-07-28T15:44:57.880575301Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-07-28T15:44:58.128398913Z Gathering data for ns/openshift-kube-scheduler... 2026-07-28T15:44:58.149005098Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-28T15:44:58.210311934Z Waiting for performance profile collector pods to become ready: 7 2026-07-28T15:44:58.367212765Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-07-28T15:44:58.661114469Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-07-28T15:44:58.930817503Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-07-28T15:44:59.192294113Z Gathering data for ns/openshift-user-workload-monitoring... 2026-07-28T15:44:59.314641026Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-07-28T15:44:59.549314635Z Collecting performance related data for node ip-10-0-135-193.ec2.internal 2026-07-28T15:44:59.557452412Z Collecting performance related data for node ip-10-0-143-125.ec2.internal 2026-07-28T15:44:59.564270359Z Collecting performance related data for node ip-10-0-138-160.ec2.internal 2026-07-28T15:44:59.741935847Z Gathering data for ns/openshift-multus... 2026-07-28T15:45:01.261851960Z Gathering data for ns/openshift-ovn-kubernetes... 2026-07-28T15:45:02.463026736Z Gathering data for ns/openshift-host-network... 2026-07-28T15:45:02.778234455Z Gathering data for ns/openshift-network-diagnostics... 2026-07-28T15:45:03.155320078Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-28T15:45:03.260288856Z Gathering data for ns/openshift-network-node-identity... 2026-07-28T15:45:03.629781162Z Gathering data for ns/openshift-network-operator... 2026-07-28T15:45:03.994101482Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-07-28T15:45:04.282591406Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-07-28T15:45:04.751093329Z Gathering data for ns/openshift-apiserver-operator... 2026-07-28T15:45:05.017249819Z Gathering data for ns/openshift-apiserver... 2026-07-28T15:45:05.373935080Z Gathering data for ns/openshift-controller-manager-operator... 2026-07-28T15:45:05.603743807Z Gathering data for ns/openshift-controller-manager... 2026-07-28T15:45:05.886377432Z Gathering data for ns/openshift-cluster-samples-operator... 2026-07-28T15:45:06.483129779Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-07-28T15:45:06.746897532Z Gathering data for ns/openshift-service-ca-operator... 2026-07-28T15:45:07.046622471Z Gathering data for ns/openshift-service-ca... 2026-07-28T15:45:07.325724593Z Collecting kubelet logs for node ip-10-0-135-193.ec2.internal 2026-07-28T15:45:07.327352744Z Collecting kubelet logs for node ip-10-0-143-125.ec2.internal 2026-07-28T15:45:07.328919163Z Collecting kubelet logs for node ip-10-0-138-160.ec2.internal 2026-07-28T15:45:07.389953273Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-07-28T15:45:07.660535123Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-vfwwp namespace 2026-07-28T15:45:07.665817482Z INFO: Node performance data collection complete. 2026-07-28T15:45:08.160012683Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-28T15:45:08.249634887Z Wrote inspect data to must-gather. 2026-07-28T15:45:08.249677409Z error: inspection completed with the errors occurred while gathering data: 2026-07-28T15:45:08.249677409Z skipping gathering secrets/support due to error: secrets "support" not found 2026-07-28T15:45:08.364150572Z error: the server doesn't have a resource type "clusters" 2026-07-28T15:45:08.490017765Z Caches written to disk