2026-09-05T02:57:48.201589180Z [disk usage checker] Started 2026-09-05T02:57:48.204877771Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 2026-09-05T02:57:48.394742404Z Gathering data for ns/openshift-cluster-version... 2026-09-05T02:57:48.510241877Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-05T02:57:48.591187407Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-05T02:57:48.676031817Z Gathering data for ns/default... 2026-09-05T02:57:48.988165474Z Gathering data for ns/openshift... 2026-09-05T02:57:49.391758383Z Gathering data for ns/kube-system... 2026-09-05T02:57:50.169793486Z Gathering data for ns/openshift-etcd... 2026-09-05T02:57:50.474598470Z Wrote inspect data to must-gather. 2026-09-05T02:57:50.474630548Z error: inspection completed with the errors occurred while gathering data: 2026-09-05T02:57:50.474630548Z namespaces "assisted-installer" not found 2026-09-05T02:57:50.743063185Z Waiting on subprocesses to finish execution. 2026-09-05T02:57:50.752999386Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-05T02:57:50.754060861Z Executing Istio gather script 2026-09-05T02:57:50.756163014Z INFO: Collecting host service logs for crio 2026-09-05T02:57:50.756496144Z INFO: Collecting host service logs for kubelet 2026-09-05T02:57:50.756843048Z INFO: Collecting host service logs for rpm-ostreed 2026-09-05T02:57:50.757181817Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-05T02:57:50.757541210Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-05T02:57:50.757884014Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-05T02:57:50.758230732Z INFO: Collecting host service logs for NetworkManager 2026-09-05T02:57:50.758585008Z INFO: Collecting host service logs for openvswitch 2026-09-05T02:57:50.758924280Z INFO: Collecting host service logs for ovs-configuration 2026-09-05T02:57:50.759265221Z INFO: Collecting host service logs for ovsdb-server 2026-09-05T02:57:50.759601363Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-05T02:57:50.759985428Z INFO: Waiting for worker host service log collection to complete ... 2026-09-05T02:57:50.767580281Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-05T02:57:50.773530830Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-05T02:57:50.776843122Z INFO: Gathering HAProxy config files 2026-09-05T02:57:50.783594696Z INFO: Waiting for node performance related collection to complete ... 2026-09-05T02:57:51.225972348Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-05T02:57:51.479396332Z error: the server doesn't have a resource type "performanceprofile" 2026-09-05T02:57:51.489432304Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-05T02:57:51.489539658Z INFO: on-disk MachineConfig config collection complete. 2026-09-05T02:57:51.765685440Z No resources found 2026-09-05T02:57:51.814543631Z INFO: Collecting Insights Archives from insights-operator-76bcd6ccf5-nxgkj 2026-09-05T02:57:51.814593763Z insights-runtime-extractor-5jkdg 2026-09-05T02:57:51.814615358Z insights-runtime-extractor-p2t5f 2026-09-05T02:57:51.814635969Z insights-runtime-extractor-p5xgb 2026-09-05T02:57:51.832119968Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-05T02:57:51.941727348Z INFO: "metallb-operator" not detected. Skipping. 2026-09-05T02:57:51.957412135Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-05T02:57:51.986853729Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-05T02:57:52.234338993Z No resources found in openshift-etcd namespace. 2026-09-05T02:57:52.417617597Z INFO: Worker host service log collection to complete. 2026-09-05T02:57:52.535131094Z Wrote inspect data to must-gather. 2026-09-05T02:57:52.580146671Z ERROR: No running kube-apiserver pods found 2026-09-05T02:57:52.831920827Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-05T02:57:52.831920827Z See 'oc rsync -h' for help and examples 2026-09-05T02:57:52.840903034Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-05T02:57:52.871664609Z Wrote inspect data to must-gather. 2026-09-05T02:57:52.969378798Z Gathering data for ns/openshift-monitoring... 2026-09-05T02:57:53.015805218Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-05T02:57:53.062504415Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-05T02:57:53.072004031Z No resources found 2026-09-05T02:57:53.188094418Z Wrote inspect data to must-gather. 2026-09-05T02:57:53.197764967Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-05T02:57:53.203984271Z Wrote inspect data to must-gather. 2026-09-05T02:57:53.240727633Z Wrote inspect data to must-gather. 2026-09-05T02:57:53.268602999Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 2026-09-05T02:57:53.280941888Z error: the server doesn't have a resource type "machineconfigs" 2026-09-05T02:57:53.290821879Z Wrote inspect data to must-gather. 2026-09-05T02:57:53.360305832Z INFO: 'previous-logs' folder not found on ip-10-0-128-80.ec2.internal, skipping... 2026-09-05T02:57:53.405790220Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-05T02:57:53.411430410Z Wrote inspect data to must-gather. 2026-09-05T02:57:53.511374218Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-05T02:57:53.520157239Z tar: Removing leading `/' from member names 2026-09-05T02:57:53.528701989Z INFO: HAProxy config collection complete. 2026-09-05T02:57:53.647976105Z INFO: 'previous-logs' folder not found on ip-10-0-132-209.ec2.internal, skipping... 2026-09-05T02:57:53.683094534Z Wrote inspect data to must-gather. 2026-09-05T02:57:53.687950258Z INFO: Getting rules from prometheus-k8s-0 2026-09-05T02:57:53.749904196Z Wrote inspect data to must-gather. 2026-09-05T02:57:53.917584511Z INFO: 'previous-logs' folder not found on ip-10-0-133-229.ec2.internal, skipping... 2026-09-05T02:57:53.917610694Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-05T02:57:53.917627827Z INFO: Machine Config Daemon termination log collection complete. 2026-09-05T02:57:53.927499525Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-05T02:57:53.956170968Z Wrote inspect data to must-gather. 2026-09-05T02:57:54.077209884Z INFO: Getting status/config from prometheus-k8s-0 2026-09-05T02:57:54.136479702Z Wrote inspect data to must-gather. 2026-09-05T02:57:54.203574923Z Wrote inspect data to must-gather. 2026-09-05T02:57:54.357936279Z Wrote inspect data to must-gather. 2026-09-05T02:57:54.489212509Z INFO: INTERCONNECT MODE 2026-09-05T02:57:54.489259681Z INFO: Gathering ovn-kubernetes DBs 2026-09-05T02:57:54.492290953Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-05T02:57:54.579029496Z Wrote inspect data to must-gather. 2026-09-05T02:57:54.656281089Z INFO: Gathering OVN_Northbound from ovnkube-node-6gqnp... 2026-09-05T02:57:54.656323866Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-05T02:57:54.656566731Z INFO: Gathering OVN_Northbound from ovnkube-node-n68wh... 2026-09-05T02:57:54.656822650Z INFO: Gathering OVN_Northbound from ovnkube-node-qz6cj... 2026-09-05T02:57:54.657081387Z INFO: Gathering OVN_Southbound from ovnkube-node-6gqnp... 2026-09-05T02:57:54.657362251Z INFO: Gathering OVN_Southbound from ovnkube-node-n68wh... 2026-09-05T02:57:54.657667022Z INFO: Gathering OVN_Southbound from ovnkube-node-qz6cj... 2026-09-05T02:57:54.691615041Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-05T02:57:54.775505745Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-05T02:57:54.918096983Z Wrote inspect data to must-gather. 2026-09-05T02:57:54.976511064Z tar: Removing leading `/' from member names 2026-09-05T02:57:54.983925927Z tar: Removing leading `/' from member names 2026-09-05T02:57:55.010146434Z tar: Removing leading `/' from member names 2026-09-05T02:57:55.013661166Z tar: Removing leading `/' from member names 2026-09-05T02:57:55.028172858Z Wrote inspect data to /must-gather/istio. 2026-09-05T02:57:55.039883791Z tar: Removing leading `/' from member names 2026-09-05T02:57:55.077137921Z tar: Removing leading `/' from member names 2026-09-05T02:57:55.135069507Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-05T02:57:55.192187131Z Wrote inspect data to must-gather. 2026-09-05T02:57:55.281341348Z Wrote inspect data to /must-gather/istio. 2026-09-05T02:57:55.286181614Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-05T02:57:55.408661531Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-05T02:57:55.438566284Z W0905 02:57:55.438497 1459 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-09-05T02:57:55.443875918Z Wrote inspect data to must-gather. 2026-09-05T02:57:55.478552779Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-05T02:57:55.510517639Z Wrote inspect data to /must-gather/istio. 2026-09-05T02:57:55.519623143Z INFO: Waiting for network log collection to complete ... 2026-09-05T02:57:55.519737178Z INFO: Waiting for ovnk database copies to complete ... 2026-09-05T02:57:55.520229456Z INFO: Copying ovnk databases complete. 2026-09-05T02:57:55.522342848Z 2.8M must-gather/network_logs/ovnk_database_store 2026-09-05T02:57:55.525533132Z ovnk_database_store/ 2026-09-05T02:57:55.525533132Z ovnk_database_store/ovnkube-node-6gqnp_nbdb 2026-09-05T02:57:55.537136180Z ovnk_database_store/ovnkube-node-n68wh_sbdb 2026-09-05T02:57:55.557491258Z ovnk_database_store/ovnkube-node-6gqnp_sbdb 2026-09-05T02:57:55.574698689Z ovnk_database_store/ovnkube-node-qz6cj_sbdb 2026-09-05T02:57:55.593926029Z ovnk_database_store/ovnkube-node-n68wh_nbdb 2026-09-05T02:57:55.600066119Z ovnk_database_store/ovnkube-node-qz6cj_nbdb 2026-09-05T02:57:55.609186753Z INFO: Network log collection complete. 2026-09-05T02:57:55.673991885Z Gathering data for ns/openshift-network-console... 2026-09-05T02:57:55.733696173Z Wrote inspect data to /must-gather/istio. 2026-09-05T02:57:55.737415772Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-05T02:57:55.765942277Z Wrote inspect data to must-gather. 2026-09-05T02:57:55.815223534Z INFO: Getting status from alertmanager-main-0 2026-09-05T02:57:55.946024967Z Wrote inspect data to /must-gather/istio. 2026-09-05T02:57:56.048753053Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-05T02:57:56.051043367Z INFO: Fallback to identify the container image from release info 2026-09-05T02:57:56.136390790Z Gathering data for ns/openshift-console-operator... 2026-09-05T02:57:56.156101723Z Wrote inspect data to /must-gather/istio. 2026-09-05T02:57:56.158508314Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-05T02:57:56.369527712Z Wrote inspect data to /must-gather/istio. 2026-09-05T02:57:56.556301814Z Wrote inspect data to /must-gather/istio. 2026-09-05T02:57:56.558821855Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-05T02:57:56.564730908Z Gathering data for ns/openshift-console... 2026-09-05T02:57:56.618171481Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f39c1daa89e19c5e987a3b759e646ffab6c1b2ab94a97ded81ddf13e73d0399 2026-09-05T02:57:56.728256009Z Wrote inspect data to /must-gather/istio. 2026-09-05T02:57:56.749686532Z daemonset.apps/perf-node-gather-daemonset created 2026-09-05T02:57:56.876816985Z Waiting for performance profile collector pods to become ready: 1 2026-09-05T02:57:56.924571047Z Wrote inspect data to /must-gather/istio. 2026-09-05T02:57:57.107808237Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-05T02:57:57.595388432Z Done executing Istio gather script 2026-09-05T02:57:57.633812112Z Gathering data for ns/openshift-dns-operator... 2026-09-05T02:57:57.902003323Z Gathering data for ns/openshift-dns... 2026-09-05T02:57:57.985201128Z Waiting for performance profile collector pods to become ready: 2 2026-09-05T02:57:58.296262453Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 2026-09-05T02:57:58.479302359Z Gathering data for ns/openshift-image-registry... 2026-09-05T02:57:58.917783294Z Gathering data for ns/openshift-ingress-operator... 2026-09-05T02:57:59.085186918Z Waiting for performance profile collector pods to become ready: 3 2026-09-05T02:57:59.259024272Z Gathering data for ns/openshift-ingress... 2026-09-05T02:57:59.565976426Z Gathering data for ns/openshift-ingress-canary... 2026-09-05T02:57:59.941470911Z Gathering data for ns/openshift-insights... 2026-09-05T02:58:00.188958276Z Waiting for performance profile collector pods to become ready: 4 2026-09-05T02:58:01.301938750Z Waiting for performance profile collector pods to become ready: 5 2026-09-05T02:58:01.705967573Z Gathering data for ns/openshift-jobset-operator... 2026-09-05T02:58:02.409941467Z Waiting for performance profile collector pods to become ready: 6 2026-09-05T02:58:02.482197000Z Gathering data for ns/openshift-config... 2026-09-05T02:58:02.739541694Z Gathering data for ns/openshift-config-managed... 2026-09-05T02:58:03.138848543Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-05T02:58:03.301192011Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 2026-09-05T02:58:03.384210037Z Gathering data for ns/openshift-kube-apiserver... 2026-09-05T02:58:03.538081180Z Waiting for performance profile collector pods to become ready: 7 2026-09-05T02:58:03.642835219Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-05T02:58:03.910166941Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-05T02:58:04.182953058Z Gathering data for ns/openshift-kube-scheduler... 2026-09-05T02:58:04.418407757Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-05T02:58:04.653197895Z Waiting for performance profile collector pods to become ready: 8 2026-09-05T02:58:04.737089083Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-05T02:58:05.019522604Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-05T02:58:05.303301234Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-05T02:58:05.759024591Z Gathering data for ns/openshift-multus... 2026-09-05T02:58:05.772974999Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-05T02:58:05.984309221Z Collecting performance related data for node ip-10-0-132-209.ec2.internal 2026-09-05T02:58:05.991143498Z Collecting performance related data for node ip-10-0-133-229.ec2.internal 2026-09-05T02:58:05.998104695Z Collecting performance related data for node ip-10-0-128-80.ec2.internal 2026-09-05T02:58:07.249630452Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-05T02:58:08.312567284Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 2026-09-05T02:58:08.437746784Z Gathering data for ns/openshift-host-network... 2026-09-05T02:58:08.804354433Z Gathering data for ns/openshift-network-diagnostics... 2026-09-05T02:58:09.299081889Z Gathering data for ns/openshift-network-node-identity... 2026-09-05T02:58:09.770098463Z Gathering data for ns/openshift-network-operator... 2026-09-05T02:58:10.135621741Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-05T02:58:10.410902617Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-05T02:58:10.890355532Z Gathering data for ns/openshift-apiserver-operator... 2026-09-05T02:58:11.194352366Z Gathering data for ns/openshift-apiserver... 2026-09-05T02:58:11.578546113Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-05T02:58:11.831991167Z Gathering data for ns/openshift-controller-manager... 2026-09-05T02:58:12.089283004Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-05T02:58:12.692091931Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-05T02:58:13.025431081Z Gathering data for ns/openshift-service-ca-operator... 2026-09-05T02:58:13.318866196Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 2026-09-05T02:58:13.327138105Z Gathering data for ns/openshift-service-ca... 2026-09-05T02:58:13.666001197Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-05T02:58:13.917136875Z Collecting kubelet logs for node ip-10-0-132-209.ec2.internal 2026-09-05T02:58:13.918852762Z Collecting kubelet logs for node ip-10-0-133-229.ec2.internal 2026-09-05T02:58:13.920641387Z Collecting kubelet logs for node ip-10-0-128-80.ec2.internal 2026-09-05T02:58:14.244280591Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-x4ppl namespace 2026-09-05T02:58:14.250302969Z INFO: Node performance data collection complete. 2026-09-05T02:58:14.588066553Z Wrote inspect data to must-gather. 2026-09-05T02:58:14.588110268Z error: inspection completed with the errors occurred while gathering data: 2026-09-05T02:58:14.588110268Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-05T02:58:14.711719849Z error: the server doesn't have a resource type "clusters" 2026-09-05T02:58:14.839928819Z Caches written to disk