2026-06-11T23:38:47.948390568Z [disk usage checker] Started 2026-06-11T23:38:47.951719894Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-06-11T23:38:48.142396279Z Gathering data for ns/openshift-cluster-version... 2026-06-11T23:38:48.267892081Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-06-11T23:38:48.344839995Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-06-11T23:38:48.416784269Z Gathering data for ns/default... 2026-06-11T23:38:48.720667431Z Gathering data for ns/openshift... 2026-06-11T23:38:49.093910372Z Gathering data for ns/kube-system... 2026-06-11T23:38:49.737037260Z Gathering data for ns/openshift-etcd... 2026-06-11T23:38:50.013818166Z Wrote inspect data to must-gather. 2026-06-11T23:38:50.013854470Z error: inspection completed with the errors occurred while gathering data: 2026-06-11T23:38:50.013854470Z namespaces "assisted-installer" not found 2026-06-11T23:38:50.413083892Z Waiting on subprocesses to finish execution. 2026-06-11T23:38:50.418022321Z INFO: Gathering machine config daemon's old logs from all nodes 2026-06-11T23:38:50.423225844Z INFO: Collecting host service logs for crio 2026-06-11T23:38:50.423603777Z INFO: Collecting host service logs for kubelet 2026-06-11T23:38:50.423986910Z INFO: Collecting host service logs for rpm-ostreed 2026-06-11T23:38:50.424353338Z INFO: Collecting host service logs for ostree-finalize-staged 2026-06-11T23:38:50.424724824Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-06-11T23:38:50.425161500Z INFO: Gathering HAProxy config files 2026-06-11T23:38:50.425362435Z INFO: Collecting host service logs for machine-config-daemon-host 2026-06-11T23:38:50.425488776Z INFO: Collecting host service logs for NetworkManager 2026-06-11T23:38:50.425798759Z INFO: Collecting host service logs for openvswitch 2026-06-11T23:38:50.426114719Z INFO: Collecting host service logs for ovs-configuration 2026-06-11T23:38:50.426425606Z INFO: Collecting host service logs for ovsdb-server 2026-06-11T23:38:50.426727519Z INFO: Collecting host service logs for ovs-vswitchd 2026-06-11T23:38:50.427187998Z INFO: Waiting for worker host service log collection to complete ... 2026-06-11T23:38:50.434896708Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-06-11T23:38:50.437299271Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-06-11T23:38:50.445241083Z Executing Istio gather script 2026-06-11T23:38:50.621439042Z INFO: Waiting for node performance related collection to complete ... 2026-06-11T23:38:50.977083481Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-06-11T23:38:51.084319803Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-06-11T23:38:51.084353902Z INFO: on-disk MachineConfig config collection complete. 2026-06-11T23:38:51.225671266Z No resources found in openshift-etcd namespace. 2026-06-11T23:38:51.302629195Z INFO: Collecting Insights Archives from insights-operator-5bbd86d6bd-pqv8m 2026-06-11T23:38:51.302674546Z insights-runtime-extractor-7m4z2 2026-06-11T23:38:51.302695687Z insights-runtime-extractor-tmjj2 2026-06-11T23:38:51.302715299Z insights-runtime-extractor-wz2gd 2026-06-11T23:38:51.364942739Z No resources found 2026-06-11T23:38:51.524799225Z INFO: Found 1 replicas - prometheus-k8s-0 2026-06-11T23:38:51.569992306Z Wrote inspect data to must-gather. 2026-06-11T23:38:51.781111780Z INFO: "sriov-network-operator" not detected. Skipping. 2026-06-11T23:38:51.811071842Z error: the server doesn't have a resource type "performanceprofile" 2026-06-11T23:38:51.910899017Z INFO: Worker host service log collection to complete. 2026-06-11T23:38:51.955918259Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-06-11T23:38:51.955918259Z See 'oc rsync -h' for help and examples 2026-06-11T23:38:52.066482633Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-06-11T23:38:52.101331298Z INFO: "metallb-operator" not detected. Skipping. 2026-06-11T23:38:52.115796652Z No resources found 2026-06-11T23:38:52.258334471Z INFO: Waiting for HAProxy config collection to complete ... 2026-06-11T23:38:52.296752957Z Wrote inspect data to must-gather. 2026-06-11T23:38:52.465527143Z Wrote inspect data to must-gather. 2026-06-11T23:38:52.510944398Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-06-11T23:38:52.569034744Z Wrote inspect data to must-gather. 2026-06-11T23:38:52.596217742Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-06-11T23:38:52.631313579Z Wrote inspect data to must-gather. 2026-06-11T23:38:52.686337744Z INFO: 'previous-logs' folder not found on ip-10-0-134-32.ec2.internal, skipping... 2026-06-11T23:38:52.732780023Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-06-11T23:38:52.745644950Z Wrote inspect data to must-gather. 2026-06-11T23:38:52.750327409Z ERROR: No running kube-apiserver pods found 2026-06-11T23:38:52.806188968Z Wrote inspect data to must-gather. 2026-06-11T23:38:52.902481704Z Gathering data for ns/openshift-monitoring... 2026-06-11T23:38:52.957150772Z tar: Removing leading `/' from member names 2026-06-11T23:38:52.965418945Z INFO: HAProxy config collection complete. 2026-06-11T23:38:52.965490690Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-06-11T23:38:52.999098461Z Wrote inspect data to must-gather. 2026-06-11T23:38:53.005744474Z error: the server doesn't have a resource type "machineconfigs" 2026-06-11T23:38:53.049447194Z INFO: 'previous-logs' folder not found on ip-10-0-137-2.ec2.internal, skipping... 2026-06-11T23:38:53.189490215Z error: the server doesn't have a resource type "machineconfigpools" 2026-06-11T23:38:53.212034536Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-06-11T23:38:53.245466631Z INFO: Getting rules from prometheus-k8s-0 2026-06-11T23:38:53.278830811Z Wrote inspect data to must-gather. 2026-06-11T23:38:53.333289504Z INFO: 'previous-logs' folder not found on ip-10-0-138-124.ec2.internal, skipping... 2026-06-11T23:38:53.333326074Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-06-11T23:38:53.333336868Z INFO: Machine Config Daemon termination log collection complete. 2026-06-11T23:38:53.421261967Z Wrote inspect data to must-gather. 2026-06-11T23:38:53.509299185Z Wrote inspect data to must-gather. 2026-06-11T23:38:53.570988117Z error: the server doesn't have a resource type "kubeletconfigs" 2026-06-11T23:38:53.629011761Z INFO: Getting status/config from prometheus-k8s-0 2026-06-11T23:38:53.781211486Z INFO: INTERCONNECT MODE 2026-06-11T23:38:53.781240593Z INFO: Gathering ovn-kubernetes DBs 2026-06-11T23:38:53.790224746Z Wrote inspect data to must-gather. 2026-06-11T23:38:53.943193983Z INFO: Gathering OVN_Northbound from ovnkube-node-7kvm8... 2026-06-11T23:38:53.943458035Z INFO: Gathering OVN_Northbound from ovnkube-node-ltcdm... 2026-06-11T23:38:53.943745772Z INFO: Gathering OVN_Northbound from ovnkube-node-pqqwb... 2026-06-11T23:38:53.944030155Z INFO: Gathering OVN_Southbound from ovnkube-node-7kvm8... 2026-06-11T23:38:53.944550615Z INFO: Gathering OVN_Southbound from ovnkube-node-ltcdm... 2026-06-11T23:38:53.944839652Z INFO: Gathering OVN_Southbound from ovnkube-node-pqqwb... 2026-06-11T23:38:53.978244463Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-06-11T23:38:54.015811678Z Wrote inspect data to must-gather. 2026-06-11T23:38:54.115428138Z INFO: Getting status/flags from prometheus-k8s-0 2026-06-11T23:38:54.213581816Z tar: Removing leading `/' from member names 2026-06-11T23:38:54.246199448Z tar: Removing leading `/' from member names 2026-06-11T23:38:54.252992768Z tar: Removing leading `/' from member names 2026-06-11T23:38:54.274255854Z tar: Removing leading `/' from member names 2026-06-11T23:38:54.287250941Z tar: Removing leading `/' from member names 2026-06-11T23:38:54.305954680Z tar: Removing leading `/' from member names 2026-06-11T23:38:54.328745286Z Wrote inspect data to /must-gather/istio. 2026-06-11T23:38:54.354576634Z Wrote inspect data to must-gather. 2026-06-11T23:38:54.410612300Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-06-11T23:38:54.432433481Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-06-11T23:38:54.563453464Z Wrote inspect data to /must-gather/istio. 2026-06-11T23:38:54.566072336Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-06-11T23:38:54.607631097Z Wrote inspect data to must-gather. 2026-06-11T23:38:54.692274585Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-06-11T23:38:54.760055942Z INFO: Waiting for network log collection to complete ... 2026-06-11T23:38:54.760096773Z INFO: Waiting for ovnk database copies to complete ... 2026-06-11T23:38:54.760586143Z INFO: Copying ovnk databases complete. 2026-06-11T23:38:54.762643540Z 3.7M must-gather/network_logs/ovnk_database_store 2026-06-11T23:38:54.765096541Z ovnk_database_store/ 2026-06-11T23:38:54.765096541Z ovnk_database_store/ovnkube-node-7kvm8_nbdb 2026-06-11T23:38:54.774016652Z Wrote inspect data to /must-gather/istio. 2026-06-11T23:38:54.776802715Z ovnk_database_store/ovnkube-node-pqqwb_sbdb 2026-06-11T23:38:54.799508601Z ovnk_database_store/ovnkube-node-pqqwb_nbdb 2026-06-11T23:38:54.808753109Z ovnk_database_store/ovnkube-node-ltcdm_nbdb 2026-06-11T23:38:54.818321714Z ovnk_database_store/ovnkube-node-7kvm8_sbdb 2026-06-11T23:38:54.842643593Z ovnk_database_store/ovnkube-node-ltcdm_sbdb 2026-06-11T23:38:54.849431163Z Wrote inspect data to must-gather. 2026-06-11T23:38:54.869011498Z INFO: Network log collection complete. 2026-06-11T23:38:54.926400386Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-06-11T23:38:54.967253959Z Wrote inspect data to /must-gather/istio. 2026-06-11T23:38:54.971154764Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-06-11T23:38:55.069397808Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-06-11T23:38:55.151331429Z Wrote inspect data to must-gather. 2026-06-11T23:38:55.173478972Z Wrote inspect data to /must-gather/istio. 2026-06-11T23:38:55.252051533Z Gathering data for ns/openshift-network-console... 2026-06-11T23:38:55.303708708Z INFO: Getting status from alertmanager-main-0 2026-06-11T23:38:55.392534255Z Wrote inspect data to /must-gather/istio. 2026-06-11T23:38:55.394668003Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-06-11T23:38:55.456704382Z Wrote inspect data to must-gather. 2026-06-11T23:38:55.618646678Z Wrote inspect data to /must-gather/istio. 2026-06-11T23:38:55.696102138Z Gathering data for ns/openshift-console-operator... 2026-06-11T23:38:55.711396922Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-06-11T23:38:55.714229534Z INFO: Fallback to identify the container image from release info 2026-06-11T23:38:55.812585697Z Wrote inspect data to /must-gather/istio. 2026-06-11T23:38:55.814569151Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-06-11T23:38:55.959464936Z Wrote inspect data to /must-gather/istio. 2026-06-11T23:38:56.058835898Z Gathering data for ns/openshift-console... 2026-06-11T23:38:56.143773922Z Wrote inspect data to /must-gather/istio. 2026-06-11T23:38:56.552688074Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d66a6b2ec52dede5d83ce04f6abfc519df1fd7dc36248741e03ecb9bf8d1762 2026-06-11T23:38:56.570258780Z Gathering data for ns/openshift-cluster-storage-operator... 2026-06-11T23:38:56.688363769Z daemonset.apps/perf-node-gather-daemonset created 2026-06-11T23:38:56.805655655Z Done executing Istio gather script 2026-06-11T23:38:56.814173327Z Waiting for performance profile collector pods to become ready: 1 2026-06-11T23:38:57.047990886Z Gathering data for ns/openshift-dns-operator... 2026-06-11T23:38:57.344144254Z Gathering data for ns/openshift-dns... 2026-06-11T23:38:57.930968601Z Waiting for performance profile collector pods to become ready: 2 2026-06-11T23:38:57.973394289Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-06-11T23:38:57.974033189Z Gathering data for ns/openshift-image-registry... 2026-06-11T23:38:58.397749157Z Gathering data for ns/openshift-ingress-operator... 2026-06-11T23:38:58.682810334Z Gathering data for ns/openshift-ingress... 2026-06-11T23:38:58.996352476Z Gathering data for ns/openshift-ingress-canary... 2026-06-11T23:38:59.042034010Z Waiting for performance profile collector pods to become ready: 3 2026-06-11T23:38:59.367987793Z Gathering data for ns/openshift-insights... 2026-06-11T23:39:00.146705250Z Waiting for performance profile collector pods to become ready: 4 2026-06-11T23:39:01.252976045Z Waiting for performance profile collector pods to become ready: 5 2026-06-11T23:39:01.325541260Z Gathering data for ns/openshift-config... 2026-06-11T23:39:01.588204249Z Gathering data for ns/openshift-config-managed... 2026-06-11T23:39:02.099433947Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-06-11T23:39:02.370210137Z Waiting for performance profile collector pods to become ready: 6 2026-06-11T23:39:02.375973659Z Gathering data for ns/openshift-kube-apiserver... 2026-06-11T23:39:02.615096321Z Gathering data for ns/openshift-kube-controller-manager... 2026-06-11T23:39:02.875907016Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-06-11T23:39:02.977963368Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-06-11T23:39:03.122102639Z Gathering data for ns/openshift-kube-scheduler... 2026-06-11T23:39:03.383533957Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-06-11T23:39:03.486302176Z Waiting for performance profile collector pods to become ready: 7 2026-06-11T23:39:03.655265388Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-06-11T23:39:03.953081796Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-06-11T23:39:04.305806553Z Gathering data for ns/openshift-user-workload-monitoring... 2026-06-11T23:39:04.601924913Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-06-11T23:39:04.824144770Z Gathering data for ns/openshift-multus... 2026-06-11T23:39:04.851147486Z Collecting performance related data for node ip-10-0-137-2.ec2.internal 2026-06-11T23:39:04.857731289Z Collecting performance related data for node ip-10-0-134-32.ec2.internal 2026-06-11T23:39:04.864785116Z Collecting performance related data for node ip-10-0-138-124.ec2.internal 2026-06-11T23:39:06.366849896Z Gathering data for ns/openshift-ovn-kubernetes... 2026-06-11T23:39:07.544055330Z Gathering data for ns/openshift-host-network... 2026-06-11T23:39:07.887700571Z Gathering data for ns/openshift-network-diagnostics... 2026-06-11T23:39:07.982547136Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-06-11T23:39:08.378820893Z Gathering data for ns/openshift-network-node-identity... 2026-06-11T23:39:08.782092084Z Gathering data for ns/openshift-network-operator... 2026-06-11T23:39:09.134198136Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-06-11T23:39:09.407289818Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-06-11T23:39:09.905591902Z Gathering data for ns/openshift-apiserver-operator... 2026-06-11T23:39:10.192170923Z Gathering data for ns/openshift-apiserver... 2026-06-11T23:39:10.549140169Z Gathering data for ns/openshift-controller-manager-operator... 2026-06-11T23:39:10.790409972Z Gathering data for ns/openshift-controller-manager... 2026-06-11T23:39:11.037126905Z Gathering data for ns/openshift-cluster-samples-operator... 2026-06-11T23:39:11.628421578Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-06-11T23:39:12.011336511Z Gathering data for ns/openshift-service-ca-operator... 2026-06-11T23:39:12.350546712Z Gathering data for ns/openshift-service-ca... 2026-06-11T23:39:12.637541997Z Collecting kubelet logs for node ip-10-0-137-2.ec2.internal 2026-06-11T23:39:12.639398762Z Collecting kubelet logs for node ip-10-0-134-32.ec2.internal 2026-06-11T23:39:12.641165604Z Collecting kubelet logs for node ip-10-0-138-124.ec2.internal 2026-06-11T23:39:12.753855524Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-06-11T23:39:12.966417621Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-mj9zp namespace 2026-06-11T23:39:12.971251060Z INFO: Node performance data collection complete. 2026-06-11T23:39:12.987808215Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-06-11T23:39:13.716943591Z Wrote inspect data to must-gather. 2026-06-11T23:39:13.716988340Z error: inspection completed with the errors occurred while gathering data: 2026-06-11T23:39:13.716988340Z skipping gathering secrets/support due to error: secrets "support" not found 2026-06-11T23:39:13.836645843Z error: the server doesn't have a resource type "clusters" 2026-06-11T23:39:13.964591193Z Caches written to disk