2026-07-08T15:24:21.307389393Z [disk usage checker] Started 2026-07-08T15:24:21.311278303Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-08T15:24:21.549788298Z Gathering data for ns/openshift-cluster-version... 2026-07-08T15:24:21.791272305Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-08T15:24:21.875761192Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-08T15:24:21.950012298Z Gathering data for ns/default... 2026-07-08T15:24:22.217556612Z Gathering data for ns/openshift... 2026-07-08T15:24:22.633121356Z Gathering data for ns/kube-system... 2026-07-08T15:24:23.346637889Z Gathering data for ns/openshift-etcd... 2026-07-08T15:24:23.623558615Z Wrote inspect data to must-gather. 2026-07-08T15:24:23.623586876Z error: inspection completed with the errors occurred while gathering data: 2026-07-08T15:24:23.623586876Z namespaces "assisted-installer" not found 2026-07-08T15:24:23.869346280Z Waiting on subprocesses to finish execution. 2026-07-08T15:24:23.872880970Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-07-08T15:24:23.877412494Z INFO: Gathering machine config daemon's old logs from all nodes 2026-07-08T15:24:23.880782277Z Executing Istio gather script 2026-07-08T15:24:23.880920826Z INFO: Gathering HAProxy config files 2026-07-08T15:24:23.903964221Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-07-08T15:24:23.904478923Z INFO: Collecting host service logs for crio 2026-07-08T15:24:23.904820550Z INFO: Collecting host service logs for kubelet 2026-07-08T15:24:23.905191420Z INFO: Collecting host service logs for rpm-ostreed 2026-07-08T15:24:23.905549002Z INFO: Collecting host service logs for ostree-finalize-staged 2026-07-08T15:24:23.905918387Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-07-08T15:24:23.906300437Z INFO: Collecting host service logs for machine-config-daemon-host 2026-07-08T15:24:23.906680615Z INFO: Collecting host service logs for NetworkManager 2026-07-08T15:24:23.907052395Z INFO: Collecting host service logs for openvswitch 2026-07-08T15:24:23.907536260Z INFO: Collecting host service logs for ovs-configuration 2026-07-08T15:24:23.907898810Z INFO: Collecting host service logs for ovsdb-server 2026-07-08T15:24:23.908286436Z INFO: Collecting host service logs for ovs-vswitchd 2026-07-08T15:24:23.908706428Z INFO: Waiting for worker host service log collection to complete ... 2026-07-08T15:24:23.986791159Z INFO: Waiting for node performance related collection to complete ... 2026-07-08T15:24:24.477896472Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-07-08T15:24:24.490425567Z INFO: "sriov-network-operator" not detected. Skipping. 2026-07-08T15:24:24.702581979Z Wrote inspect data to must-gather. 2026-07-08T15:24:24.752716593Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-07-08T15:24:24.872157307Z INFO: "metallb-operator" not detected. Skipping. 2026-07-08T15:24:24.938974508Z INFO: Found 1 replicas - prometheus-k8s-0 2026-07-08T15:24:24.957418138Z INFO: Collecting Insights Archives from insights-operator-5c847dcfc4-685lv 2026-07-08T15:24:24.957466784Z insights-runtime-extractor-5w2lk 2026-07-08T15:24:24.957486983Z insights-runtime-extractor-bqxzr 2026-07-08T15:24:24.957506584Z insights-runtime-extractor-ckgs5 2026-07-08T15:24:25.205911853Z No resources found 2026-07-08T15:24:25.231646461Z error: the server doesn't have a resource type "performanceprofile" 2026-07-08T15:24:25.239274563Z INFO: Waiting for HAProxy config collection to complete ... 2026-07-08T15:24:25.273344072Z INFO: Worker host service log collection to complete. 2026-07-08T15:24:25.494925939Z INFO: 'previous-logs' folder not found on ip-10-0-136-156.ec2.internal, skipping... 2026-07-08T15:24:25.511990244Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-07-08T15:24:25.618433395Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-07-08T15:24:25.618433395Z See 'oc rsync -h' for help and examples 2026-07-08T15:24:25.627992469Z Wrote inspect data to must-gather. 2026-07-08T15:24:25.799150456Z Wrote inspect data to must-gather. 2026-07-08T15:24:25.812335908Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-07-08T15:24:25.812408365Z INFO: on-disk MachineConfig config collection complete. 2026-07-08T15:24:25.900102250Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-08T15:24:25.918313103Z No resources found 2026-07-08T15:24:25.963658281Z Wrote inspect data to must-gather. 2026-07-08T15:24:25.964298117Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-07-08T15:24:25.978464132Z ERROR: No running kube-apiserver pods found 2026-07-08T15:24:25.991898786Z No resources found in openshift-etcd namespace. 2026-07-08T15:24:26.028796410Z Wrote inspect data to must-gather. 2026-07-08T15:24:26.142526078Z Wrote inspect data to must-gather. 2026-07-08T15:24:26.263849449Z Wrote inspect data to must-gather. 2026-07-08T15:24:26.312809818Z tar: Removing leading `/' from member names 2026-07-08T15:24:26.319202164Z INFO: HAProxy config collection complete. 2026-07-08T15:24:26.319699144Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-08T15:24:26.328753956Z Gathering data for ns/openshift-monitoring... 2026-07-08T15:24:26.339578959Z INFO: 'previous-logs' folder not found on ip-10-0-139-120.ec2.internal, skipping... 2026-07-08T15:24:26.679850597Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-07-08T15:24:31.324586289Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-08T15:24:32.810678052Z Wrote inspect data to must-gather. 2026-07-08T15:24:32.986425421Z Wrote inspect data to must-gather. 2026-07-08T15:24:33.160533195Z Wrote inspect data to must-gather. 2026-07-08T15:24:33.373463432Z INFO: INTERCONNECT MODE 2026-07-08T15:24:33.373502351Z INFO: Gathering ovn-kubernetes DBs 2026-07-08T15:24:33.497712953Z INFO: Gathering OVN_Northbound from ovnkube-node-69vr2... 2026-07-08T15:24:33.497930394Z INFO: Gathering OVN_Northbound from ovnkube-node-wc4qc... 2026-07-08T15:24:33.498173557Z INFO: Gathering OVN_Northbound from ovnkube-node-xrlcm... 2026-07-08T15:24:33.498456347Z INFO: Gathering OVN_Southbound from ovnkube-node-69vr2... 2026-07-08T15:24:33.498750150Z INFO: Gathering OVN_Southbound from ovnkube-node-wc4qc... 2026-07-08T15:24:33.499004056Z INFO: Gathering OVN_Southbound from ovnkube-node-xrlcm... 2026-07-08T15:24:33.610439789Z error: the server doesn't have a resource type "machineconfigs" 2026-07-08T15:24:33.669962800Z INFO: Getting rules from prometheus-k8s-0 2026-07-08T15:24:33.741401264Z tar: Removing leading `/' from member names 2026-07-08T15:24:33.761269294Z tar: Removing leading `/' from member names 2026-07-08T15:24:33.784820943Z tar: Removing leading `/' from member names 2026-07-08T15:24:33.786342933Z tar: Removing leading `/' from member names 2026-07-08T15:24:33.794312922Z tar: Removing leading `/' from member names 2026-07-08T15:24:33.798877553Z INFO: 'previous-logs' folder not found on ip-10-0-141-42.ec2.internal, skipping... 2026-07-08T15:24:33.799467950Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-07-08T15:24:33.799467950Z INFO: Machine Config Daemon termination log collection complete. 2026-07-08T15:24:33.814885972Z tar: Removing leading `/' from member names 2026-07-08T15:24:33.845966650Z error: the server doesn't have a resource type "machineconfigpools" 2026-07-08T15:24:34.082805910Z Wrote inspect data to must-gather. 2026-07-08T15:24:34.122503975Z INFO: Getting status/config from prometheus-k8s-0 2026-07-08T15:24:34.238935450Z error: the server doesn't have a resource type "kubeletconfigs" 2026-07-08T15:24:34.261280685Z INFO: Waiting for network log collection to complete ... 2026-07-08T15:24:34.261324674Z INFO: Waiting for ovnk database copies to complete ... 2026-07-08T15:24:34.261809008Z INFO: Copying ovnk databases complete. 2026-07-08T15:24:34.263718925Z 2.5M must-gather/network_logs/ovnk_database_store 2026-07-08T15:24:34.265984525Z ovnk_database_store/ 2026-07-08T15:24:34.266024291Z ovnk_database_store/ovnkube-node-wc4qc_nbdb 2026-07-08T15:24:34.273768196Z ovnk_database_store/ovnkube-node-69vr2_nbdb 2026-07-08T15:24:34.281429036Z ovnk_database_store/ovnkube-node-xrlcm_sbdb 2026-07-08T15:24:34.298893988Z ovnk_database_store/ovnkube-node-wc4qc_sbdb 2026-07-08T15:24:34.314764734Z ovnk_database_store/ovnkube-node-69vr2_sbdb 2026-07-08T15:24:34.331181569Z ovnk_database_store/ovnkube-node-xrlcm_nbdb 2026-07-08T15:24:34.463556438Z Wrote inspect data to must-gather. 2026-07-08T15:24:34.480270137Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-07-08T15:24:34.543987879Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-08T15:24:34.572392183Z INFO: Getting status/flags from prometheus-k8s-0 2026-07-08T15:24:34.654424189Z Wrote inspect data to /must-gather/istio. 2026-07-08T15:24:34.658008317Z Wrote inspect data to must-gather. 2026-07-08T15:24:34.782228068Z INFO: Network log collection complete. 2026-07-08T15:24:34.814143929Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-07-08T15:24:34.862482869Z Wrote inspect data to /must-gather/istio. 2026-07-08T15:24:34.865927320Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-07-08T15:24:34.882499413Z Wrote inspect data to must-gather. 2026-07-08T15:24:35.040319487Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-07-08T15:24:35.055375841Z Wrote inspect data to /must-gather/istio. 2026-07-08T15:24:35.091185155Z W0708 15:24:35.091119 1458 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-07-08T15:24:35.097001913Z Wrote inspect data to must-gather. 2026-07-08T15:24:35.178693785Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-08T15:24:35.264310626Z Wrote inspect data to /must-gather/istio. 2026-07-08T15:24:35.267414848Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-07-08T15:24:35.283539544Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-07-08T15:24:35.366263758Z Gathering data for ns/openshift-network-console... 2026-07-08T15:24:35.413203783Z Wrote inspect data to must-gather. 2026-07-08T15:24:35.474850977Z Wrote inspect data to /must-gather/istio. 2026-07-08T15:24:35.675165033Z INFO: Getting status from alertmanager-main-0 2026-07-08T15:24:35.687904375Z Wrote inspect data to /must-gather/istio. 2026-07-08T15:24:35.691701385Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-07-08T15:24:35.748366909Z Wrote inspect data to must-gather. 2026-07-08T15:24:35.831518379Z Gathering data for ns/openshift-console-operator... 2026-07-08T15:24:35.946409966Z Wrote inspect data to /must-gather/istio. 2026-07-08T15:24:36.045624213Z Wrote inspect data to must-gather. 2026-07-08T15:24:36.122253567Z Wrote inspect data to /must-gather/istio. 2026-07-08T15:24:36.125797873Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-07-08T15:24:36.228436998Z Gathering data for ns/openshift-console... 2026-07-08T15:24:36.294583784Z Wrote inspect data to /must-gather/istio. 2026-07-08T15:24:36.298219476Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-07-08T15:24:36.300917302Z INFO: Fallback to identify the container image from release info 2026-07-08T15:24:36.332286864Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-08T15:24:36.489819356Z Wrote inspect data to /must-gather/istio. 2026-07-08T15:24:36.740115795Z Gathering data for ns/openshift-cluster-storage-operator... 2026-07-08T15:24:37.109088993Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5be8cdcf34f35a89a70b6bfa49e66897641871237570b3e09766f83a0a473e6a 2026-07-08T15:24:37.157638297Z Done executing Istio gather script 2026-07-08T15:24:37.242234416Z daemonset.apps/perf-node-gather-daemonset created 2026-07-08T15:24:37.251340317Z Gathering data for ns/openshift-dns-operator... 2026-07-08T15:24:37.368768812Z Waiting for performance profile collector pods to become ready: 1 2026-07-08T15:24:37.553861321Z Gathering data for ns/openshift-dns... 2026-07-08T15:24:38.134124245Z Gathering data for ns/openshift-image-registry... 2026-07-08T15:24:38.495276113Z Waiting for performance profile collector pods to become ready: 2 2026-07-08T15:24:38.628817163Z Gathering data for ns/openshift-ingress-operator... 2026-07-08T15:24:38.954955189Z Gathering data for ns/openshift-ingress... 2026-07-08T15:24:39.283862174Z Gathering data for ns/openshift-ingress-canary... 2026-07-08T15:24:39.622230825Z Waiting for performance profile collector pods to become ready: 3 2026-07-08T15:24:39.675388778Z Gathering data for ns/openshift-insights... 2026-07-08T15:24:40.731447790Z Waiting for performance profile collector pods to become ready: 4 2026-07-08T15:24:41.341159670Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-08T15:24:41.494831359Z Gathering data for ns/openshift-config... 2026-07-08T15:24:41.749489401Z Gathering data for ns/openshift-config-managed... 2026-07-08T15:24:41.847786375Z Waiting for performance profile collector pods to become ready: 5 2026-07-08T15:24:42.136151350Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-07-08T15:24:42.369591440Z Gathering data for ns/openshift-kube-apiserver... 2026-07-08T15:24:42.603435536Z Gathering data for ns/openshift-kube-controller-manager... 2026-07-08T15:24:42.864435031Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-07-08T15:24:42.965751533Z Waiting for performance profile collector pods to become ready: 6 2026-07-08T15:24:43.141056203Z Gathering data for ns/openshift-kube-scheduler... 2026-07-08T15:24:43.364111038Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-07-08T15:24:43.623942057Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-07-08T15:24:43.926100463Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-07-08T15:24:44.079451482Z Waiting for performance profile collector pods to become ready: 7 2026-07-08T15:24:44.229426083Z Gathering data for ns/openshift-user-workload-monitoring... 2026-07-08T15:24:44.670305639Z Gathering data for ns/openshift-multus... 2026-07-08T15:24:45.178218466Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-07-08T15:24:45.402258194Z Collecting performance related data for node ip-10-0-141-42.ec2.internal 2026-07-08T15:24:45.415187148Z Collecting performance related data for node ip-10-0-136-156.ec2.internal 2026-07-08T15:24:45.422477446Z Collecting performance related data for node ip-10-0-139-120.ec2.internal 2026-07-08T15:24:46.105225302Z Gathering data for ns/openshift-ovn-kubernetes... 2026-07-08T15:24:46.345611370Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-08T15:24:47.333166875Z Gathering data for ns/openshift-host-network... 2026-07-08T15:24:47.642796819Z Gathering data for ns/openshift-network-diagnostics... 2026-07-08T15:24:48.216522836Z Gathering data for ns/openshift-network-node-identity... 2026-07-08T15:24:48.588518391Z Gathering data for ns/openshift-network-operator... 2026-07-08T15:24:48.941910058Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-07-08T15:24:49.231821538Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-07-08T15:24:49.677922771Z Gathering data for ns/openshift-apiserver-operator... 2026-07-08T15:24:49.951956420Z Gathering data for ns/openshift-apiserver... 2026-07-08T15:24:50.332179953Z Gathering data for ns/openshift-controller-manager-operator... 2026-07-08T15:24:50.617979376Z Gathering data for ns/openshift-controller-manager... 2026-07-08T15:24:50.957549145Z Gathering data for ns/openshift-cluster-samples-operator... 2026-07-08T15:24:51.350020462Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-08T15:24:51.585079807Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-07-08T15:24:51.871311179Z Gathering data for ns/openshift-service-ca-operator... 2026-07-08T15:24:52.164688173Z Gathering data for ns/openshift-service-ca... 2026-07-08T15:24:52.495017064Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-07-08T15:24:53.351576334Z Wrote inspect data to must-gather. 2026-07-08T15:24:53.351621385Z error: inspection completed with the errors occurred while gathering data: 2026-07-08T15:24:53.351621385Z skipping gathering secrets/support due to error: secrets "support" not found 2026-07-08T15:24:53.390525463Z Collecting kubelet logs for node ip-10-0-141-42.ec2.internal 2026-07-08T15:24:53.392294274Z Collecting kubelet logs for node ip-10-0-136-156.ec2.internal 2026-07-08T15:24:53.393881199Z Collecting kubelet logs for node ip-10-0-139-120.ec2.internal 2026-07-08T15:24:53.678350172Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-jtbn9 namespace 2026-07-08T15:24:53.683211020Z INFO: Node performance data collection complete. 2026-07-08T15:24:53.806859672Z error: the server doesn't have a resource type "clusters" 2026-07-08T15:24:53.938734730Z Caches written to disk