2026-09-07T14:55:44.369701940Z [disk usage checker] Started 2026-09-07T14:55:44.374032035Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-07T14:55:44.577826562Z Gathering data for ns/openshift-cluster-version... 2026-09-07T14:55:44.725987104Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-07T14:55:44.838978067Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-07T14:55:44.924815307Z Gathering data for ns/default... 2026-09-07T14:55:45.272725594Z Gathering data for ns/openshift... 2026-09-07T14:55:45.692150569Z Gathering data for ns/kube-system... 2026-09-07T14:55:46.383100647Z Gathering data for ns/openshift-etcd... 2026-09-07T14:55:46.687235399Z Wrote inspect data to must-gather. 2026-09-07T14:55:46.687266462Z error: inspection completed with the errors occurred while gathering data: 2026-09-07T14:55:46.687266462Z namespaces "assisted-installer" not found 2026-09-07T14:55:47.076202824Z Waiting on subprocesses to finish execution. 2026-09-07T14:55:47.081016771Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-07T14:55:47.084564408Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-07T14:55:47.087467703Z INFO: Collecting host service logs for crio 2026-09-07T14:55:47.087796294Z INFO: Collecting host service logs for kubelet 2026-09-07T14:55:47.088330496Z INFO: Collecting host service logs for rpm-ostreed 2026-09-07T14:55:47.088675128Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-07T14:55:47.089025682Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-07T14:55:47.089383098Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-07T14:55:47.089710499Z INFO: Collecting host service logs for NetworkManager 2026-09-07T14:55:47.090040599Z INFO: Collecting host service logs for openvswitch 2026-09-07T14:55:47.090395506Z INFO: Collecting host service logs for ovs-configuration 2026-09-07T14:55:47.090733083Z INFO: Collecting host service logs for ovsdb-server 2026-09-07T14:55:47.091229539Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-07T14:55:47.091678252Z INFO: Waiting for worker host service log collection to complete ... 2026-09-07T14:55:47.097038584Z INFO: Gathering HAProxy config files 2026-09-07T14:55:47.099054237Z Executing Istio gather script 2026-09-07T14:55:47.106399363Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-07T14:55:47.123398090Z INFO: Waiting for node performance related collection to complete ... 2026-09-07T14:55:47.889865841Z error: the server doesn't have a resource type "performanceprofile" 2026-09-07T14:55:47.998313281Z No resources found in openshift-etcd namespace. 2026-09-07T14:55:48.026377188Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-07T14:55:48.122632582Z No resources found 2026-09-07T14:55:48.134798530Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-07T14:55:48.155077423Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-07T14:55:48.222359210Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-07T14:55:48.398608952Z Wrote inspect data to must-gather. 2026-09-07T14:55:48.422241685Z INFO: Collecting Insights Archives from insights-operator-76bcd6ccf5-pqkm5 2026-09-07T14:55:48.422318220Z insights-runtime-extractor-468sz 2026-09-07T14:55:48.422342805Z insights-runtime-extractor-54zrt 2026-09-07T14:55:48.422378397Z insights-runtime-extractor-mt4j7 2026-09-07T14:55:48.559815868Z INFO: HAProxy config collection complete. 2026-09-07T14:55:48.582218905Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-07T14:55:48.582263096Z INFO: on-disk MachineConfig config collection complete. 2026-09-07T14:55:48.629644719Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-07T14:55:48.644269382Z INFO: "metallb-operator" not detected. Skipping. 2026-09-07T14:55:48.751144124Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-07T14:55:48.919176064Z ERROR: No running kube-apiserver pods found 2026-09-07T14:55:48.922274397Z INFO: Worker host service log collection to complete. 2026-09-07T14:55:49.252938500Z Wrote inspect data to must-gather. 2026-09-07T14:55:49.350242256Z Wrote inspect data to must-gather. 2026-09-07T14:55:49.459408262Z Wrote inspect data to must-gather. 2026-09-07T14:55:49.472429357Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-07T14:55:49.482710701Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-07T14:55:49.547685108Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-07T14:55:49.547685108Z See 'oc rsync -h' for help and examples 2026-09-07T14:55:49.629338430Z Wrote inspect data to must-gather. 2026-09-07T14:55:49.691535581Z Wrote inspect data to must-gather. 2026-09-07T14:55:49.709053474Z No resources found 2026-09-07T14:55:49.726152589Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-07T14:55:49.778102356Z Wrote inspect data to must-gather. 2026-09-07T14:55:49.782568796Z Gathering data for ns/openshift-monitoring... 2026-09-07T14:55:49.791467275Z INFO: 'previous-logs' folder not found on ip-10-0-128-182.ec2.internal, skipping... 2026-09-07T14:55:49.797089982Z error: the server doesn't have a resource type "machineconfigs" 2026-09-07T14:55:50.010376005Z INFO: Getting rules from prometheus-k8s-0 2026-09-07T14:55:50.015783290Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-07T14:55:50.064423396Z Wrote inspect data to must-gather. 2026-09-07T14:55:50.144715257Z INFO: 'previous-logs' folder not found on ip-10-0-140-32.ec2.internal, skipping... 2026-09-07T14:55:50.296152446Z Wrote inspect data to must-gather. 2026-09-07T14:55:50.336025361Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-07T14:55:50.380403782Z Wrote inspect data to must-gather. 2026-09-07T14:55:50.460726911Z INFO: 'previous-logs' folder not found on ip-10-0-143-201.ec2.internal, skipping... 2026-09-07T14:55:50.460768123Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-07T14:55:50.460804584Z INFO: Machine Config Daemon termination log collection complete. 2026-09-07T14:55:50.490148524Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-07T14:55:50.498198675Z INFO: Getting status/config from prometheus-k8s-0 2026-09-07T14:55:50.651925131Z Wrote inspect data to must-gather. 2026-09-07T14:55:50.720670440Z Wrote inspect data to must-gather. 2026-09-07T14:55:50.928247035Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-07T14:55:50.950831528Z Wrote inspect data to must-gather. 2026-09-07T14:55:50.961436664Z INFO: INTERCONNECT MODE 2026-09-07T14:55:50.961491255Z INFO: Gathering ovn-kubernetes DBs 2026-09-07T14:55:50.981161214Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-07T14:55:51.139463962Z INFO: Gathering OVN_Northbound from ovnkube-node-5hrn4... 2026-09-07T14:55:51.139842337Z INFO: Gathering OVN_Northbound from ovnkube-node-77s62... 2026-09-07T14:55:51.140146903Z INFO: Gathering OVN_Northbound from ovnkube-node-vcscq... 2026-09-07T14:55:51.140612750Z INFO: Gathering OVN_Southbound from ovnkube-node-5hrn4... 2026-09-07T14:55:51.140933398Z INFO: Gathering OVN_Southbound from ovnkube-node-77s62... 2026-09-07T14:55:51.141212556Z INFO: Gathering OVN_Southbound from ovnkube-node-vcscq... 2026-09-07T14:55:51.178277820Z Wrote inspect data to /must-gather/istio. 2026-09-07T14:55:51.201024486Z Wrote inspect data to must-gather. 2026-09-07T14:55:51.262615675Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-07T14:55:51.458700658Z tar: Removing leading `/' from member names 2026-09-07T14:55:51.477507241Z tar: Removing leading `/' from member names 2026-09-07T14:55:51.489648175Z tar: Removing leading `/' from member names 2026-09-07T14:55:51.552451876Z tar: Removing leading `/' from member names 2026-09-07T14:55:51.552693760Z tar: Removing leading `/' from member names 2026-09-07T14:55:51.557575178Z tar: Removing leading `/' from member names 2026-09-07T14:55:51.577680079Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-07T14:55:51.583466798Z Wrote inspect data to must-gather. 2026-09-07T14:55:51.637891400Z Wrote inspect data to /must-gather/istio. 2026-09-07T14:55:51.641539768Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-07T14:55:51.647440897Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-07T14:55:51.844991466Z Wrote inspect data to must-gather. 2026-09-07T14:55:51.909242806Z Wrote inspect data to /must-gather/istio. 2026-09-07T14:55:51.957580971Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-07T14:55:52.089994656Z INFO: Waiting for network log collection to complete ... 2026-09-07T14:55:52.090086507Z INFO: Waiting for ovnk database copies to complete ... 2026-09-07T14:55:52.090652454Z INFO: Copying ovnk databases complete. 2026-09-07T14:55:52.092926781Z 2.9M must-gather/network_logs/ovnk_database_store 2026-09-07T14:55:52.096177027Z ovnk_database_store/ 2026-09-07T14:55:52.096177027Z ovnk_database_store/ovnkube-node-77s62_sbdb 2026-09-07T14:55:52.114375871Z ovnk_database_store/ovnkube-node-vcscq_nbdb 2026-09-07T14:55:52.116257510Z Wrote inspect data to must-gather. 2026-09-07T14:55:52.123157762Z ovnk_database_store/ovnkube-node-5hrn4_sbdb 2026-09-07T14:55:52.141415914Z ovnk_database_store/ovnkube-node-5hrn4_nbdb 2026-09-07T14:55:52.149353465Z ovnk_database_store/ovnkube-node-77s62_nbdb 2026-09-07T14:55:52.157983776Z ovnk_database_store/ovnkube-node-vcscq_sbdb 2026-09-07T14:55:52.180733806Z INFO: Network log collection complete. 2026-09-07T14:55:52.183198120Z Wrote inspect data to /must-gather/istio. 2026-09-07T14:55:52.186193525Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-07T14:55:52.290202076Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-07T14:55:52.393467865Z Wrote inspect data to /must-gather/istio. 2026-09-07T14:55:52.402081857Z INFO: Getting status from alertmanager-main-0 2026-09-07T14:55:52.444705621Z Wrote inspect data to must-gather. 2026-09-07T14:55:52.512713013Z Gathering data for ns/openshift-network-console... 2026-09-07T14:55:52.651519397Z Wrote inspect data to /must-gather/istio. 2026-09-07T14:55:52.654846425Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-07T14:55:52.741325588Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-07T14:55:52.744314752Z INFO: Fallback to identify the container image from release info 2026-09-07T14:55:52.878688346Z Wrote inspect data to /must-gather/istio. 2026-09-07T14:55:52.979730720Z Gathering data for ns/openshift-console-operator... 2026-09-07T14:55:53.086957671Z Wrote inspect data to /must-gather/istio. 2026-09-07T14:55:53.090018730Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-07T14:55:53.247171665Z Wrote inspect data to /must-gather/istio. 2026-09-07T14:55:53.359589920Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f39c1daa89e19c5e987a3b759e646ffab6c1b2ab94a97ded81ddf13e73d0399 2026-09-07T14:55:53.404977721Z Gathering data for ns/openshift-console... 2026-09-07T14:55:53.450254254Z Wrote inspect data to /must-gather/istio. 2026-09-07T14:55:53.493163093Z daemonset.apps/perf-node-gather-daemonset created 2026-09-07T14:55:53.636330339Z Waiting for performance profile collector pods to become ready: 1 2026-09-07T14:55:53.980214056Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-07T14:55:54.179009410Z Done executing Istio gather script 2026-09-07T14:55:54.495893355Z Gathering data for ns/openshift-dns-operator... 2026-09-07T14:55:54.500967054Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-07T14:55:54.759084873Z Waiting for performance profile collector pods to become ready: 2 2026-09-07T14:55:54.829927664Z Gathering data for ns/openshift-dns... 2026-09-07T14:55:55.488771876Z Gathering data for ns/openshift-image-registry... 2026-09-07T14:55:55.893571853Z Waiting for performance profile collector pods to become ready: 3 2026-09-07T14:55:55.982270387Z Gathering data for ns/openshift-ingress-operator... 2026-09-07T14:55:56.316346885Z Gathering data for ns/openshift-ingress... 2026-09-07T14:55:56.677568968Z Gathering data for ns/openshift-ingress-canary... 2026-09-07T14:55:57.022915006Z Waiting for performance profile collector pods to become ready: 4 2026-09-07T14:55:57.094348064Z Gathering data for ns/openshift-insights... 2026-09-07T14:55:58.136257237Z Waiting for performance profile collector pods to become ready: 5 2026-09-07T14:55:58.926911046Z Gathering data for ns/openshift-jobset-operator... 2026-09-07T14:55:59.261159330Z Waiting for performance profile collector pods to become ready: 6 2026-09-07T14:55:59.505855378Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-07T14:55:59.768335469Z Gathering data for ns/openshift-config... 2026-09-07T14:56:00.049372730Z Gathering data for ns/openshift-config-managed... 2026-09-07T14:56:00.377094972Z Waiting for performance profile collector pods to become ready: 7 2026-09-07T14:56:00.487967404Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-07T14:56:00.808204636Z Gathering data for ns/openshift-kube-apiserver... 2026-09-07T14:56:01.073079270Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-07T14:56:01.373982434Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-07T14:56:01.507365148Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-07T14:56:01.706811780Z Gathering data for ns/openshift-kube-scheduler... 2026-09-07T14:56:01.784776539Z Collecting performance related data for node ip-10-0-143-201.ec2.internal 2026-09-07T14:56:01.792736328Z Collecting performance related data for node ip-10-0-128-182.ec2.internal 2026-09-07T14:56:01.801095112Z Collecting performance related data for node ip-10-0-140-32.ec2.internal 2026-09-07T14:56:02.038040962Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-07T14:56:02.405777557Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-07T14:56:02.841053349Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-07T14:56:03.225055129Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-07T14:56:03.802538904Z Gathering data for ns/openshift-multus... 2026-09-07T14:56:04.510944040Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-07T14:56:05.342352633Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-07T14:56:06.826093428Z Gathering data for ns/openshift-host-network... 2026-09-07T14:56:07.158549420Z Gathering data for ns/openshift-network-diagnostics... 2026-09-07T14:56:07.730977753Z Gathering data for ns/openshift-network-node-identity... 2026-09-07T14:56:08.203669933Z Gathering data for ns/openshift-network-operator... 2026-09-07T14:56:08.616720241Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-07T14:56:08.894128652Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-07T14:56:09.411761030Z Gathering data for ns/openshift-apiserver-operator... 2026-09-07T14:56:09.515768948Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-07T14:56:09.723722738Z Gathering data for ns/openshift-apiserver... 2026-09-07T14:56:10.139760705Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-07T14:56:10.382523967Z Collecting kubelet logs for node ip-10-0-143-201.ec2.internal 2026-09-07T14:56:10.384531368Z Collecting kubelet logs for node ip-10-0-128-182.ec2.internal 2026-09-07T14:56:10.386494283Z Collecting kubelet logs for node ip-10-0-140-32.ec2.internal 2026-09-07T14:56:10.424735296Z Gathering data for ns/openshift-controller-manager... 2026-09-07T14:56:10.748676753Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-58wzh namespace 2026-09-07T14:56:10.754136371Z INFO: Node performance data collection complete. 2026-09-07T14:56:10.785846254Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-07T14:56:11.482214115Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-07T14:56:11.772840594Z Gathering data for ns/openshift-service-ca-operator... 2026-09-07T14:56:12.102710043Z Gathering data for ns/openshift-service-ca... 2026-09-07T14:56:12.472857730Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-07T14:56:13.408736653Z Wrote inspect data to must-gather. 2026-09-07T14:56:13.408789321Z error: inspection completed with the errors occurred while gathering data: 2026-09-07T14:56:13.408789321Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-07T14:56:13.538964282Z error: the server doesn't have a resource type "clusters" 2026-09-07T14:56:13.670006925Z Caches written to disk