2026-04-17T11:26:47.327862822Z [disk usage checker] Started 2026-04-17T11:26:47.331525749Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-04-17T11:26:47.551609119Z Gathering data for ns/openshift-cluster-version... 2026-04-17T11:26:47.697020484Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-04-17T11:26:47.828002620Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-17T11:26:47.918778008Z Gathering data for ns/default... 2026-04-17T11:26:48.263261217Z Gathering data for ns/openshift... 2026-04-17T11:26:48.715969644Z Gathering data for ns/kube-system... 2026-04-17T11:26:49.472238247Z Gathering data for ns/openshift-etcd... 2026-04-17T11:26:49.787492657Z Wrote inspect data to must-gather. 2026-04-17T11:26:49.787530843Z error: inspection completed with the errors occurred while gathering data: 2026-04-17T11:26:49.787530843Z namespaces "assisted-installer" not found 2026-04-17T11:26:49.891281355Z Waiting on subprocesses to finish execution. 2026-04-17T11:26:49.898670688Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-04-17T11:26:49.900848125Z INFO: Gathering machine config daemon's old logs from all nodes 2026-04-17T11:26:49.906792947Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-04-17T11:26:49.910119734Z INFO: Collecting host service logs for crio 2026-04-17T11:26:49.910424051Z INFO: Collecting host service logs for kubelet 2026-04-17T11:26:49.910707626Z INFO: Collecting host service logs for rpm-ostreed 2026-04-17T11:26:49.911010232Z INFO: Collecting host service logs for ostree-finalize-staged 2026-04-17T11:26:49.911321700Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-04-17T11:26:49.911725135Z INFO: Collecting host service logs for machine-config-daemon-host 2026-04-17T11:26:49.912024288Z INFO: Collecting host service logs for NetworkManager 2026-04-17T11:26:49.912321433Z INFO: Collecting host service logs for openvswitch 2026-04-17T11:26:49.912649401Z INFO: Collecting host service logs for ovs-configuration 2026-04-17T11:26:49.912957495Z INFO: Collecting host service logs for ovsdb-server 2026-04-17T11:26:49.913267318Z INFO: Collecting host service logs for ovs-vswitchd 2026-04-17T11:26:49.913693216Z INFO: Waiting for worker host service log collection to complete ... 2026-04-17T11:26:49.915088209Z INFO: Gathering HAProxy config files 2026-04-17T11:26:49.989957435Z INFO: Waiting for node performance related collection to complete ... 2026-04-17T11:26:50.566563550Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-k76l5 2026-04-17T11:26:50.566563550Z insights-runtime-extractor-8gmpm 2026-04-17T11:26:50.566563550Z insights-runtime-extractor-9mvrz 2026-04-17T11:26:50.566563550Z insights-runtime-extractor-wpkcf 2026-04-17T11:26:50.893050495Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-04-17T11:26:50.926127881Z INFO: "metallb-operator" not detected. Skipping. 2026-04-17T11:26:50.961400231Z INFO: "sriov-network-operator" not detected. Skipping. 2026-04-17T11:26:51.147822929Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-04-17T11:26:51.267215104Z INFO: Found 1 replicas - prometheus-k8s-0 2026-04-17T11:26:51.314506996Z Wrote inspect data to must-gather. 2026-04-17T11:26:51.337790748Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-04-17T11:26:51.337906950Z INFO: on-disk MachineConfig config collection complete. 2026-04-17T11:26:51.357722360Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-04-17T11:26:51.357722360Z See 'oc rsync -h' for help and examples 2026-04-17T11:26:51.426093358Z INFO: Worker host service log collection to complete. 2026-04-17T11:26:51.508644717Z error: the server doesn't have a resource type "performanceprofile" 2026-04-17T11:26:51.541948585Z No resources found 2026-04-17T11:26:51.665228775Z Wrote inspect data to must-gather. 2026-04-17T11:26:51.732215022Z No resources found in openshift-etcd namespace. 2026-04-17T11:26:51.849266347Z No resources found 2026-04-17T11:26:51.884857530Z ERROR: No running kube-apiserver pods found 2026-04-17T11:26:51.949636200Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-17T11:26:52.057007451Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-04-17T11:26:52.110803060Z INFO: Waiting for HAProxy config collection to complete ... 2026-04-17T11:26:52.136083074Z Gathering data for ns/openshift-monitoring... 2026-04-17T11:26:52.142150309Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-04-17T11:26:52.176524140Z Wrote inspect data to must-gather. 2026-04-17T11:26:52.182648334Z Wrote inspect data to must-gather. 2026-04-17T11:26:52.185378139Z Wrote inspect data to must-gather. 2026-04-17T11:26:52.227490679Z Wrote inspect data to must-gather. 2026-04-17T11:26:52.301571349Z INFO: 'previous-logs' folder not found on ip-10-0-135-81.ec2.internal, skipping... 2026-04-17T11:26:52.318518453Z Wrote inspect data to must-gather. 2026-04-17T11:26:52.337203259Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-04-17T11:26:52.385177809Z error: the server doesn't have a resource type "machineconfigs" 2026-04-17T11:26:52.415033935Z tar: Removing leading `/' from member names 2026-04-17T11:26:52.422847879Z INFO: HAProxy config collection complete. 2026-04-17T11:26:52.576291041Z INFO: 'previous-logs' folder not found on ip-10-0-136-177.ec2.internal, skipping... 2026-04-17T11:26:52.578236037Z INFO: Getting rules from prometheus-k8s-0 2026-04-17T11:26:52.589086345Z Wrote inspect data to must-gather. 2026-04-17T11:26:52.668454684Z Wrote inspect data to must-gather. 2026-04-17T11:26:52.849824463Z Wrote inspect data to must-gather. 2026-04-17T11:26:52.859676410Z INFO: 'previous-logs' folder not found on ip-10-0-142-247.ec2.internal, skipping... 2026-04-17T11:26:52.859714494Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-04-17T11:26:52.859749910Z INFO: Machine Config Daemon termination log collection complete. 2026-04-17T11:26:52.908967694Z Wrote inspect data to must-gather. 2026-04-17T11:26:53.026827866Z INFO: Getting status/config from prometheus-k8s-0 2026-04-17T11:26:53.076222617Z error: the server doesn't have a resource type "kubeletconfigs" 2026-04-17T11:26:53.120284215Z Wrote inspect data to must-gather. 2026-04-17T11:26:53.289213545Z Wrote inspect data to must-gather. 2026-04-17T11:26:53.435623065Z INFO: INTERCONNECT MODE 2026-04-17T11:26:53.435657905Z INFO: Gathering ovn-kubernetes DBs 2026-04-17T11:26:53.497945549Z INFO: Getting status/flags from prometheus-k8s-0 2026-04-17T11:26:53.514208510Z Wrote inspect data to must-gather. 2026-04-17T11:26:53.617251550Z INFO: Gathering OVN_Northbound from ovnkube-node-65xlv... 2026-04-17T11:26:53.617564869Z INFO: Gathering OVN_Northbound from ovnkube-node-bvdth... 2026-04-17T11:26:53.617836462Z INFO: Gathering OVN_Northbound from ovnkube-node-q4q4t... 2026-04-17T11:26:53.618191917Z INFO: Gathering OVN_Southbound from ovnkube-node-65xlv... 2026-04-17T11:26:53.618532325Z INFO: Gathering OVN_Southbound from ovnkube-node-bvdth... 2026-04-17T11:26:53.618816648Z INFO: Gathering OVN_Southbound from ovnkube-node-q4q4t... 2026-04-17T11:26:53.802572672Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-04-17T11:26:53.858230234Z Wrote inspect data to must-gather. 2026-04-17T11:26:53.926351368Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-04-17T11:26:53.931871084Z tar: Removing leading `/' from member names 2026-04-17T11:26:53.932624008Z tar: Removing leading `/' from member names 2026-04-17T11:26:53.966766167Z tar: Removing leading `/' from member names 2026-04-17T11:26:53.979570605Z tar: Removing leading `/' from member names 2026-04-17T11:26:53.998224251Z tar: Removing leading `/' from member names 2026-04-17T11:26:53.998495523Z tar: Removing leading `/' from member names 2026-04-17T11:26:54.118185017Z Wrote inspect data to must-gather. 2026-04-17T11:26:54.158798945Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-04-17T11:26:54.377200120Z Wrote inspect data to must-gather. 2026-04-17T11:26:54.473039738Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-04-17T11:26:54.481613213Z INFO: Waiting for network log collection to complete ... 2026-04-17T11:26:54.481731422Z INFO: Waiting for ovnk database copies to complete ... 2026-04-17T11:26:54.482225004Z INFO: Copying ovnk databases complete. 2026-04-17T11:26:54.484184913Z 2.9M must-gather/network_logs/ovnk_database_store 2026-04-17T11:26:54.486751681Z ovnk_database_store/ 2026-04-17T11:26:54.486751681Z ovnk_database_store/ovnkube-node-q4q4t_sbdb 2026-04-17T11:26:54.506548456Z ovnk_database_store/ovnkube-node-q4q4t_nbdb 2026-04-17T11:26:54.514885981Z ovnk_database_store/ovnkube-node-65xlv_sbdb 2026-04-17T11:26:54.532736286Z ovnk_database_store/ovnkube-node-bvdth_nbdb 2026-04-17T11:26:54.540248352Z ovnk_database_store/ovnkube-node-bvdth_sbdb 2026-04-17T11:26:54.559604595Z ovnk_database_store/ovnkube-node-65xlv_nbdb 2026-04-17T11:26:54.571551235Z INFO: Network log collection complete. 2026-04-17T11:26:54.614467146Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-17T11:26:54.689739828Z Wrote inspect data to must-gather. 2026-04-17T11:26:54.795964533Z Gathering data for ns/openshift-network-console... 2026-04-17T11:26:54.924323312Z INFO: Getting status from alertmanager-main-0 2026-04-17T11:26:55.000775754Z Wrote inspect data to must-gather. 2026-04-17T11:26:55.294943834Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-04-17T11:26:55.297326420Z INFO: Fallback to identify the container image from release info 2026-04-17T11:26:55.303385829Z Gathering data for ns/openshift-console-operator... 2026-04-17T11:26:55.748085048Z Gathering data for ns/openshift-console... 2026-04-17T11:26:55.854673851Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d 2026-04-17T11:26:56.009007659Z daemonset.apps/perf-node-gather-daemonset created 2026-04-17T11:26:56.160675217Z Waiting for performance profile collector pods to become ready: 1 2026-04-17T11:26:56.288228802Z Gathering data for ns/openshift-cluster-storage-operator... 2026-04-17T11:26:56.706756731Z Gathering data for ns/openshift-dns-operator... 2026-04-17T11:26:57.021702063Z Gathering data for ns/openshift-dns... 2026-04-17T11:26:57.284625209Z Waiting for performance profile collector pods to become ready: 2 2026-04-17T11:26:57.342173753Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-04-17T11:26:57.881772058Z Gathering data for ns/openshift-image-registry... 2026-04-17T11:26:58.423154750Z Waiting for performance profile collector pods to become ready: 3 2026-04-17T11:26:58.506647723Z Gathering data for ns/openshift-ingress-operator... 2026-04-17T11:26:58.858028625Z Gathering data for ns/openshift-ingress... 2026-04-17T11:26:59.266690080Z Gathering data for ns/openshift-ingress-canary... 2026-04-17T11:26:59.556816428Z Waiting for performance profile collector pods to become ready: 4 2026-04-17T11:26:59.768227207Z Gathering data for ns/openshift-insights... 2026-04-17T11:27:00.676294292Z Waiting for performance profile collector pods to become ready: 5 2026-04-17T11:27:01.805184611Z Waiting for performance profile collector pods to become ready: 6 2026-04-17T11:27:02.033832097Z Gathering data for ns/kserve... 2026-04-17T11:27:02.347001304Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-04-17T11:27:02.934816516Z Waiting for performance profile collector pods to become ready: 7 2026-04-17T11:27:03.456890767Z Gathering data for ns/openshift-config... 2026-04-17T11:27:03.838368731Z Gathering data for ns/openshift-config-managed... 2026-04-17T11:27:04.055695121Z Waiting for performance profile collector pods to become ready: 8 2026-04-17T11:27:04.314552192Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-04-17T11:27:04.600072773Z Gathering data for ns/openshift-kube-apiserver... 2026-04-17T11:27:04.878095788Z Gathering data for ns/openshift-kube-controller-manager... 2026-04-17T11:27:05.189270533Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-04-17T11:27:05.191733159Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-04-17T11:27:05.484699783Z Collecting performance related data for node ip-10-0-142-247.ec2.internal 2026-04-17T11:27:05.493242403Z Collecting performance related data for node ip-10-0-135-81.ec2.internal 2026-04-17T11:27:05.503248605Z Collecting performance related data for node ip-10-0-136-177.ec2.internal 2026-04-17T11:27:05.563958908Z Gathering data for ns/openshift-kube-scheduler... 2026-04-17T11:27:05.918101522Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-04-17T11:27:06.319314211Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-04-17T11:27:06.770448709Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-04-17T11:27:07.202612022Z Gathering data for ns/openshift-user-workload-monitoring... 2026-04-17T11:27:07.351611694Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-04-17T11:27:07.853574563Z Gathering data for ns/openshift-multus... 2026-04-17T11:27:09.396555143Z Gathering data for ns/openshift-ovn-kubernetes... 2026-04-17T11:27:11.286280967Z Gathering data for ns/openshift-host-network... 2026-04-17T11:27:11.663196854Z Gathering data for ns/openshift-network-diagnostics... 2026-04-17T11:27:12.222981549Z Gathering data for ns/openshift-network-node-identity... 2026-04-17T11:27:12.356413031Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-04-17T11:27:12.671618638Z Gathering data for ns/openshift-network-operator... 2026-04-17T11:27:13.134991019Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-04-17T11:27:13.412390807Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-04-17T11:27:13.949397929Z Gathering data for ns/openshift-apiserver-operator... 2026-04-17T11:27:14.024923782Z Collecting kubelet logs for node ip-10-0-142-247.ec2.internal 2026-04-17T11:27:14.026713232Z Collecting kubelet logs for node ip-10-0-135-81.ec2.internal 2026-04-17T11:27:14.028418835Z Collecting kubelet logs for node ip-10-0-136-177.ec2.internal 2026-04-17T11:27:14.303327448Z Gathering data for ns/openshift-apiserver... 2026-04-17T11:27:14.414832499Z daemonset.apps "perf-node-gather-daemonset" deleted 2026-04-17T11:27:14.420605628Z INFO: Node performance data collection complete. 2026-04-17T11:27:14.781950445Z Gathering data for ns/openshift-controller-manager-operator... 2026-04-17T11:27:15.063780973Z Gathering data for ns/openshift-controller-manager... 2026-04-17T11:27:15.347407657Z Gathering data for ns/openshift-cluster-samples-operator... 2026-04-17T11:27:16.012484407Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-04-17T11:27:16.352919266Z Gathering data for ns/openshift-service-ca-operator... 2026-04-17T11:27:16.672325378Z Gathering data for ns/openshift-service-ca... 2026-04-17T11:27:17.057987582Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-04-17T11:27:17.363820246Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-04-17T11:27:18.010368034Z Wrote inspect data to must-gather. 2026-04-17T11:27:18.010411998Z error: inspection completed with the errors occurred while gathering data: 2026-04-17T11:27:18.010411998Z skipping gathering secrets/support due to error: secrets "support" not found 2026-04-17T11:27:18.152830925Z error: the server doesn't have a resource type "clusters" 2026-04-17T11:27:18.285067965Z Caches written to disk