2026-04-25T00:18:38.623051479Z [disk usage checker] Started 2026-04-25T00:18:38.626681459Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-04-25T00:18:38.834565017Z Gathering data for ns/openshift-cluster-version... 2026-04-25T00:18:38.963496848Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-04-25T00:18:39.050027460Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-25T00:18:39.132266356Z Gathering data for ns/default... 2026-04-25T00:18:39.414884785Z Gathering data for ns/openshift... 2026-04-25T00:18:39.828266769Z Gathering data for ns/kube-system... 2026-04-25T00:18:40.497555677Z Gathering data for ns/openshift-etcd... 2026-04-25T00:18:40.798097078Z Wrote inspect data to must-gather. 2026-04-25T00:18:40.798128153Z error: inspection completed with the errors occurred while gathering data: 2026-04-25T00:18:40.798128153Z namespaces "assisted-installer" not found 2026-04-25T00:18:41.103536021Z Waiting on subprocesses to finish execution. 2026-04-25T00:18:41.109251457Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-04-25T00:18:41.114885084Z INFO: Gathering HAProxy config files 2026-04-25T00:18:41.116929980Z INFO: Collecting host service logs for crio 2026-04-25T00:18:41.117243874Z INFO: Collecting host service logs for kubelet 2026-04-25T00:18:41.117572908Z INFO: Collecting host service logs for rpm-ostreed 2026-04-25T00:18:41.117913798Z INFO: Collecting host service logs for ostree-finalize-staged 2026-04-25T00:18:41.118238218Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-04-25T00:18:41.118549374Z INFO: Collecting host service logs for machine-config-daemon-host 2026-04-25T00:18:41.119915972Z INFO: Collecting host service logs for NetworkManager 2026-04-25T00:18:41.120254635Z INFO: Collecting host service logs for openvswitch 2026-04-25T00:18:41.120475499Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-04-25T00:18:41.120607444Z INFO: Collecting host service logs for ovs-configuration 2026-04-25T00:18:41.121044123Z INFO: Collecting host service logs for ovsdb-server 2026-04-25T00:18:41.121371431Z INFO: Collecting host service logs for ovs-vswitchd 2026-04-25T00:18:41.121801844Z INFO: Waiting for worker host service log collection to complete ... 2026-04-25T00:18:41.126381089Z INFO: Gathering machine config daemon's old logs from all nodes 2026-04-25T00:18:41.517765737Z INFO: Waiting for node performance related collection to complete ... 2026-04-25T00:18:41.726892780Z INFO: "metallb-operator" not detected. Skipping. 2026-04-25T00:18:41.894227560Z No resources found in openshift-etcd namespace. 2026-04-25T00:18:41.908952564Z Wrote inspect data to must-gather. 2026-04-25T00:18:42.200066307Z INFO: "sriov-network-operator" not detected. Skipping. 2026-04-25T00:18:42.217350637Z Wrote inspect data to must-gather. 2026-04-25T00:18:42.259034469Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-04-25T00:18:42.259034469Z INFO: on-disk MachineConfig config collection complete. 2026-04-25T00:18:42.345640170Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-04-25T00:18:42.406808953Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-sfnrq 2026-04-25T00:18:42.406808953Z insights-runtime-extractor-gvs6p 2026-04-25T00:18:42.406808953Z insights-runtime-extractor-jm94p 2026-04-25T00:18:42.406808953Z insights-runtime-extractor-w5st9 2026-04-25T00:18:42.485351055Z No resources found 2026-04-25T00:18:42.510906185Z INFO: Worker host service log collection to complete. 2026-04-25T00:18:42.511338989Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-04-25T00:18:42.576781384Z INFO: Waiting for HAProxy config collection to complete ... 2026-04-25T00:18:42.582300776Z INFO: Found 1 replicas - prometheus-k8s-0 2026-04-25T00:18:42.584851642Z ERROR: No running kube-apiserver pods found 2026-04-25T00:18:42.593623827Z error: the server doesn't have a resource type "performanceprofile" 2026-04-25T00:18:42.655364267Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-04-25T00:18:42.655364267Z See 'oc rsync -h' for help and examples 2026-04-25T00:18:42.862952467Z tar: Removing leading `/' from member names 2026-04-25T00:18:42.868218990Z INFO: HAProxy config collection complete. 2026-04-25T00:18:42.895634016Z Gathering data for ns/openshift-monitoring... 2026-04-25T00:18:43.104595622Z INFO: 'previous-logs' folder not found on ip-10-0-132-64.ec2.internal, skipping... 2026-04-25T00:18:43.183659105Z Wrote inspect data to must-gather. 2026-04-25T00:18:43.209562393Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-04-25T00:18:43.307095367Z Wrote inspect data to must-gather. 2026-04-25T00:18:43.349871708Z No resources found 2026-04-25T00:18:43.354571028Z Wrote inspect data to must-gather. 2026-04-25T00:18:43.415594507Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-04-25T00:18:43.425637119Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-25T00:18:43.534015214Z Wrote inspect data to must-gather. 2026-04-25T00:18:43.536938051Z error: the server doesn't have a resource type "machineconfigs" 2026-04-25T00:18:43.579015333Z Wrote inspect data to must-gather. 2026-04-25T00:18:43.633873710Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-04-25T00:18:43.680103039Z INFO: 'previous-logs' folder not found on ip-10-0-138-5.ec2.internal, skipping... 2026-04-25T00:18:43.778033836Z Wrote inspect data to must-gather. 2026-04-25T00:18:43.820054401Z INFO: Getting rules from prometheus-k8s-0 2026-04-25T00:18:43.860897783Z Wrote inspect data to must-gather. 2026-04-25T00:18:43.952902422Z INFO: 'previous-logs' folder not found on ip-10-0-139-62.ec2.internal, skipping... 2026-04-25T00:18:43.952950723Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-04-25T00:18:43.952964076Z INFO: Machine Config Daemon termination log collection complete. 2026-04-25T00:18:44.018735040Z Wrote inspect data to must-gather. 2026-04-25T00:18:44.102928179Z Wrote inspect data to must-gather. 2026-04-25T00:18:44.177405266Z error: the server doesn't have a resource type "kubeletconfigs" 2026-04-25T00:18:44.256525227Z INFO: Getting status/config from prometheus-k8s-0 2026-04-25T00:18:44.370205670Z Wrote inspect data to must-gather. 2026-04-25T00:18:44.396337718Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-04-25T00:18:44.427216485Z Wrote inspect data to must-gather. 2026-04-25T00:18:44.666722300Z Wrote inspect data to must-gather. 2026-04-25T00:18:44.671824574Z INFO: INTERCONNECT MODE 2026-04-25T00:18:44.671852487Z INFO: Gathering ovn-kubernetes DBs 2026-04-25T00:18:44.683132144Z INFO: Getting status/flags from prometheus-k8s-0 2026-04-25T00:18:44.852455809Z INFO: Gathering OVN_Northbound from ovnkube-node-9rpfg... 2026-04-25T00:18:44.852744522Z INFO: Gathering OVN_Northbound from ovnkube-node-mj7ls... 2026-04-25T00:18:44.852998969Z INFO: Gathering OVN_Northbound from ovnkube-node-tfv9v... 2026-04-25T00:18:44.853352973Z INFO: Gathering OVN_Southbound from ovnkube-node-9rpfg... 2026-04-25T00:18:44.853851559Z INFO: Gathering OVN_Southbound from ovnkube-node-mj7ls... 2026-04-25T00:18:44.854156205Z INFO: Gathering OVN_Southbound from ovnkube-node-tfv9v... 2026-04-25T00:18:44.914586306Z W0425 00:18:44.914537 1176 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-04-25T00:18:44.922017100Z Wrote inspect data to must-gather. 2026-04-25T00:18:44.967076767Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-04-25T00:18:45.059314813Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-25T00:18:45.095345148Z tar: Removing leading `/' from member names 2026-04-25T00:18:45.212233284Z tar: Removing leading `/' from member names 2026-04-25T00:18:45.230302488Z tar: Removing leading `/' from member names 2026-04-25T00:18:45.235989916Z tar: Removing leading `/' from member names 2026-04-25T00:18:45.238939610Z tar: Removing leading `/' from member names 2026-04-25T00:18:45.238939610Z tar: Removing leading `/' from member names 2026-04-25T00:18:45.301856728Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-04-25T00:18:45.304680589Z Gathering data for ns/openshift-network-console... 2026-04-25T00:18:45.388727144Z Wrote inspect data to must-gather. 2026-04-25T00:18:45.574156914Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-04-25T00:18:45.754852486Z Wrote inspect data to must-gather. 2026-04-25T00:18:45.778051947Z INFO: Waiting for network log collection to complete ... 2026-04-25T00:18:45.778096007Z INFO: Waiting for ovnk database copies to complete ... 2026-04-25T00:18:45.778489845Z INFO: Copying ovnk databases complete. 2026-04-25T00:18:45.780468451Z 3.4M must-gather/network_logs/ovnk_database_store 2026-04-25T00:18:45.784333400Z ovnk_database_store/ 2026-04-25T00:18:45.784380833Z ovnk_database_store/ovnkube-node-9rpfg_nbdb 2026-04-25T00:18:45.794271759Z ovnk_database_store/ovnkube-node-9rpfg_sbdb 2026-04-25T00:18:45.813680634Z ovnk_database_store/ovnkube-node-mj7ls_nbdb 2026-04-25T00:18:45.814598589Z Gathering data for ns/openshift-console-operator... 2026-04-25T00:18:45.822137008Z ovnk_database_store/ovnkube-node-tfv9v_nbdb 2026-04-25T00:18:45.830683566Z ovnk_database_store/ovnkube-node-mj7ls_sbdb 2026-04-25T00:18:45.855829187Z ovnk_database_store/ovnkube-node-tfv9v_sbdb 2026-04-25T00:18:45.882250277Z INFO: Network log collection complete. 2026-04-25T00:18:46.062117520Z INFO: Getting status from alertmanager-main-0 2026-04-25T00:18:46.074594557Z Wrote inspect data to must-gather. 2026-04-25T00:18:46.211813936Z Gathering data for ns/openshift-console... 2026-04-25T00:18:46.387782147Z Wrote inspect data to must-gather. 2026-04-25T00:18:46.645544268Z Gathering data for ns/openshift-cluster-storage-operator... 2026-04-25T00:18:46.647205810Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-04-25T00:18:46.649320576Z INFO: Fallback to identify the container image from release info 2026-04-25T00:18:46.974491822Z Gathering data for ns/openshift-dns-operator... 2026-04-25T00:18:47.170176843Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d 2026-04-25T00:18:47.253217332Z Gathering data for ns/openshift-dns... 2026-04-25T00:18:47.303265890Z daemonset.apps/perf-node-gather-daemonset created 2026-04-25T00:18:47.430269168Z Waiting for performance profile collector pods to become ready: 1 2026-04-25T00:18:47.849207461Z Gathering data for ns/openshift-image-registry... 2026-04-25T00:18:48.302093503Z Gathering data for ns/openshift-ingress-operator... 2026-04-25T00:18:48.576448135Z Waiting for performance profile collector pods to become ready: 2 2026-04-25T00:18:48.639863755Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-04-25T00:18:48.661247425Z Gathering data for ns/openshift-ingress... 2026-04-25T00:18:48.975627134Z Gathering data for ns/openshift-ingress-canary... 2026-04-25T00:18:49.341667759Z Gathering data for ns/openshift-insights... 2026-04-25T00:18:49.716522273Z Waiting for performance profile collector pods to become ready: 3 2026-04-25T00:18:50.842460855Z Waiting for performance profile collector pods to become ready: 4 2026-04-25T00:18:51.545810206Z Gathering data for ns/kserve... 2026-04-25T00:18:51.971636114Z Waiting for performance profile collector pods to become ready: 5 2026-04-25T00:18:53.042532831Z Gathering data for ns/openshift-config... 2026-04-25T00:18:53.100441258Z Waiting for performance profile collector pods to become ready: 6 2026-04-25T00:18:53.323728005Z Gathering data for ns/openshift-config-managed... 2026-04-25T00:18:53.644672682Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-04-25T00:18:53.740470498Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-04-25T00:18:54.061231228Z Gathering data for ns/openshift-kube-apiserver... 2026-04-25T00:18:54.216250867Z Waiting for performance profile collector pods to become ready: 7 2026-04-25T00:18:54.362584937Z Gathering data for ns/openshift-kube-controller-manager... 2026-04-25T00:18:54.673067506Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-04-25T00:18:54.989240444Z Gathering data for ns/openshift-kube-scheduler... 2026-04-25T00:18:55.230987024Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-04-25T00:18:55.353791171Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-04-25T00:18:55.531396670Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-04-25T00:18:55.620082266Z Collecting performance related data for node ip-10-0-132-64.ec2.internal 2026-04-25T00:18:55.627494389Z Collecting performance related data for node ip-10-0-138-5.ec2.internal 2026-04-25T00:18:55.635478011Z Collecting performance related data for node ip-10-0-139-62.ec2.internal 2026-04-25T00:18:55.988843794Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-04-25T00:18:56.370400173Z Gathering data for ns/openshift-user-workload-monitoring... 2026-04-25T00:18:56.993675283Z Gathering data for ns/openshift-multus... 2026-04-25T00:18:58.376983217Z Gathering data for ns/openshift-ovn-kubernetes... 2026-04-25T00:18:58.650254508Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-04-25T00:18:59.790879500Z Gathering data for ns/openshift-host-network... 2026-04-25T00:19:00.104743425Z Gathering data for ns/openshift-network-diagnostics... 2026-04-25T00:19:00.635531248Z Gathering data for ns/openshift-network-node-identity... 2026-04-25T00:19:01.033654932Z Gathering data for ns/openshift-network-operator... 2026-04-25T00:19:01.397059326Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-04-25T00:19:01.646984831Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-04-25T00:19:02.093265283Z Gathering data for ns/openshift-apiserver-operator... 2026-04-25T00:19:02.404580439Z Gathering data for ns/openshift-apiserver... 2026-04-25T00:19:02.804991095Z Gathering data for ns/openshift-controller-manager-operator... 2026-04-25T00:19:03.112583662Z Gathering data for ns/openshift-controller-manager... 2026-04-25T00:19:03.367635127Z Gathering data for ns/openshift-cluster-samples-operator... 2026-04-25T00:19:03.656963874Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-04-25T00:19:04.012610878Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-04-25T00:19:04.119206447Z Collecting kubelet logs for node ip-10-0-132-64.ec2.internal 2026-04-25T00:19:04.120993583Z Collecting kubelet logs for node ip-10-0-138-5.ec2.internal 2026-04-25T00:19:04.122822080Z Collecting kubelet logs for node ip-10-0-139-62.ec2.internal 2026-04-25T00:19:04.346233795Z Gathering data for ns/openshift-service-ca-operator... 2026-04-25T00:19:04.523550260Z daemonset.apps "perf-node-gather-daemonset" deleted 2026-04-25T00:19:04.528361946Z INFO: Node performance data collection complete. 2026-04-25T00:19:04.659453517Z Gathering data for ns/openshift-service-ca... 2026-04-25T00:19:05.012826743Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-04-25T00:19:05.854309527Z Wrote inspect data to must-gather. 2026-04-25T00:19:05.854360769Z error: inspection completed with the errors occurred while gathering data: 2026-04-25T00:19:05.854360769Z skipping gathering secrets/support due to error: secrets "support" not found 2026-04-25T00:19:05.984106563Z error: the server doesn't have a resource type "clusters" 2026-04-25T00:19:06.117861690Z Caches written to disk