2026-04-23T13:54:44.005222936Z [disk usage checker] Started 2026-04-23T13:54:44.008623642Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-04-23T13:54:44.210104760Z Gathering data for ns/openshift-cluster-version... 2026-04-23T13:54:44.352334578Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-04-23T13:54:44.434863623Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-23T13:54:44.517309406Z Gathering data for ns/default... 2026-04-23T13:54:44.818854889Z Gathering data for ns/openshift... 2026-04-23T13:54:45.237873892Z Gathering data for ns/kube-system... 2026-04-23T13:54:45.971730572Z Gathering data for ns/openshift-etcd... 2026-04-23T13:54:46.300324588Z Wrote inspect data to must-gather. 2026-04-23T13:54:46.300504361Z error: inspection completed with the errors occurred while gathering data: 2026-04-23T13:54:46.300504361Z namespaces "assisted-installer" not found 2026-04-23T13:54:46.372728100Z Waiting on subprocesses to finish execution. 2026-04-23T13:54:46.384403187Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-04-23T13:54:46.385400287Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-04-23T13:54:46.388028594Z INFO: Gathering HAProxy config files 2026-04-23T13:54:46.404129306Z INFO: Waiting for node performance related collection to complete ... 2026-04-23T13:54:46.409856502Z INFO: Gathering machine config daemon's old logs from all nodes 2026-04-23T13:54:46.414439692Z INFO: Collecting host service logs for crio 2026-04-23T13:54:46.414821833Z INFO: Collecting host service logs for kubelet 2026-04-23T13:54:46.415137922Z INFO: Collecting host service logs for rpm-ostreed 2026-04-23T13:54:46.415436151Z INFO: Collecting host service logs for ostree-finalize-staged 2026-04-23T13:54:46.415743486Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-04-23T13:54:46.416034552Z INFO: Collecting host service logs for machine-config-daemon-host 2026-04-23T13:54:46.416315993Z INFO: Collecting host service logs for NetworkManager 2026-04-23T13:54:46.416616979Z INFO: Collecting host service logs for openvswitch 2026-04-23T13:54:46.416897712Z INFO: Collecting host service logs for ovs-configuration 2026-04-23T13:54:46.417175003Z INFO: Collecting host service logs for ovsdb-server 2026-04-23T13:54:46.417465774Z INFO: Collecting host service logs for ovs-vswitchd 2026-04-23T13:54:46.417825862Z INFO: Waiting for worker host service log collection to complete ... 2026-04-23T13:54:46.958245359Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-04-23T13:54:46.958339778Z INFO: on-disk MachineConfig config collection complete. 2026-04-23T13:54:47.217589422Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-5kw89 2026-04-23T13:54:47.217653549Z insights-runtime-extractor-mp5jl 2026-04-23T13:54:47.217674749Z insights-runtime-extractor-rb4nw 2026-04-23T13:54:47.217693642Z insights-runtime-extractor-vc88g 2026-04-23T13:54:47.229324321Z INFO: Found 1 replicas - prometheus-k8s-0 2026-04-23T13:54:47.517265432Z INFO: "metallb-operator" not detected. Skipping. 2026-04-23T13:54:47.524384123Z INFO: "sriov-network-operator" not detected. Skipping. 2026-04-23T13:54:47.541990022Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-04-23T13:54:47.542453785Z No resources found 2026-04-23T13:54:47.542832000Z error: the server doesn't have a resource type "performanceprofile" 2026-04-23T13:54:47.711635111Z Wrote inspect data to must-gather. 2026-04-23T13:54:47.720078620Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-04-23T13:54:47.752069531Z No resources found in openshift-etcd namespace. 2026-04-23T13:54:47.830359897Z INFO: Worker host service log collection to complete. 2026-04-23T13:54:47.863489631Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-04-23T13:54:47.863489631Z See 'oc rsync -h' for help and examples 2026-04-23T13:54:47.946639295Z Wrote inspect data to must-gather. 2026-04-23T13:54:48.004977673Z ERROR: No running kube-apiserver pods found 2026-04-23T13:54:48.061568091Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-04-23T13:54:48.089710268Z Wrote inspect data to must-gather. 2026-04-23T13:54:48.197625274Z INFO: Waiting for HAProxy config collection to complete ... 2026-04-23T13:54:48.277401671Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-04-23T13:54:48.444216002Z INFO: 'previous-logs' folder not found on ip-10-0-129-103.ec2.internal, skipping... 2026-04-23T13:54:48.479937911Z Gathering data for ns/openshift-monitoring... 2026-04-23T13:54:48.516117316Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-23T13:54:48.540134529Z Wrote inspect data to must-gather. 2026-04-23T13:54:48.672337307Z Wrote inspect data to must-gather. 2026-04-23T13:54:48.712286117Z No resources found 2026-04-23T13:54:48.721090692Z error: the server doesn't have a resource type "machineconfigs" 2026-04-23T13:54:48.849927538Z INFO: Getting rules from prometheus-k8s-0 2026-04-23T13:54:48.872346698Z Wrote inspect data to must-gather. 2026-04-23T13:54:48.880877912Z Wrote inspect data to must-gather. 2026-04-23T13:54:48.881412034Z tar: Removing leading `/' from member names 2026-04-23T13:54:48.887714062Z INFO: HAProxy config collection complete. 2026-04-23T13:54:48.910591822Z INFO: 'previous-logs' folder not found on ip-10-0-134-22.ec2.internal, skipping... 2026-04-23T13:54:49.012320435Z Wrote inspect data to must-gather. 2026-04-23T13:54:49.016227653Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-04-23T13:54:49.128906951Z Wrote inspect data to must-gather. 2026-04-23T13:54:49.164213953Z INFO: 'previous-logs' folder not found on ip-10-0-136-158.ec2.internal, skipping... 2026-04-23T13:54:49.164243942Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-04-23T13:54:49.164251113Z INFO: Machine Config Daemon termination log collection complete. 2026-04-23T13:54:49.241397870Z Wrote inspect data to must-gather. 2026-04-23T13:54:49.279765874Z INFO: Getting status/config from prometheus-k8s-0 2026-04-23T13:54:49.373652911Z Wrote inspect data to must-gather. 2026-04-23T13:54:49.402501192Z error: the server doesn't have a resource type "kubeletconfigs" 2026-04-23T13:54:49.603648473Z Wrote inspect data to must-gather. 2026-04-23T13:54:49.607929743Z Wrote inspect data to must-gather. 2026-04-23T13:54:49.689454212Z INFO: Getting status/flags from prometheus-k8s-0 2026-04-23T13:54:49.818592317Z Wrote inspect data to must-gather. 2026-04-23T13:54:49.907059619Z INFO: INTERCONNECT MODE 2026-04-23T13:54:49.907094337Z INFO: Gathering ovn-kubernetes DBs 2026-04-23T13:54:49.932776030Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-04-23T13:54:49.979074357Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-04-23T13:54:50.060944641Z Wrote inspect data to must-gather. 2026-04-23T13:54:50.085943957Z INFO: Gathering OVN_Northbound from ovnkube-node-4shpw... 2026-04-23T13:54:50.086235808Z INFO: Gathering OVN_Northbound from ovnkube-node-72hmc... 2026-04-23T13:54:50.086465889Z INFO: Gathering OVN_Northbound from ovnkube-node-f6dwj... 2026-04-23T13:54:50.086798614Z INFO: Gathering OVN_Southbound from ovnkube-node-4shpw... 2026-04-23T13:54:50.087260669Z INFO: Gathering OVN_Southbound from ovnkube-node-72hmc... 2026-04-23T13:54:50.087510660Z INFO: Gathering OVN_Southbound from ovnkube-node-f6dwj... 2026-04-23T13:54:50.262685232Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-04-23T13:54:50.376606561Z tar: Removing leading `/' from member names 2026-04-23T13:54:50.380020364Z tar: Removing leading `/' from member names 2026-04-23T13:54:50.398667161Z tar: Removing leading `/' from member names 2026-04-23T13:54:50.406913910Z tar: Removing leading `/' from member names 2026-04-23T13:54:50.413490084Z tar: Removing leading `/' from member names 2026-04-23T13:54:50.417188488Z tar: Removing leading `/' from member names 2026-04-23T13:54:50.428987320Z Wrote inspect data to must-gather. 2026-04-23T13:54:50.579565227Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-04-23T13:54:50.661901831Z W0423 13:54:50.661853 1282 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-04-23T13:54:50.669395468Z Wrote inspect data to must-gather. 2026-04-23T13:54:50.696678016Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-04-23T13:54:50.894861443Z Gathering data for ns/openshift-network-console... 2026-04-23T13:54:50.969056285Z INFO: Waiting for network log collection to complete ... 2026-04-23T13:54:50.969106597Z INFO: Waiting for ovnk database copies to complete ... 2026-04-23T13:54:50.969617281Z INFO: Copying ovnk databases complete. 2026-04-23T13:54:50.971502283Z 3.3M must-gather/network_logs/ovnk_database_store 2026-04-23T13:54:50.974172192Z ovnk_database_store/ 2026-04-23T13:54:50.974202161Z ovnk_database_store/ovnkube-node-f6dwj_nbdb 2026-04-23T13:54:50.986960374Z ovnk_database_store/ovnkube-node-4shpw_sbdb 2026-04-23T13:54:51.003596125Z Wrote inspect data to must-gather. 2026-04-23T13:54:51.006551704Z ovnk_database_store/ovnkube-node-72hmc_sbdb 2026-04-23T13:54:51.020935555Z INFO: Getting status from alertmanager-main-0 2026-04-23T13:54:51.028179801Z ovnk_database_store/ovnkube-node-f6dwj_sbdb 2026-04-23T13:54:51.050815679Z ovnk_database_store/ovnkube-node-72hmc_nbdb 2026-04-23T13:54:51.059915257Z ovnk_database_store/ovnkube-node-4shpw_nbdb 2026-04-23T13:54:51.069969103Z INFO: Network log collection complete. 2026-04-23T13:54:51.296257559Z Wrote inspect data to must-gather. 2026-04-23T13:54:51.346113186Z Gathering data for ns/openshift-console-operator... 2026-04-23T13:54:51.545283207Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-04-23T13:54:51.547399908Z INFO: Fallback to identify the container image from release info 2026-04-23T13:54:51.707683166Z Gathering data for ns/openshift-console... 2026-04-23T13:54:52.137569369Z Gathering data for ns/openshift-cluster-storage-operator... 2026-04-23T13:54:52.255920872Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d 2026-04-23T13:54:52.378348983Z daemonset.apps/perf-node-gather-daemonset created 2026-04-23T13:54:52.511010787Z Gathering data for ns/openshift-dns-operator... 2026-04-23T13:54:52.517023066Z Waiting for performance profile collector pods to become ready: 1 2026-04-23T13:54:52.794012632Z Gathering data for ns/openshift-dns... 2026-04-23T13:54:53.340242627Z Gathering data for ns/openshift-image-registry... 2026-04-23T13:54:53.633865628Z Waiting for performance profile collector pods to become ready: 2 2026-04-23T13:54:53.784547912Z Gathering data for ns/openshift-ingress-operator... 2026-04-23T13:54:54.022790310Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-04-23T13:54:54.074980319Z Gathering data for ns/openshift-ingress... 2026-04-23T13:54:54.364182427Z Gathering data for ns/openshift-ingress-canary... 2026-04-23T13:54:54.738718256Z Gathering data for ns/openshift-insights... 2026-04-23T13:54:54.767705887Z Waiting for performance profile collector pods to become ready: 3 2026-04-23T13:54:55.894489920Z Waiting for performance profile collector pods to become ready: 4 2026-04-23T13:54:56.787793615Z Gathering data for ns/kserve... 2026-04-23T13:54:57.023181705Z Waiting for performance profile collector pods to become ready: 5 2026-04-23T13:54:58.125218713Z Gathering data for ns/openshift-config... 2026-04-23T13:54:58.147182523Z Waiting for performance profile collector pods to become ready: 6 2026-04-23T13:54:58.384837462Z Gathering data for ns/openshift-config-managed... 2026-04-23T13:54:58.824894470Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-04-23T13:54:59.027988947Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-04-23T13:54:59.073115064Z Gathering data for ns/openshift-kube-apiserver... 2026-04-23T13:54:59.272654891Z Waiting for performance profile collector pods to become ready: 7 2026-04-23T13:54:59.327435116Z Gathering data for ns/openshift-kube-controller-manager... 2026-04-23T13:54:59.600440606Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-04-23T13:54:59.865681619Z Gathering data for ns/openshift-kube-scheduler... 2026-04-23T13:55:00.109013948Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-04-23T13:55:00.387340164Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-04-23T13:55:00.408646106Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-04-23T13:55:00.669857444Z Collecting performance related data for node ip-10-0-129-103.ec2.internal 2026-04-23T13:55:00.679249417Z Collecting performance related data for node ip-10-0-134-22.ec2.internal 2026-04-23T13:55:00.688942310Z Collecting performance related data for node ip-10-0-136-158.ec2.internal 2026-04-23T13:55:00.718568700Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-04-23T13:55:01.071584775Z Gathering data for ns/openshift-user-workload-monitoring... 2026-04-23T13:55:01.814638772Z Gathering data for ns/openshift-multus... 2026-04-23T13:55:03.223866265Z Gathering data for ns/openshift-ovn-kubernetes... 2026-04-23T13:55:04.034268163Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-04-23T13:55:04.632367915Z Gathering data for ns/openshift-host-network... 2026-04-23T13:55:04.970908502Z Gathering data for ns/openshift-network-diagnostics... 2026-04-23T13:55:05.475751333Z Gathering data for ns/openshift-network-node-identity... 2026-04-23T13:55:05.951852303Z Gathering data for ns/openshift-network-operator... 2026-04-23T13:55:06.323386785Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-04-23T13:55:06.585305408Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-04-23T13:55:07.011352312Z Gathering data for ns/openshift-apiserver-operator... 2026-04-23T13:55:07.282750692Z Gathering data for ns/openshift-apiserver... 2026-04-23T13:55:07.718884374Z Gathering data for ns/openshift-controller-manager-operator... 2026-04-23T13:55:07.983580767Z Gathering data for ns/openshift-controller-manager... 2026-04-23T13:55:08.254480686Z Gathering data for ns/openshift-cluster-samples-operator... 2026-04-23T13:55:08.477205005Z Collecting kubelet logs for node ip-10-0-129-103.ec2.internal 2026-04-23T13:55:08.479253859Z Collecting kubelet logs for node ip-10-0-134-22.ec2.internal 2026-04-23T13:55:08.482052793Z Collecting kubelet logs for node ip-10-0-136-158.ec2.internal 2026-04-23T13:55:08.866216160Z daemonset.apps "perf-node-gather-daemonset" deleted 2026-04-23T13:55:08.871186853Z INFO: Node performance data collection complete. 2026-04-23T13:55:08.972202805Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-04-23T13:55:09.042520474Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-04-23T13:55:09.275746201Z Gathering data for ns/openshift-service-ca-operator... 2026-04-23T13:55:09.554635319Z Gathering data for ns/openshift-service-ca... 2026-04-23T13:55:09.886416281Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-04-23T13:55:10.754012125Z Wrote inspect data to must-gather. 2026-04-23T13:55:10.754061592Z error: inspection completed with the errors occurred while gathering data: 2026-04-23T13:55:10.754061592Z skipping gathering secrets/support due to error: secrets "support" not found 2026-04-23T13:55:10.881925762Z error: the server doesn't have a resource type "clusters" 2026-04-23T13:55:11.019490084Z Caches written to disk