2026-04-20T10:37:22.382601904Z [disk usage checker] Started
2026-04-20T10:37:22.386278588Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:37:22.759155192Z Gathering data for ns/openshift-cluster-version...
2026-04-20T10:37:22.937697441Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-20T10:37:23.052299297Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-20T10:37:23.153416582Z Gathering data for ns/default...
2026-04-20T10:37:23.546701008Z Gathering data for ns/openshift...
2026-04-20T10:37:24.079086284Z Gathering data for ns/kube-system...
2026-04-20T10:37:24.871843184Z Gathering data for ns/openshift-etcd...
2026-04-20T10:37:24.932610424Z Waiting on subprocesses to finish execution.
2026-04-20T10:37:24.938872049Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-20T10:37:24.941705580Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-20T10:37:24.947016295Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-20T10:37:24.949029811Z INFO: Gathering HAProxy config files
2026-04-20T10:37:24.957144392Z INFO: Collecting host service logs for crio
2026-04-20T10:37:24.957502550Z INFO: Collecting host service logs for kubelet
2026-04-20T10:37:24.957812040Z INFO: Collecting host service logs for rpm-ostreed
2026-04-20T10:37:24.958126585Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-20T10:37:24.958455355Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-20T10:37:24.958765159Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-20T10:37:24.959080547Z INFO: Collecting host service logs for NetworkManager
2026-04-20T10:37:24.959405956Z INFO: Collecting host service logs for openvswitch
2026-04-20T10:37:24.959716082Z INFO: Collecting host service logs for ovs-configuration
2026-04-20T10:37:24.960028591Z INFO: Collecting host service logs for ovsdb-server
2026-04-20T10:37:24.960340800Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-20T10:37:24.960742716Z INFO: Waiting for worker host service log collection to complete ...
2026-04-20T10:37:25.007228580Z INFO: Waiting for node performance related collection to complete ...
2026-04-20T10:37:25.461015896Z INFO: "metallb-operator" not detected. Skipping.
2026-04-20T10:37:25.534549319Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-s5fn5
2026-04-20T10:37:25.534601708Z insights-runtime-extractor-84cjk
2026-04-20T10:37:25.534622251Z insights-runtime-extractor-b5fs4
2026-04-20T10:37:25.534642060Z insights-runtime-extractor-msdsc
2026-04-20T10:37:25.751207183Z Wrote inspect data to must-gather.
2026-04-20T10:37:25.751307985Z error: inspection completed with the errors occurred while gathering data:
2026-04-20T10:37:25.751307985Z     namespaces "assisted-installer" not found
2026-04-20T10:37:25.836144835Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-20T10:37:26.005164664Z No resources found in openshift-etcd namespace.
2026-04-20T10:37:26.121840534Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-20T10:37:26.127529171Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-20T10:37:26.127529171Z See 'oc rsync -h' for help and examples
2026-04-20T10:37:26.192104066Z error: the server doesn't have a resource type "performanceprofile"
2026-04-20T10:37:26.241497105Z Wrote inspect data to must-gather.
2026-04-20T10:37:26.242735387Z ERROR: No running kube-apiserver pods found
2026-04-20T10:37:26.267439694Z No resources found
2026-04-20T10:37:26.273562320Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-20T10:37:26.349677540Z INFO: 'previous-logs' folder not found on ip-10-0-137-106.ec2.internal, skipping...
2026-04-20T10:37:26.466505080Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-20T10:37:26.466548034Z INFO: on-disk MachineConfig config collection complete.
2026-04-20T10:37:26.606737777Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-20T10:37:26.619559053Z INFO: Worker host service log collection to complete.
2026-04-20T10:37:26.903679645Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-20T10:37:26.942371814Z Wrote inspect data to must-gather.
2026-04-20T10:37:26.974042564Z Wrote inspect data to must-gather.
2026-04-20T10:37:26.975894170Z No resources found
2026-04-20T10:37:26.977273520Z Wrote inspect data to must-gather.
2026-04-20T10:37:27.097699206Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-20T10:37:27.177692840Z Gathering data for ns/openshift-monitoring...
2026-04-20T10:37:27.180449501Z Wrote inspect data to must-gather.
2026-04-20T10:37:27.234825651Z Wrote inspect data to must-gather.
2026-04-20T10:37:27.246562626Z Wrote inspect data to must-gather.
2026-04-20T10:37:27.268368672Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-20T10:37:27.274806199Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-20T10:37:27.347219978Z INFO: 'previous-logs' folder not found on ip-10-0-138-148.ec2.internal, skipping...
2026-04-20T10:37:27.361313884Z tar: Removing leading `/' from member names
2026-04-20T10:37:27.367244483Z INFO: HAProxy config collection complete.
2026-04-20T10:37:27.392315629Z error: the server doesn't have a resource type "machineconfigs"
2026-04-20T10:37:27.397377373Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:37:27.515819882Z Wrote inspect data to must-gather.
2026-04-20T10:37:27.602037194Z INFO: 'previous-logs' folder not found on ip-10-0-140-95.ec2.internal, skipping...
2026-04-20T10:37:27.602081267Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-20T10:37:27.602081267Z INFO: Machine Config Daemon termination log collection complete.
2026-04-20T10:37:27.620938479Z Wrote inspect data to must-gather.
2026-04-20T10:37:27.665638567Z INFO: Getting rules from prometheus-k8s-0
2026-04-20T10:37:27.771601160Z Wrote inspect data to must-gather.
2026-04-20T10:37:27.830324946Z Wrote inspect data to must-gather.
2026-04-20T10:37:27.984849100Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-20T10:37:28.001473516Z Wrote inspect data to must-gather.
2026-04-20T10:37:28.048015258Z INFO: Getting status/config from prometheus-k8s-0
2026-04-20T10:37:28.197689312Z Wrote inspect data to must-gather.
2026-04-20T10:37:28.266466221Z INFO: INTERCONNECT MODE
2026-04-20T10:37:28.266466221Z INFO: Gathering ovn-kubernetes DBs
2026-04-20T10:37:28.396601020Z Wrote inspect data to must-gather.
2026-04-20T10:37:28.435916096Z INFO: Gathering OVN_Northbound from ovnkube-node-69b6q...
2026-04-20T10:37:28.436200406Z INFO: Gathering OVN_Northbound from ovnkube-node-bxbxw...
2026-04-20T10:37:28.436443142Z INFO: Gathering OVN_Northbound from ovnkube-node-zwdv7...
2026-04-20T10:37:28.436717818Z INFO: Gathering OVN_Southbound from ovnkube-node-69b6q...
2026-04-20T10:37:28.436957795Z INFO: Gathering OVN_Southbound from ovnkube-node-bxbxw...
2026-04-20T10:37:28.437201117Z INFO: Gathering OVN_Southbound from ovnkube-node-zwdv7...
2026-04-20T10:37:28.445807216Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-20T10:37:28.728062630Z Wrote inspect data to must-gather.
2026-04-20T10:37:28.756385547Z tar: Removing leading `/' from member names
2026-04-20T10:37:28.789536190Z tar: Removing leading `/' from member names
2026-04-20T10:37:28.790485955Z tar: Removing leading `/' from member names
2026-04-20T10:37:28.811777398Z tar: Removing leading `/' from member names
2026-04-20T10:37:28.868159352Z tar: Removing leading `/' from member names
2026-04-20T10:37:28.868159352Z tar: Removing leading `/' from member names
2026-04-20T10:37:28.882901410Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-20T10:37:28.911230049Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-20T10:37:28.953122761Z Wrote inspect data to must-gather.
2026-04-20T10:37:29.207065590Z Wrote inspect data to must-gather.
2026-04-20T10:37:29.216135024Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-20T10:37:29.264662229Z INFO: Waiting for network log collection to complete ...
2026-04-20T10:37:29.264711544Z INFO: Waiting for ovnk database copies to complete ...
2026-04-20T10:37:29.265199651Z INFO: Copying ovnk databases complete.
2026-04-20T10:37:29.267089406Z 3.1M	must-gather/network_logs/ovnk_database_store
2026-04-20T10:37:29.271170017Z ovnk_database_store/
2026-04-20T10:37:29.271170017Z ovnk_database_store/ovnkube-node-bxbxw_nbdb
2026-04-20T10:37:29.280839759Z ovnk_database_store/ovnkube-node-69b6q_sbdb
2026-04-20T10:37:29.298520531Z ovnk_database_store/ovnkube-node-69b6q_nbdb
2026-04-20T10:37:29.307162979Z ovnk_database_store/ovnkube-node-bxbxw_sbdb
2026-04-20T10:37:29.327103777Z ovnk_database_store/ovnkube-node-zwdv7_nbdb
2026-04-20T10:37:29.336936716Z ovnk_database_store/ovnkube-node-zwdv7_sbdb
2026-04-20T10:37:29.359101886Z INFO: Network log collection complete.
2026-04-20T10:37:29.423329349Z W0420 10:37:29.423278    1373 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-04-20T10:37:29.430548968Z Wrote inspect data to must-gather.
2026-04-20T10:37:29.518467726Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-20T10:37:29.521081499Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-20T10:37:29.704858285Z Wrote inspect data to must-gather.
2026-04-20T10:37:29.727394466Z Gathering data for ns/openshift-network-console...
2026-04-20T10:37:29.975379589Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-20T10:37:29.977751670Z INFO: Fallback to identify the container image from release info
2026-04-20T10:37:29.986154693Z INFO: Getting status from alertmanager-main-0
2026-04-20T10:37:30.278508063Z Gathering data for ns/openshift-console-operator...
2026-04-20T10:37:30.589635612Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-20T10:37:30.743585319Z daemonset.apps/perf-node-gather-daemonset created
2026-04-20T10:37:30.888536978Z Waiting for performance profile collector pods to become ready: 1
2026-04-20T10:37:30.928863124Z Gathering data for ns/openshift-console...
2026-04-20T10:37:31.506290861Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-20T10:37:31.924467613Z Gathering data for ns/openshift-dns-operator...
2026-04-20T10:37:32.031459916Z Waiting for performance profile collector pods to become ready: 2
2026-04-20T10:37:32.299406627Z Gathering data for ns/openshift-dns...
2026-04-20T10:37:32.402337587Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:37:33.165281635Z Waiting for performance profile collector pods to become ready: 3
2026-04-20T10:37:33.178777862Z Gathering data for ns/openshift-image-registry...
2026-04-20T10:37:33.856597825Z Gathering data for ns/openshift-ingress-operator...
2026-04-20T10:37:34.291386164Z Gathering data for ns/openshift-ingress...
2026-04-20T10:37:34.295095571Z Waiting for performance profile collector pods to become ready: 4
2026-04-20T10:37:34.673791532Z Gathering data for ns/openshift-ingress-canary...
2026-04-20T10:37:35.151955921Z Gathering data for ns/openshift-insights...
2026-04-20T10:37:35.411115982Z Waiting for performance profile collector pods to become ready: 5
2026-04-20T10:37:36.522650622Z Waiting for performance profile collector pods to become ready: 6
2026-04-20T10:37:37.187645177Z Gathering data for ns/openshift-jobset-operator...
2026-04-20T10:37:37.407011340Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:37:37.651228066Z Waiting for performance profile collector pods to become ready: 7
2026-04-20T10:37:38.060523179Z Gathering data for ns/openshift-config...
2026-04-20T10:37:38.412790535Z Gathering data for ns/openshift-config-managed...
2026-04-20T10:37:38.776298876Z Waiting for performance profile collector pods to become ready: 8
2026-04-20T10:37:38.853947843Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-20T10:37:39.155012729Z Gathering data for ns/openshift-kube-apiserver...
2026-04-20T10:37:39.461646854Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-20T10:37:39.746585120Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-20T10:37:39.893080345Z Waiting for performance profile collector pods to become ready: 9
2026-04-20T10:37:40.039052584Z Gathering data for ns/openshift-kube-scheduler...
2026-04-20T10:37:40.325878444Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-20T10:37:40.674090030Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-20T10:37:41.025633769Z Waiting for performance profile collector pods to become ready: 10
2026-04-20T10:37:41.057709440Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-20T10:37:41.451712631Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-20T10:37:41.988887779Z Gathering data for ns/openshift-multus...
2026-04-20T10:37:42.137767131Z Waiting for performance profile collector pods to become ready: 11
2026-04-20T10:37:42.411818885Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:37:43.264036515Z Waiting for performance profile collector pods to become ready: 12
2026-04-20T10:37:43.485933053Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-20T10:37:44.372703527Z Waiting for performance profile collector pods to become ready: 13
2026-04-20T10:37:45.206565817Z Gathering data for ns/openshift-host-network...
2026-04-20T10:37:45.511168223Z Waiting for performance profile collector pods to become ready: 14
2026-04-20T10:37:45.569948994Z Gathering data for ns/openshift-network-diagnostics...
2026-04-20T10:37:46.092120179Z Gathering data for ns/openshift-network-node-identity...
2026-04-20T10:37:46.500566170Z Gathering data for ns/openshift-network-operator...
2026-04-20T10:37:46.620444941Z Waiting for performance profile collector pods to become ready: 15
2026-04-20T10:37:46.921905334Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-20T10:37:47.263582108Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-20T10:37:47.418575419Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:37:47.749277064Z Waiting for performance profile collector pods to become ready: 16
2026-04-20T10:37:47.792532243Z Gathering data for ns/openshift-apiserver-operator...
2026-04-20T10:37:48.099873319Z Gathering data for ns/openshift-apiserver...
2026-04-20T10:37:48.546436098Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-20T10:37:48.874481983Z Waiting for performance profile collector pods to become ready: 17
2026-04-20T10:37:48.924890411Z Gathering data for ns/openshift-controller-manager...
2026-04-20T10:37:49.357856796Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-20T10:37:49.992675882Z Waiting for performance profile collector pods to become ready: 18
2026-04-20T10:37:50.178518590Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-20T10:37:50.526879948Z Gathering data for ns/openshift-service-ca-operator...
2026-04-20T10:37:50.955098191Z Gathering data for ns/openshift-service-ca...
2026-04-20T10:37:51.135125991Z Waiting for performance profile collector pods to become ready: 19
2026-04-20T10:37:51.492709110Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-20T10:37:52.254402335Z Waiting for performance profile collector pods to become ready: 20
2026-04-20T10:37:52.423695505Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:37:52.829155073Z Wrote inspect data to must-gather.
2026-04-20T10:37:52.829201048Z error: inspection completed with the errors occurred while gathering data:
2026-04-20T10:37:52.829201048Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-20T10:37:53.365459409Z Waiting for performance profile collector pods to become ready: 21
2026-04-20T10:37:54.472923578Z Waiting for performance profile collector pods to become ready: 22
2026-04-20T10:37:55.584080401Z Waiting for performance profile collector pods to become ready: 23
2026-04-20T10:37:56.693991665Z Waiting for performance profile collector pods to become ready: 24
2026-04-20T10:37:57.428620502Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:37:57.814851170Z Waiting for performance profile collector pods to become ready: 25
2026-04-20T10:37:58.931069610Z Waiting for performance profile collector pods to become ready: 26
2026-04-20T10:38:00.040704100Z Waiting for performance profile collector pods to become ready: 27
2026-04-20T10:38:01.153383524Z Waiting for performance profile collector pods to become ready: 28
2026-04-20T10:38:02.264201063Z Waiting for performance profile collector pods to become ready: 29
2026-04-20T10:38:02.433174650Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:38:03.376750290Z Waiting for performance profile collector pods to become ready: 30
2026-04-20T10:38:04.489647026Z Waiting for performance profile collector pods to become ready: 31
2026-04-20T10:38:05.600917051Z Waiting for performance profile collector pods to become ready: 32
2026-04-20T10:38:06.720550338Z Waiting for performance profile collector pods to become ready: 33
2026-04-20T10:38:07.437861181Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:38:07.829064572Z Waiting for performance profile collector pods to become ready: 34
2026-04-20T10:38:08.941639695Z Waiting for performance profile collector pods to become ready: 35
2026-04-20T10:38:10.053034643Z Waiting for performance profile collector pods to become ready: 36
2026-04-20T10:38:11.168072493Z Waiting for performance profile collector pods to become ready: 37
2026-04-20T10:38:12.278176090Z Waiting for performance profile collector pods to become ready: 38
2026-04-20T10:38:12.442645151Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:38:13.391164719Z Waiting for performance profile collector pods to become ready: 39
2026-04-20T10:38:14.503526364Z Waiting for performance profile collector pods to become ready: 40
2026-04-20T10:38:15.613056869Z Waiting for performance profile collector pods to become ready: 41
2026-04-20T10:38:16.723812127Z Waiting for performance profile collector pods to become ready: 42
2026-04-20T10:38:17.447427528Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:38:17.833101553Z Waiting for performance profile collector pods to become ready: 43
2026-04-20T10:38:18.944829913Z Waiting for performance profile collector pods to become ready: 44
2026-04-20T10:38:20.055918886Z Waiting for performance profile collector pods to become ready: 45
2026-04-20T10:38:21.166024347Z Waiting for performance profile collector pods to become ready: 46
2026-04-20T10:38:22.278466720Z Waiting for performance profile collector pods to become ready: 47
2026-04-20T10:38:22.452150146Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:38:23.397739851Z Waiting for performance profile collector pods to become ready: 48
2026-04-20T10:38:24.513435027Z Waiting for performance profile collector pods to become ready: 49
2026-04-20T10:38:25.626317832Z Waiting for performance profile collector pods to become ready: 50
2026-04-20T10:38:26.732745505Z Waiting for performance profile collector pods to become ready: 51
2026-04-20T10:38:27.457370727Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:38:27.841858300Z Waiting for performance profile collector pods to become ready: 52
2026-04-20T10:38:28.947797396Z Waiting for performance profile collector pods to become ready: 53
2026-04-20T10:38:30.057328548Z Waiting for performance profile collector pods to become ready: 54
2026-04-20T10:38:31.165036630Z Waiting for performance profile collector pods to become ready: 55
2026-04-20T10:38:32.279177544Z Waiting for performance profile collector pods to become ready: 56
2026-04-20T10:38:32.462148469Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:38:33.390511140Z Waiting for performance profile collector pods to become ready: 57
2026-04-20T10:38:34.497570069Z Waiting for performance profile collector pods to become ready: 58
2026-04-20T10:38:35.612099938Z Waiting for performance profile collector pods to become ready: 59
2026-04-20T10:38:36.723951013Z Waiting for performance profile collector pods to become ready: 60
2026-04-20T10:38:37.466735859Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:38:37.829160117Z Waiting for performance profile collector pods to become ready: 61
2026-04-20T10:38:38.937929054Z Waiting for performance profile collector pods to become ready: 62
2026-04-20T10:38:40.045555051Z Waiting for performance profile collector pods to become ready: 63
2026-04-20T10:38:41.150579361Z Waiting for performance profile collector pods to become ready: 64
2026-04-20T10:38:42.256961776Z Waiting for performance profile collector pods to become ready: 65
2026-04-20T10:38:42.471430252Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:38:43.367291283Z Waiting for performance profile collector pods to become ready: 66
2026-04-20T10:38:44.476198234Z Waiting for performance profile collector pods to become ready: 67
2026-04-20T10:38:45.587075602Z Waiting for performance profile collector pods to become ready: 68
2026-04-20T10:38:46.699114468Z Waiting for performance profile collector pods to become ready: 69
2026-04-20T10:38:47.476140729Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:38:47.810511731Z Waiting for performance profile collector pods to become ready: 70
2026-04-20T10:38:48.915303389Z Waiting for performance profile collector pods to become ready: 71
2026-04-20T10:38:50.023980993Z Waiting for performance profile collector pods to become ready: 72
2026-04-20T10:38:51.133211994Z Waiting for performance profile collector pods to become ready: 73
2026-04-20T10:38:52.243455648Z Waiting for performance profile collector pods to become ready: 74
2026-04-20T10:38:52.480973540Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:38:53.359529904Z Waiting for performance profile collector pods to become ready: 75
2026-04-20T10:38:54.470996867Z Waiting for performance profile collector pods to become ready: 76
2026-04-20T10:38:55.577670977Z Waiting for performance profile collector pods to become ready: 77
2026-04-20T10:38:56.687369740Z Waiting for performance profile collector pods to become ready: 78
2026-04-20T10:38:57.485679866Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:38:57.795854144Z Waiting for performance profile collector pods to become ready: 79
2026-04-20T10:38:58.907376809Z Waiting for performance profile collector pods to become ready: 80
2026-04-20T10:39:00.015035885Z Waiting for performance profile collector pods to become ready: 81
2026-04-20T10:39:01.125219651Z Waiting for performance profile collector pods to become ready: 82
2026-04-20T10:39:02.237780576Z Waiting for performance profile collector pods to become ready: 83
2026-04-20T10:39:02.490445696Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:39:03.345490023Z Waiting for performance profile collector pods to become ready: 84
2026-04-20T10:39:04.452419591Z Waiting for performance profile collector pods to become ready: 85
2026-04-20T10:39:05.560645271Z Waiting for performance profile collector pods to become ready: 86
2026-04-20T10:39:06.666650342Z Waiting for performance profile collector pods to become ready: 87
2026-04-20T10:39:07.495113079Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:39:07.776911119Z Waiting for performance profile collector pods to become ready: 88
2026-04-20T10:39:08.885142088Z Waiting for performance profile collector pods to become ready: 89
2026-04-20T10:39:09.989564737Z Waiting for performance profile collector pods to become ready: 90
2026-04-20T10:39:11.098124223Z Waiting for performance profile collector pods to become ready: 91
2026-04-20T10:39:12.209325051Z Waiting for performance profile collector pods to become ready: 92
2026-04-20T10:39:12.499821091Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:39:13.322394494Z Waiting for performance profile collector pods to become ready: 93
2026-04-20T10:39:14.432579250Z Waiting for performance profile collector pods to become ready: 94
2026-04-20T10:39:15.541012655Z Waiting for performance profile collector pods to become ready: 95
2026-04-20T10:39:16.649109483Z Waiting for performance profile collector pods to become ready: 96
2026-04-20T10:39:17.504500603Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:39:17.758984530Z Waiting for performance profile collector pods to become ready: 97
2026-04-20T10:39:18.868531473Z Waiting for performance profile collector pods to become ready: 98
2026-04-20T10:39:19.977028751Z Waiting for performance profile collector pods to become ready: 99
2026-04-20T10:39:21.084739481Z Waiting for performance profile collector pods to become ready: 100
2026-04-20T10:39:22.196393570Z Waiting for performance profile collector pods to become ready: 101
2026-04-20T10:39:22.509123763Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:39:23.311050391Z Waiting for performance profile collector pods to become ready: 102
2026-04-20T10:39:24.420470109Z Waiting for performance profile collector pods to become ready: 103
2026-04-20T10:39:25.529986461Z Waiting for performance profile collector pods to become ready: 104
2026-04-20T10:39:26.637384605Z Waiting for performance profile collector pods to become ready: 105
2026-04-20T10:39:27.513896343Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:39:27.748044639Z Waiting for performance profile collector pods to become ready: 106
2026-04-20T10:39:28.854860789Z Waiting for performance profile collector pods to become ready: 107
2026-04-20T10:39:29.958889872Z Waiting for performance profile collector pods to become ready: 108
2026-04-20T10:39:31.064866902Z Waiting for performance profile collector pods to become ready: 109
2026-04-20T10:39:32.173073622Z Waiting for performance profile collector pods to become ready: 110
2026-04-20T10:39:32.518584581Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:39:33.280437922Z Waiting for performance profile collector pods to become ready: 111
2026-04-20T10:39:34.386311278Z Waiting for performance profile collector pods to become ready: 112
2026-04-20T10:39:35.493651400Z Waiting for performance profile collector pods to become ready: 113
2026-04-20T10:39:36.600545175Z Waiting for performance profile collector pods to become ready: 114
2026-04-20T10:39:37.523263235Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:39:37.711046989Z Waiting for performance profile collector pods to become ready: 115
2026-04-20T10:39:38.820297118Z Waiting for performance profile collector pods to become ready: 116
2026-04-20T10:39:39.927646328Z Waiting for performance profile collector pods to become ready: 117
2026-04-20T10:39:41.037675742Z Waiting for performance profile collector pods to become ready: 118
2026-04-20T10:39:42.146046802Z Waiting for performance profile collector pods to become ready: 119
2026-04-20T10:39:42.527982556Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:39:43.253848944Z Waiting for performance profile collector pods to become ready: 120
2026-04-20T10:39:44.360862960Z Waiting for performance profile collector pods to become ready: 121
2026-04-20T10:39:45.469865338Z Waiting for performance profile collector pods to become ready: 122
2026-04-20T10:39:46.578218340Z Waiting for performance profile collector pods to become ready: 123
2026-04-20T10:39:47.532628687Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:39:47.683308114Z Waiting for performance profile collector pods to become ready: 124
2026-04-20T10:39:48.790130055Z Waiting for performance profile collector pods to become ready: 125
2026-04-20T10:39:49.896856827Z Waiting for performance profile collector pods to become ready: 126
2026-04-20T10:39:51.005040514Z Waiting for performance profile collector pods to become ready: 127
2026-04-20T10:39:52.115722384Z Waiting for performance profile collector pods to become ready: 128
2026-04-20T10:39:52.537216462Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:39:53.228297973Z Waiting for performance profile collector pods to become ready: 129
2026-04-20T10:39:54.338444776Z Waiting for performance profile collector pods to become ready: 130
2026-04-20T10:39:55.447275003Z Waiting for performance profile collector pods to become ready: 131
2026-04-20T10:39:56.556175923Z Waiting for performance profile collector pods to become ready: 132
2026-04-20T10:39:57.541973629Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:39:57.664720828Z Waiting for performance profile collector pods to become ready: 133
2026-04-20T10:39:58.774003656Z Waiting for performance profile collector pods to become ready: 134
2026-04-20T10:39:59.881082889Z Waiting for performance profile collector pods to become ready: 135
2026-04-20T10:40:00.986849612Z Waiting for performance profile collector pods to become ready: 136
2026-04-20T10:40:02.095956496Z Waiting for performance profile collector pods to become ready: 137
2026-04-20T10:40:02.546699540Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:40:03.206862814Z Waiting for performance profile collector pods to become ready: 138
2026-04-20T10:40:04.317421527Z Waiting for performance profile collector pods to become ready: 139
2026-04-20T10:40:05.426233983Z Waiting for performance profile collector pods to become ready: 140
2026-04-20T10:40:06.534103597Z Waiting for performance profile collector pods to become ready: 141
2026-04-20T10:40:07.551567881Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:40:07.643335047Z Waiting for performance profile collector pods to become ready: 142
2026-04-20T10:40:08.757813382Z Waiting for performance profile collector pods to become ready: 143
2026-04-20T10:40:09.868473987Z Waiting for performance profile collector pods to become ready: 144
2026-04-20T10:40:10.978014876Z Waiting for performance profile collector pods to become ready: 145
2026-04-20T10:40:12.090183935Z Waiting for performance profile collector pods to become ready: 146
2026-04-20T10:40:12.556288523Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:40:13.201275558Z Waiting for performance profile collector pods to become ready: 147
2026-04-20T10:40:14.308375308Z Waiting for performance profile collector pods to become ready: 148
2026-04-20T10:40:15.417871799Z Waiting for performance profile collector pods to become ready: 149
2026-04-20T10:40:16.528677220Z Waiting for performance profile collector pods to become ready: 150
2026-04-20T10:40:17.561040744Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:40:17.635851467Z Waiting for performance profile collector pods to become ready: 151
2026-04-20T10:40:18.747304040Z Waiting for performance profile collector pods to become ready: 152
2026-04-20T10:40:19.854785877Z Waiting for performance profile collector pods to become ready: 153
2026-04-20T10:40:20.962007429Z Waiting for performance profile collector pods to become ready: 154
2026-04-20T10:40:22.070442652Z Waiting for performance profile collector pods to become ready: 155
2026-04-20T10:40:22.565665479Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:40:23.178791112Z Waiting for performance profile collector pods to become ready: 156
2026-04-20T10:40:24.285053947Z Waiting for performance profile collector pods to become ready: 157
2026-04-20T10:40:25.392586620Z Waiting for performance profile collector pods to become ready: 158
2026-04-20T10:40:26.496630529Z Waiting for performance profile collector pods to become ready: 159
2026-04-20T10:40:27.570340722Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:40:27.606538033Z Waiting for performance profile collector pods to become ready: 160
2026-04-20T10:40:28.713648539Z Waiting for performance profile collector pods to become ready: 161
2026-04-20T10:40:29.820862864Z Waiting for performance profile collector pods to become ready: 162
2026-04-20T10:40:30.927407230Z Waiting for performance profile collector pods to become ready: 163
2026-04-20T10:40:32.033252173Z Waiting for performance profile collector pods to become ready: 164
2026-04-20T10:40:32.575119919Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:40:33.143923205Z Waiting for performance profile collector pods to become ready: 165
2026-04-20T10:40:34.252885566Z Waiting for performance profile collector pods to become ready: 166
2026-04-20T10:40:35.362383324Z Waiting for performance profile collector pods to become ready: 167
2026-04-20T10:40:36.470076451Z Waiting for performance profile collector pods to become ready: 168
2026-04-20T10:40:37.576916499Z Waiting for performance profile collector pods to become ready: 169
2026-04-20T10:40:37.579830379Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:40:38.683768523Z Waiting for performance profile collector pods to become ready: 170
2026-04-20T10:40:39.792378414Z Waiting for performance profile collector pods to become ready: 171
2026-04-20T10:40:40.902279873Z Waiting for performance profile collector pods to become ready: 172
2026-04-20T10:40:42.008730968Z Waiting for performance profile collector pods to become ready: 173
2026-04-20T10:40:42.584515634Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:40:43.113264341Z Waiting for performance profile collector pods to become ready: 174
2026-04-20T10:40:44.221829069Z Waiting for performance profile collector pods to become ready: 175
2026-04-20T10:40:45.329126206Z Waiting for performance profile collector pods to become ready: 176
2026-04-20T10:40:46.436027835Z Waiting for performance profile collector pods to become ready: 177
2026-04-20T10:40:47.545272932Z Waiting for performance profile collector pods to become ready: 178
2026-04-20T10:40:47.589379169Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:40:48.653293068Z Waiting for performance profile collector pods to become ready: 179
2026-04-20T10:40:49.762029051Z Waiting for performance profile collector pods to become ready: 180
2026-04-20T10:40:50.868912416Z Waiting for performance profile collector pods to become ready: 181
2026-04-20T10:40:51.982170103Z Waiting for performance profile collector pods to become ready: 182
2026-04-20T10:40:52.594217150Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:40:53.092903615Z Waiting for performance profile collector pods to become ready: 183
2026-04-20T10:40:54.201526887Z Waiting for performance profile collector pods to become ready: 184
2026-04-20T10:40:55.308711884Z Waiting for performance profile collector pods to become ready: 185
2026-04-20T10:40:56.417746311Z Waiting for performance profile collector pods to become ready: 186
2026-04-20T10:40:57.530856791Z Waiting for performance profile collector pods to become ready: 187
2026-04-20T10:40:57.598995751Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:40:58.644015948Z Waiting for performance profile collector pods to become ready: 188
2026-04-20T10:40:59.750361801Z Waiting for performance profile collector pods to become ready: 189
2026-04-20T10:41:00.859114419Z Waiting for performance profile collector pods to become ready: 190
2026-04-20T10:41:01.961654991Z Waiting for performance profile collector pods to become ready: 191
2026-04-20T10:41:02.603733895Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:41:03.069570197Z Waiting for performance profile collector pods to become ready: 192
2026-04-20T10:41:04.175957540Z Waiting for performance profile collector pods to become ready: 193
2026-04-20T10:41:05.284061297Z Waiting for performance profile collector pods to become ready: 194
2026-04-20T10:41:06.394845724Z Waiting for performance profile collector pods to become ready: 195
2026-04-20T10:41:07.504458712Z Waiting for performance profile collector pods to become ready: 196
2026-04-20T10:41:07.608487137Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:41:08.611056167Z Waiting for performance profile collector pods to become ready: 197
2026-04-20T10:41:09.721913710Z Waiting for performance profile collector pods to become ready: 198
2026-04-20T10:41:10.830663958Z Waiting for performance profile collector pods to become ready: 199
2026-04-20T10:41:11.937502587Z Waiting for performance profile collector pods to become ready: 200
2026-04-20T10:41:12.613233731Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:41:13.044407451Z Waiting for performance profile collector pods to become ready: 201
2026-04-20T10:41:14.154529622Z Waiting for performance profile collector pods to become ready: 202
2026-04-20T10:41:15.259914506Z Waiting for performance profile collector pods to become ready: 203
2026-04-20T10:41:16.372722125Z Waiting for performance profile collector pods to become ready: 204
2026-04-20T10:41:17.480992466Z Waiting for performance profile collector pods to become ready: 205
2026-04-20T10:41:17.617964919Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:41:18.587518034Z Waiting for performance profile collector pods to become ready: 206
2026-04-20T10:41:19.695545590Z Waiting for performance profile collector pods to become ready: 207
2026-04-20T10:41:20.801985449Z Waiting for performance profile collector pods to become ready: 208
2026-04-20T10:41:21.908773977Z Waiting for performance profile collector pods to become ready: 209
2026-04-20T10:41:22.622638413Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:41:23.014364578Z Waiting for performance profile collector pods to become ready: 210
2026-04-20T10:41:24.124636704Z Waiting for performance profile collector pods to become ready: 211
2026-04-20T10:41:25.234313574Z Waiting for performance profile collector pods to become ready: 212
2026-04-20T10:41:26.341086231Z Waiting for performance profile collector pods to become ready: 213
2026-04-20T10:41:27.451391983Z Waiting for performance profile collector pods to become ready: 214
2026-04-20T10:41:27.627399241Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:41:28.557272495Z Waiting for performance profile collector pods to become ready: 215
2026-04-20T10:41:29.665451237Z Waiting for performance profile collector pods to become ready: 216
2026-04-20T10:41:30.769329827Z Waiting for performance profile collector pods to become ready: 217
2026-04-20T10:41:31.877641628Z Waiting for performance profile collector pods to become ready: 218
2026-04-20T10:41:32.632043721Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:41:32.984267363Z Waiting for performance profile collector pods to become ready: 219
2026-04-20T10:41:34.093524362Z Waiting for performance profile collector pods to become ready: 220
2026-04-20T10:41:35.201815672Z Waiting for performance profile collector pods to become ready: 221
2026-04-20T10:41:36.306784567Z Waiting for performance profile collector pods to become ready: 222
2026-04-20T10:41:37.413786411Z Waiting for performance profile collector pods to become ready: 223
2026-04-20T10:41:37.636737827Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:41:38.526812694Z Waiting for performance profile collector pods to become ready: 224
2026-04-20T10:41:39.637908260Z Waiting for performance profile collector pods to become ready: 225
2026-04-20T10:41:40.747436986Z Waiting for performance profile collector pods to become ready: 226
2026-04-20T10:41:41.856631435Z Waiting for performance profile collector pods to become ready: 227
2026-04-20T10:41:42.641478400Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:41:42.966645330Z Waiting for performance profile collector pods to become ready: 228
2026-04-20T10:41:44.079744407Z Waiting for performance profile collector pods to become ready: 229
2026-04-20T10:41:45.187737846Z Waiting for performance profile collector pods to become ready: 230
2026-04-20T10:41:46.299047990Z Waiting for performance profile collector pods to become ready: 231
2026-04-20T10:41:47.411075332Z Waiting for performance profile collector pods to become ready: 232
2026-04-20T10:41:47.646214668Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:41:48.520159796Z Waiting for performance profile collector pods to become ready: 233
2026-04-20T10:41:49.627291896Z Waiting for performance profile collector pods to become ready: 234
2026-04-20T10:41:50.740597006Z Waiting for performance profile collector pods to become ready: 235
2026-04-20T10:41:51.846732773Z Waiting for performance profile collector pods to become ready: 236
2026-04-20T10:41:52.650983510Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:41:52.959360529Z Waiting for performance profile collector pods to become ready: 237
2026-04-20T10:41:54.083879693Z Waiting for performance profile collector pods to become ready: 238
2026-04-20T10:41:55.194807792Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-20T10:41:55.422120243Z Collecting performance related data for node ip-10-0-140-95.ec2.internal
2026-04-20T10:41:55.429627756Z Collecting performance related data for node ip-10-0-137-106.ec2.internal
2026-04-20T10:41:55.437340025Z Collecting performance related data for node ip-10-0-138-148.ec2.internal
2026-04-20T10:41:57.655975968Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:42:02.660858133Z [disk usage checker] Volume usage percentage: current = 39 ; allowed = 70
2026-04-20T10:42:03.424072738Z Collecting kubelet logs for node ip-10-0-140-95.ec2.internal
2026-04-20T10:42:03.425898374Z Collecting kubelet logs for node ip-10-0-137-106.ec2.internal
2026-04-20T10:42:03.427648215Z Collecting kubelet logs for node ip-10-0-138-148.ec2.internal
2026-04-20T10:42:03.747192753Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-20T10:42:03.752961431Z INFO: Node performance data collection complete.
2026-04-20T10:42:03.885136091Z error: the server doesn't have a resource type "clusters"
2026-04-20T10:42:03.915861574Z Caches written to disk