2026-05-28T08:41:45.836424865Z [disk usage checker] Started
2026-05-28T08:41:45.840072211Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-28T08:41:46.055341726Z Gathering data for ns/openshift-cluster-version...
2026-05-28T08:41:46.242001094Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-28T08:41:46.360488496Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-28T08:41:46.454759278Z Gathering data for ns/default...
2026-05-28T08:41:46.761169321Z Gathering data for ns/openshift...
2026-05-28T08:41:47.232973235Z Gathering data for ns/kube-system...
2026-05-28T08:41:48.077222964Z Gathering data for ns/openshift-etcd...
2026-05-28T08:41:48.235305189Z Waiting on subprocesses to finish execution.
2026-05-28T08:41:48.243742066Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-28T08:41:48.244874698Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-28T08:41:48.246890155Z INFO: Gathering HAProxy config files
2026-05-28T08:41:48.248577396Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-28T08:41:48.257825293Z INFO: Collecting host service logs for crio
2026-05-28T08:41:48.258225773Z INFO: Collecting host service logs for kubelet
2026-05-28T08:41:48.258575881Z INFO: Collecting host service logs for rpm-ostreed
2026-05-28T08:41:48.259032084Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-28T08:41:48.259401205Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-28T08:41:48.259783824Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-28T08:41:48.260156955Z INFO: Collecting host service logs for NetworkManager
2026-05-28T08:41:48.260478855Z INFO: Collecting host service logs for openvswitch
2026-05-28T08:41:48.260859371Z INFO: Collecting host service logs for ovs-configuration
2026-05-28T08:41:48.261238424Z INFO: Collecting host service logs for ovsdb-server
2026-05-28T08:41:48.261586862Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-28T08:41:48.262037894Z INFO: Waiting for worker host service log collection to complete ...
2026-05-28T08:41:48.315395948Z INFO: Waiting for node performance related collection to complete ...
2026-05-28T08:41:48.948719013Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-28T08:41:49.115304232Z No resources found in openshift-etcd namespace.
2026-05-28T08:41:49.258242361Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-28T08:41:49.350892483Z Wrote inspect data to must-gather.
2026-05-28T08:41:49.350994588Z error: inspection completed with the errors occurred while gathering data:
2026-05-28T08:41:49.350994588Z     namespaces "assisted-installer" not found
2026-05-28T08:41:49.351715301Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-8w6wr
2026-05-28T08:41:49.351776128Z insights-runtime-extractor-9b87z
2026-05-28T08:41:49.351798179Z insights-runtime-extractor-t85hn
2026-05-28T08:41:49.351818611Z insights-runtime-extractor-v8cmf
2026-05-28T08:41:49.420648297Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-28T08:41:49.422288606Z INFO: on-disk MachineConfig config collection complete.
2026-05-28T08:41:49.465424795Z INFO: "metallb-operator" not detected. Skipping.
2026-05-28T08:41:49.500451594Z INFO: Worker host service log collection to complete.
2026-05-28T08:41:49.552312995Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-28T08:41:49.592909554Z No resources found
2026-05-28T08:41:49.602120673Z Wrote inspect data to must-gather.
2026-05-28T08:41:49.661257602Z error: the server doesn't have a resource type "performanceprofile"
2026-05-28T08:41:49.730280025Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-28T08:41:49.764925968Z Wrote inspect data to must-gather.
2026-05-28T08:41:49.773074253Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-28T08:41:50.038099943Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-28T08:41:50.038099943Z See 'oc rsync -h' for help and examples
2026-05-28T08:41:50.059087146Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-28T08:41:50.210348101Z ERROR: No running kube-apiserver pods found
2026-05-28T08:41:50.262242755Z Wrote inspect data to must-gather.
2026-05-28T08:41:50.435331953Z Wrote inspect data to must-gather.
2026-05-28T08:41:50.440695082Z Wrote inspect data to must-gather.
2026-05-28T08:41:50.460728883Z Wrote inspect data to must-gather.
2026-05-28T08:41:50.467249797Z Wrote inspect data to must-gather.
2026-05-28T08:41:50.487379420Z INFO: 'previous-logs' folder not found on ip-10-0-133-172.ec2.internal, skipping...
2026-05-28T08:41:50.487910402Z No resources found
2026-05-28T08:41:50.529702270Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-28T08:41:50.531966651Z tar: Removing leading `/' from member names
2026-05-28T08:41:50.540948486Z INFO: HAProxy config collection complete.
2026-05-28T08:41:50.549858903Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-28T08:41:50.642691777Z error: the server doesn't have a resource type "machineconfigs"
2026-05-28T08:41:50.760068363Z Gathering data for ns/openshift-monitoring...
2026-05-28T08:41:50.761066771Z Wrote inspect data to must-gather.
2026-05-28T08:41:50.790200143Z INFO: 'previous-logs' folder not found on ip-10-0-135-144.ec2.internal, skipping...
2026-05-28T08:41:50.845933668Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-28T08:41:50.861275414Z Wrote inspect data to must-gather.
2026-05-28T08:41:50.953570019Z INFO: Getting rules from prometheus-k8s-0
2026-05-28T08:41:51.057238197Z Wrote inspect data to must-gather.
2026-05-28T08:41:51.062413381Z INFO: 'previous-logs' folder not found on ip-10-0-137-158.ec2.internal, skipping...
2026-05-28T08:41:51.062445372Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-28T08:41:51.062479192Z INFO: Machine Config Daemon termination log collection complete.
2026-05-28T08:41:51.101045065Z Wrote inspect data to must-gather.
2026-05-28T08:41:51.254043275Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-28T08:41:51.295189273Z Wrote inspect data to must-gather.
2026-05-28T08:41:51.365883717Z INFO: Getting status/config from prometheus-k8s-0
2026-05-28T08:41:51.465006842Z Wrote inspect data to must-gather.
2026-05-28T08:41:51.565151626Z INFO: INTERCONNECT MODE
2026-05-28T08:41:51.565215273Z INFO: Gathering ovn-kubernetes DBs
2026-05-28T08:41:51.658109733Z Wrote inspect data to must-gather.
2026-05-28T08:41:51.711266450Z INFO: Gathering OVN_Northbound from ovnkube-node-gr4r5...
2026-05-28T08:41:51.711484912Z INFO: Gathering OVN_Northbound from ovnkube-node-hpqw6...
2026-05-28T08:41:51.711716807Z INFO: Gathering OVN_Northbound from ovnkube-node-r6pbc...
2026-05-28T08:41:51.711973835Z INFO: Gathering OVN_Southbound from ovnkube-node-gr4r5...
2026-05-28T08:41:51.712267155Z INFO: Gathering OVN_Southbound from ovnkube-node-hpqw6...
2026-05-28T08:41:51.712601922Z INFO: Gathering OVN_Southbound from ovnkube-node-r6pbc...
2026-05-28T08:41:51.858641410Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-28T08:41:51.971059605Z Wrote inspect data to must-gather.
2026-05-28T08:41:51.993787510Z tar: Removing leading `/' from member names
2026-05-28T08:41:52.000734946Z tar: Removing leading `/' from member names
2026-05-28T08:41:52.058339464Z tar: Removing leading `/' from member names
2026-05-28T08:41:52.070563313Z tar: Removing leading `/' from member names
2026-05-28T08:41:52.094801451Z tar: Removing leading `/' from member names
2026-05-28T08:41:52.094858644Z tar: Removing leading `/' from member names
2026-05-28T08:41:52.172728941Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-28T08:41:52.202732837Z Wrote inspect data to must-gather.
2026-05-28T08:41:52.424925399Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-28T08:41:52.435220806Z Wrote inspect data to must-gather.
2026-05-28T08:41:52.462656564Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-28T08:41:52.493730206Z INFO: Waiting for network log collection to complete ...
2026-05-28T08:41:52.493776881Z INFO: Waiting for ovnk database copies to complete ...
2026-05-28T08:41:52.494281797Z INFO: Copying ovnk databases complete.
2026-05-28T08:41:52.496438791Z 3.7M	must-gather/network_logs/ovnk_database_store
2026-05-28T08:41:52.498735502Z ovnk_database_store/
2026-05-28T08:41:52.498735502Z ovnk_database_store/ovnkube-node-hpqw6_sbdb
2026-05-28T08:41:52.558743914Z ovnk_database_store/ovnkube-node-hpqw6_nbdb
2026-05-28T08:41:52.568999379Z ovnk_database_store/ovnkube-node-r6pbc_sbdb
2026-05-28T08:41:52.587254026Z ovnk_database_store/ovnkube-node-r6pbc_nbdb
2026-05-28T08:41:52.595886030Z ovnk_database_store/ovnkube-node-gr4r5_sbdb
2026-05-28T08:41:52.616122320Z ovnk_database_store/ovnkube-node-gr4r5_nbdb
2026-05-28T08:41:52.626441051Z INFO: Network log collection complete.
2026-05-28T08:41:52.641071942Z Wrote inspect data to must-gather.
2026-05-28T08:41:52.740091173Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-28T08:41:52.827640150Z Wrote inspect data to must-gather.
2026-05-28T08:41:53.039240135Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-28T08:41:53.080104171Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-28T08:41:53.082717521Z INFO: Fallback to identify the container image from release info
2026-05-28T08:41:53.125136607Z INFO: Getting status from alertmanager-main-0
2026-05-28T08:41:53.218455392Z Gathering data for ns/openshift-network-console...
2026-05-28T08:41:53.649740591Z Gathering data for ns/openshift-console-operator...
2026-05-28T08:41:53.727112614Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda
2026-05-28T08:41:53.840967605Z daemonset.apps/perf-node-gather-daemonset created
2026-05-28T08:41:53.981421960Z Waiting for performance profile collector pods to become ready: 1
2026-05-28T08:41:54.137376484Z Gathering data for ns/openshift-console...
2026-05-28T08:41:54.631633972Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-28T08:41:54.975821229Z Gathering data for ns/openshift-dns-operator...
2026-05-28T08:41:55.099235101Z Waiting for performance profile collector pods to become ready: 2
2026-05-28T08:41:55.344786565Z Gathering data for ns/openshift-dns...
2026-05-28T08:41:55.851572954Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-28T08:41:55.967482424Z Gathering data for ns/openshift-image-registry...
2026-05-28T08:41:56.227893813Z Waiting for performance profile collector pods to become ready: 3
2026-05-28T08:41:56.457465531Z Gathering data for ns/openshift-ingress-operator...
2026-05-28T08:41:56.775283191Z Gathering data for ns/openshift-ingress...
2026-05-28T08:41:57.121938084Z Gathering data for ns/openshift-ingress-canary...
2026-05-28T08:41:57.349314442Z Waiting for performance profile collector pods to become ready: 4
2026-05-28T08:41:57.632305325Z Gathering data for ns/openshift-insights...
2026-05-28T08:41:58.468567066Z Waiting for performance profile collector pods to become ready: 5
2026-05-28T08:41:59.582155759Z Waiting for performance profile collector pods to become ready: 6
2026-05-28T08:41:59.863179785Z Gathering data for ns/openshift-config...
2026-05-28T08:42:00.137517749Z Gathering data for ns/openshift-config-managed...
2026-05-28T08:42:00.584893662Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-28T08:42:00.710027336Z Waiting for performance profile collector pods to become ready: 7
2026-05-28T08:42:00.856508017Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-28T08:42:00.959006096Z Gathering data for ns/openshift-kube-apiserver...
2026-05-28T08:42:01.245984520Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-28T08:42:01.591118811Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-28T08:42:01.830832551Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-28T08:42:01.960724202Z Gathering data for ns/openshift-kube-scheduler...
2026-05-28T08:42:02.094809331Z Collecting performance related data for node ip-10-0-133-172.ec2.internal
2026-05-28T08:42:02.101635953Z Collecting performance related data for node ip-10-0-135-144.ec2.internal
2026-05-28T08:42:02.108851409Z Collecting performance related data for node ip-10-0-137-158.ec2.internal
2026-05-28T08:42:02.283518723Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-28T08:42:02.636109381Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-28T08:42:03.062957503Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-28T08:42:03.485632730Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-28T08:42:04.172264588Z Gathering data for ns/openshift-multus...
2026-05-28T08:42:05.718255321Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-28T08:42:05.861253710Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-28T08:42:07.376542375Z Gathering data for ns/openshift-host-network...
2026-05-28T08:42:07.788019451Z Gathering data for ns/openshift-network-diagnostics...
2026-05-28T08:42:08.376849442Z Gathering data for ns/openshift-network-node-identity...
2026-05-28T08:42:08.845517463Z Gathering data for ns/openshift-network-operator...
2026-05-28T08:42:09.299688330Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-28T08:42:09.571941368Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-28T08:42:10.063946903Z Gathering data for ns/openshift-apiserver-operator...
2026-05-28T08:42:10.123596751Z Collecting kubelet logs for node ip-10-0-133-172.ec2.internal
2026-05-28T08:42:10.125872465Z Collecting kubelet logs for node ip-10-0-135-144.ec2.internal
2026-05-28T08:42:10.128016450Z Collecting kubelet logs for node ip-10-0-137-158.ec2.internal
2026-05-28T08:42:10.400441272Z Gathering data for ns/openshift-apiserver...
2026-05-28T08:42:10.479931492Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-28T08:42:10.484948132Z INFO: Node performance data collection complete.
2026-05-28T08:42:10.833751281Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-28T08:42:10.868784168Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-28T08:42:11.152772643Z Gathering data for ns/openshift-controller-manager...
2026-05-28T08:42:11.456088610Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-28T08:42:12.116426867Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-28T08:42:12.447652215Z Gathering data for ns/openshift-service-ca-operator...
2026-05-28T08:42:12.793235138Z Gathering data for ns/openshift-service-ca...
2026-05-28T08:42:13.217557425Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-28T08:42:14.219466785Z Wrote inspect data to must-gather.
2026-05-28T08:42:14.219507600Z error: inspection completed with the errors occurred while gathering data:
2026-05-28T08:42:14.219507600Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-28T08:42:14.351516458Z error: the server doesn't have a resource type "clusters"
2026-05-28T08:42:14.508906323Z Caches written to disk