2026-04-16T20:41:01.148110912Z [disk usage checker] Started
2026-04-16T20:41:01.151489935Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T20:41:01.354870682Z Gathering data for ns/openshift-cluster-version...
2026-04-16T20:41:01.500994760Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T20:41:01.582415802Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T20:41:01.659747227Z Gathering data for ns/default...
2026-04-16T20:41:01.954726764Z Gathering data for ns/openshift...
2026-04-16T20:41:02.345152798Z Gathering data for ns/kube-system...
2026-04-16T20:41:03.045657539Z Gathering data for ns/openshift-etcd...
2026-04-16T20:41:03.347936861Z Wrote inspect data to must-gather.
2026-04-16T20:41:03.348094371Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T20:41:03.348094371Z     namespaces "assisted-installer" not found
2026-04-16T20:41:03.615065736Z Waiting on subprocesses to finish execution.
2026-04-16T20:41:03.624138205Z INFO: Collecting host service logs for crio
2026-04-16T20:41:03.624469626Z INFO: Collecting host service logs for kubelet
2026-04-16T20:41:03.624807574Z INFO: Collecting host service logs for rpm-ostreed
2026-04-16T20:41:03.625169355Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-16T20:41:03.625510230Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-16T20:41:03.625866583Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-16T20:41:03.626205205Z INFO: Collecting host service logs for NetworkManager
2026-04-16T20:41:03.626536087Z INFO: Collecting host service logs for openvswitch
2026-04-16T20:41:03.626877324Z INFO: Collecting host service logs for ovs-configuration
2026-04-16T20:41:03.627278885Z INFO: Collecting host service logs for ovsdb-server
2026-04-16T20:41:03.627672297Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-16T20:41:03.628178261Z INFO: Waiting for worker host service log collection to complete ...
2026-04-16T20:41:03.629364721Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-16T20:41:03.630112153Z INFO: Gathering HAProxy config files
2026-04-16T20:41:03.632196018Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-16T20:41:03.661012784Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-16T20:41:03.736492421Z INFO: Waiting for node performance related collection to complete ...
2026-04-16T20:41:04.440279760Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-16T20:41:04.440312097Z INFO: on-disk MachineConfig config collection complete.
2026-04-16T20:41:04.487478262Z No resources found
2026-04-16T20:41:04.551347267Z No resources found in openshift-etcd namespace.
2026-04-16T20:41:04.636465402Z Wrote inspect data to must-gather.
2026-04-16T20:41:04.738341223Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-16T20:41:04.784077784Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-mwqfp
2026-04-16T20:41:04.784128852Z insights-runtime-extractor-89xkw
2026-04-16T20:41:04.784148540Z insights-runtime-extractor-9wh9h
2026-04-16T20:41:04.784168069Z insights-runtime-extractor-qsrkv
2026-04-16T20:41:04.785315239Z error: the server doesn't have a resource type "performanceprofile"
2026-04-16T20:41:04.830268086Z INFO: "metallb-operator" not detected. Skipping.
2026-04-16T20:41:04.874157363Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-16T20:41:04.899095549Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-16T20:41:04.949504471Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-16T20:41:05.065803007Z INFO: Worker host service log collection to complete.
2026-04-16T20:41:05.082085383Z Wrote inspect data to must-gather.
2026-04-16T20:41:05.116841377Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-16T20:41:05.116841377Z See 'oc rsync -h' for help and examples
2026-04-16T20:41:05.128949485Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-16T20:41:05.204679674Z ERROR: No running kube-apiserver pods found
2026-04-16T20:41:05.375010451Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-16T20:41:05.604267366Z tar: Removing leading `/' from member names
2026-04-16T20:41:05.613538938Z INFO: HAProxy config collection complete.
2026-04-16T20:41:05.619517976Z Gathering data for ns/openshift-monitoring...
2026-04-16T20:41:05.689928220Z Wrote inspect data to must-gather.
2026-04-16T20:41:05.760116768Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-16T20:41:05.765229175Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T20:41:05.824877631Z INFO: 'previous-logs' folder not found on ip-10-0-128-201.ec2.internal, skipping...
2026-04-16T20:41:05.866461137Z Wrote inspect data to must-gather.
2026-04-16T20:41:05.911344207Z No resources found
2026-04-16T20:41:05.960604359Z Wrote inspect data to must-gather.
2026-04-16T20:41:05.973280895Z Wrote inspect data to must-gather.
2026-04-16T20:41:06.013601396Z Wrote inspect data to must-gather.
2026-04-16T20:41:06.058652916Z error: the server doesn't have a resource type "machineconfigs"
2026-04-16T20:41:06.138884407Z INFO: 'previous-logs' folder not found on ip-10-0-130-164.ec2.internal, skipping...
2026-04-16T20:41:06.160962596Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T20:41:06.246865344Z INFO: Getting rules from prometheus-k8s-0
2026-04-16T20:41:06.253955329Z Wrote inspect data to must-gather.
2026-04-16T20:41:06.308231010Z Wrote inspect data to must-gather.
2026-04-16T20:41:06.399853580Z INFO: 'previous-logs' folder not found on ip-10-0-140-191.ec2.internal, skipping...
2026-04-16T20:41:06.399889096Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-16T20:41:06.399918019Z INFO: Machine Config Daemon termination log collection complete.
2026-04-16T20:41:06.519768276Z Wrote inspect data to must-gather.
2026-04-16T20:41:06.531575899Z Wrote inspect data to must-gather.
2026-04-16T20:41:06.666483504Z INFO: Getting status/config from prometheus-k8s-0
2026-04-16T20:41:06.681100876Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-16T20:41:06.760346035Z Wrote inspect data to must-gather.
2026-04-16T20:41:06.894255643Z Wrote inspect data to must-gather.
2026-04-16T20:41:07.068656035Z INFO: INTERCONNECT MODE
2026-04-16T20:41:07.068656035Z INFO: Gathering ovn-kubernetes DBs
2026-04-16T20:41:07.097977475Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T20:41:07.124913951Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-16T20:41:07.139762863Z Wrote inspect data to must-gather.
2026-04-16T20:41:07.246248059Z INFO: Gathering OVN_Northbound from ovnkube-node-cspns...
2026-04-16T20:41:07.246597392Z INFO: Gathering OVN_Northbound from ovnkube-node-k4p2g...
2026-04-16T20:41:07.246901441Z INFO: Gathering OVN_Northbound from ovnkube-node-t5dmh...
2026-04-16T20:41:07.247231205Z INFO: Gathering OVN_Southbound from ovnkube-node-cspns...
2026-04-16T20:41:07.247488530Z INFO: Gathering OVN_Southbound from ovnkube-node-k4p2g...
2026-04-16T20:41:07.247764616Z INFO: Gathering OVN_Southbound from ovnkube-node-t5dmh...
2026-04-16T20:41:07.423377966Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-16T20:41:07.495634662Z Wrote inspect data to must-gather.
2026-04-16T20:41:07.527940330Z tar: Removing leading `/' from member names
2026-04-16T20:41:07.531902053Z tar: Removing leading `/' from member names
2026-04-16T20:41:07.549660117Z tar: Removing leading `/' from member names
2026-04-16T20:41:07.557598682Z tar: Removing leading `/' from member names
2026-04-16T20:41:07.588489843Z tar: Removing leading `/' from member names
2026-04-16T20:41:07.589446915Z tar: Removing leading `/' from member names
2026-04-16T20:41:07.719576588Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-16T20:41:07.753506715Z W0416 20:41:07.753444    1285 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-04-16T20:41:07.761471232Z Wrote inspect data to must-gather.
2026-04-16T20:41:07.886156379Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T20:41:08.013208739Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-16T20:41:08.091026042Z Gathering data for ns/openshift-network-console...
2026-04-16T20:41:08.095273874Z INFO: Waiting for network log collection to complete ...
2026-04-16T20:41:08.095299798Z INFO: Waiting for ovnk database copies to complete ...
2026-04-16T20:41:08.095675424Z INFO: Copying ovnk databases complete.
2026-04-16T20:41:08.097288039Z 3.9M	must-gather/network_logs/ovnk_database_store
2026-04-16T20:41:08.099346085Z ovnk_database_store/
2026-04-16T20:41:08.099346085Z ovnk_database_store/ovnkube-node-k4p2g_sbdb
2026-04-16T20:41:08.124471087Z Wrote inspect data to must-gather.
2026-04-16T20:41:08.124682801Z ovnk_database_store/ovnkube-node-k4p2g_nbdb
2026-04-16T20:41:08.136478115Z ovnk_database_store/ovnkube-node-cspns_sbdb
2026-04-16T20:41:08.164809296Z ovnk_database_store/ovnkube-node-cspns_nbdb
2026-04-16T20:41:08.175673993Z ovnk_database_store/ovnkube-node-t5dmh_sbdb
2026-04-16T20:41:08.198219622Z ovnk_database_store/ovnkube-node-t5dmh_nbdb
2026-04-16T20:41:08.212292621Z INFO: Network log collection complete.
2026-04-16T20:41:08.427668065Z INFO: Getting status from alertmanager-main-0
2026-04-16T20:41:08.447379983Z Wrote inspect data to must-gather.
2026-04-16T20:41:08.549981536Z Gathering data for ns/openshift-console-operator...
2026-04-16T20:41:08.735786712Z Wrote inspect data to must-gather.
2026-04-16T20:41:08.906146045Z Gathering data for ns/openshift-console...
2026-04-16T20:41:08.993253151Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-16T20:41:08.996069620Z INFO: Fallback to identify the container image from release info
2026-04-16T20:41:09.341546304Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-16T20:41:09.481518635Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-16T20:41:09.609802656Z daemonset.apps/perf-node-gather-daemonset created
2026-04-16T20:41:09.705278823Z Gathering data for ns/openshift-dns-operator...
2026-04-16T20:41:09.753349368Z Waiting for performance profile collector pods to become ready: 1
2026-04-16T20:41:09.974006532Z Gathering data for ns/openshift-dns...
2026-04-16T20:41:10.536153442Z Gathering data for ns/openshift-image-registry...
2026-04-16T20:41:10.877487900Z Waiting for performance profile collector pods to become ready: 2
2026-04-16T20:41:10.991857432Z Gathering data for ns/openshift-ingress-operator...
2026-04-16T20:41:11.166226380Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T20:41:11.296297979Z Gathering data for ns/openshift-ingress...
2026-04-16T20:41:11.597429807Z Gathering data for ns/openshift-ingress-canary...
2026-04-16T20:41:11.973360757Z Gathering data for ns/openshift-insights...
2026-04-16T20:41:12.013942461Z Waiting for performance profile collector pods to become ready: 3
2026-04-16T20:41:13.126603662Z Waiting for performance profile collector pods to become ready: 4
2026-04-16T20:41:14.182756069Z Gathering data for ns/kserve...
2026-04-16T20:41:14.257201706Z Waiting for performance profile collector pods to become ready: 5
2026-04-16T20:41:15.382487635Z Waiting for performance profile collector pods to become ready: 6
2026-04-16T20:41:15.947847013Z Gathering data for ns/openshift-config...
2026-04-16T20:41:16.170911394Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T20:41:16.195268029Z Gathering data for ns/openshift-config-managed...
2026-04-16T20:41:16.499690355Z Waiting for performance profile collector pods to become ready: 7
2026-04-16T20:41:16.606939925Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-16T20:41:16.909347950Z Gathering data for ns/openshift-kube-apiserver...
2026-04-16T20:41:17.177407024Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-16T20:41:17.505435286Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-16T20:41:17.636965731Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-16T20:41:17.821696395Z Gathering data for ns/openshift-kube-scheduler...
2026-04-16T20:41:17.932221755Z Collecting performance related data for node ip-10-0-128-201.ec2.internal
2026-04-16T20:41:17.938896001Z Collecting performance related data for node ip-10-0-130-164.ec2.internal
2026-04-16T20:41:17.945847345Z Collecting performance related data for node ip-10-0-140-191.ec2.internal
2026-04-16T20:41:18.135955792Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-16T20:41:18.455775433Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-16T20:41:18.856427035Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-16T20:41:19.245366510Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-16T20:41:19.801142854Z Gathering data for ns/openshift-multus...
2026-04-16T20:41:21.175819462Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T20:41:21.227166728Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-16T20:41:22.740095582Z Gathering data for ns/openshift-host-network...
2026-04-16T20:41:23.031614981Z Gathering data for ns/openshift-network-diagnostics...
2026-04-16T20:41:23.549068693Z Gathering data for ns/openshift-network-node-identity...
2026-04-16T20:41:23.934092834Z Gathering data for ns/openshift-network-operator...
2026-04-16T20:41:24.307965945Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-16T20:41:24.566081755Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-16T20:41:25.021015572Z Gathering data for ns/openshift-apiserver-operator...
2026-04-16T20:41:25.296534225Z Gathering data for ns/openshift-apiserver...
2026-04-16T20:41:25.665922928Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-16T20:41:25.933813021Z Gathering data for ns/openshift-controller-manager...
2026-04-16T20:41:26.139896111Z Collecting kubelet logs for node ip-10-0-128-201.ec2.internal
2026-04-16T20:41:26.142421253Z Collecting kubelet logs for node ip-10-0-130-164.ec2.internal
2026-04-16T20:41:26.144115404Z Collecting kubelet logs for node ip-10-0-140-191.ec2.internal
2026-04-16T20:41:26.185394946Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T20:41:26.211862389Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-16T20:41:26.520687054Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-16T20:41:26.525955937Z INFO: Node performance data collection complete.
2026-04-16T20:41:26.897740055Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-16T20:41:27.169695700Z Gathering data for ns/openshift-service-ca-operator...
2026-04-16T20:41:27.476374823Z Gathering data for ns/openshift-service-ca...
2026-04-16T20:41:27.813387060Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-16T20:41:28.650592107Z Wrote inspect data to must-gather.
2026-04-16T20:41:28.650639622Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T20:41:28.650639622Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-16T20:41:28.782731923Z error: the server doesn't have a resource type "clusters"
2026-04-16T20:41:28.929113033Z Caches written to disk