2026-05-29T17:33:37.241361046Z [disk usage checker] Started
2026-05-29T17:33:37.245043403Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-29T17:33:37.478783921Z Gathering data for ns/openshift-cluster-version...
2026-05-29T17:33:37.642615516Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-29T17:33:37.735384418Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-29T17:33:37.833429933Z Gathering data for ns/default...
2026-05-29T17:33:38.422901896Z Gathering data for ns/openshift...
2026-05-29T17:33:38.884677114Z Gathering data for ns/kube-system...
2026-05-29T17:33:39.562888795Z Gathering data for ns/openshift-etcd...
2026-05-29T17:33:39.692091425Z Waiting on subprocesses to finish execution.
2026-05-29T17:33:39.695288585Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-29T17:33:39.700993885Z INFO: Gathering HAProxy config files
2026-05-29T17:33:39.701749049Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-29T17:33:39.707738846Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-29T17:33:39.715703025Z INFO: Collecting host service logs for crio
2026-05-29T17:33:39.716007827Z INFO: Collecting host service logs for kubelet
2026-05-29T17:33:39.716305630Z INFO: Collecting host service logs for rpm-ostreed
2026-05-29T17:33:39.716591629Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-29T17:33:39.716908310Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-29T17:33:39.717200023Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-29T17:33:39.717489726Z INFO: Collecting host service logs for NetworkManager
2026-05-29T17:33:39.717800108Z INFO: Collecting host service logs for openvswitch
2026-05-29T17:33:39.718714857Z INFO: Collecting host service logs for ovs-configuration
2026-05-29T17:33:39.719087749Z INFO: Collecting host service logs for ovsdb-server
2026-05-29T17:33:39.719380769Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-29T17:33:39.719807387Z INFO: Waiting for worker host service log collection to complete ...
2026-05-29T17:33:39.765361028Z INFO: Waiting for node performance related collection to complete ...
2026-05-29T17:33:40.557737818Z No resources found in openshift-etcd namespace.
2026-05-29T17:33:40.661187470Z INFO: "metallb-operator" not detected. Skipping.
2026-05-29T17:33:40.670440093Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-29T17:33:40.684317949Z No resources found
2026-05-29T17:33:40.717052068Z INFO: Collecting Insights Archives from insights-operator-788bf58df5-jvk9t
2026-05-29T17:33:40.717105859Z insights-runtime-extractor-5wsgv
2026-05-29T17:33:40.717125965Z insights-runtime-extractor-n9wt9
2026-05-29T17:33:40.717145461Z insights-runtime-extractor-wnfnm
2026-05-29T17:33:40.772256410Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-29T17:33:40.772350851Z INFO: on-disk MachineConfig config collection complete.
2026-05-29T17:33:40.892717819Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-29T17:33:41.020853803Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-29T17:33:41.059952337Z Wrote inspect data to must-gather.
2026-05-29T17:33:41.148042351Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-29T17:33:41.193202198Z error: the server doesn't have a resource type "performanceprofile"
2026-05-29T17:33:41.525051839Z ERROR: No running kube-apiserver pods found
2026-05-29T17:33:41.560744208Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-29T17:33:41.726376732Z INFO: Worker host service log collection to complete.
2026-05-29T17:33:41.965350495Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-29T17:33:41.965350495Z See 'oc rsync -h' for help and examples
2026-05-29T17:33:42.047241285Z Wrote inspect data to must-gather.
2026-05-29T17:33:42.047350407Z error: inspection completed with the errors occurred while gathering data:
2026-05-29T17:33:42.047350407Z     namespaces "assisted-installer" not found
2026-05-29T17:33:42.164286849Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-29T17:33:42.258658288Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-29T17:33:42.264535966Z Wrote inspect data to must-gather.
2026-05-29T17:33:42.281494748Z tar: Removing leading `/' from member names
2026-05-29T17:33:42.287577476Z INFO: HAProxy config collection complete.
2026-05-29T17:33:42.358244656Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-29T17:33:42.372290678Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-29T17:33:42.416807185Z No resources found
2026-05-29T17:33:42.633508241Z Wrote inspect data to must-gather.
2026-05-29T17:33:42.641303801Z INFO: 'previous-logs' folder not found on ip-10-0-128-169.ec2.internal, skipping...
2026-05-29T17:33:42.641955214Z Wrote inspect data to must-gather.
2026-05-29T17:33:42.747968476Z Wrote inspect data to must-gather.
2026-05-29T17:33:42.751098757Z Wrote inspect data to must-gather.
2026-05-29T17:33:42.997700894Z INFO: 'previous-logs' folder not found on ip-10-0-137-133.ec2.internal, skipping...
2026-05-29T17:33:43.008661295Z error: the server doesn't have a resource type "machineconfigs"
2026-05-29T17:33:43.040713070Z INFO: Getting rules from prometheus-k8s-0
2026-05-29T17:33:43.078696798Z Gathering data for ns/kuadrant-system...
2026-05-29T17:33:43.286371176Z Wrote inspect data to must-gather.
2026-05-29T17:33:43.366345630Z INFO: 'previous-logs' folder not found on ip-10-0-141-197.ec2.internal, skipping...
2026-05-29T17:33:43.366541352Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-29T17:33:43.366541352Z INFO: Machine Config Daemon termination log collection complete.
2026-05-29T17:33:43.549557917Z INFO: Getting status/config from prometheus-k8s-0
2026-05-29T17:33:43.567217908Z Wrote inspect data to must-gather.
2026-05-29T17:33:43.909388403Z Wrote inspect data to must-gather.
2026-05-29T17:33:44.106297452Z Wrote inspect data to must-gather.
2026-05-29T17:33:44.113089784Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-29T17:33:44.289046803Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-29T17:33:44.343251807Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-29T17:33:44.420863060Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-29T17:33:44.513079511Z Wrote inspect data to must-gather.
2026-05-29T17:33:44.709794551Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-29T17:33:44.813422728Z W0529 17:33:44.813360    1143 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-05-29T17:33:44.849414665Z Wrote inspect data to must-gather.
2026-05-29T17:33:44.876870838Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-29T17:33:44.902825434Z INFO: INTERCONNECT MODE
2026-05-29T17:33:44.902874534Z INFO: Gathering ovn-kubernetes DBs
2026-05-29T17:33:45.010012605Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-29T17:33:45.127290785Z Gathering data for ns/openshift-monitoring...
2026-05-29T17:33:45.144772675Z INFO: Gathering OVN_Northbound from ovnkube-node-bf6q2...
2026-05-29T17:33:45.145125247Z INFO: Gathering OVN_Northbound from ovnkube-node-gpcnk...
2026-05-29T17:33:45.145408773Z INFO: Gathering OVN_Northbound from ovnkube-node-nfv6s...
2026-05-29T17:33:45.145727356Z INFO: Gathering OVN_Southbound from ovnkube-node-bf6q2...
2026-05-29T17:33:45.146020360Z INFO: Gathering OVN_Southbound from ovnkube-node-gpcnk...
2026-05-29T17:33:45.146290682Z INFO: Gathering OVN_Southbound from ovnkube-node-nfv6s...
2026-05-29T17:33:45.483898242Z tar: Removing leading `/' from member names
2026-05-29T17:33:45.560318262Z tar: Removing leading `/' from member names
2026-05-29T17:33:45.573492105Z tar: Removing leading `/' from member names
2026-05-29T17:33:45.590464374Z tar: Removing leading `/' from member names
2026-05-29T17:33:45.627516919Z tar: Removing leading `/' from member names
2026-05-29T17:33:45.627677669Z tar: Removing leading `/' from member names
2026-05-29T17:33:45.645615094Z Wrote inspect data to must-gather.
2026-05-29T17:33:45.680656779Z INFO: Getting status from alertmanager-main-0
2026-05-29T17:33:46.195062467Z INFO: Waiting for network log collection to complete ...
2026-05-29T17:33:46.195111553Z INFO: Waiting for ovnk database copies to complete ...
2026-05-29T17:33:46.195629990Z INFO: Copying ovnk databases complete.
2026-05-29T17:33:46.197542114Z 4.4M	must-gather/network_logs/ovnk_database_store
2026-05-29T17:33:46.200336993Z ovnk_database_store/
2026-05-29T17:33:46.200367656Z ovnk_database_store/ovnkube-node-nfv6s_sbdb
2026-05-29T17:33:46.223340670Z ovnk_database_store/ovnkube-node-nfv6s_nbdb
2026-05-29T17:33:46.228009625Z Wrote inspect data to must-gather.
2026-05-29T17:33:46.233824227Z ovnk_database_store/ovnkube-node-gpcnk_sbdb
2026-05-29T17:33:46.259821700Z ovnk_database_store/ovnkube-node-gpcnk_nbdb
2026-05-29T17:33:46.271332821Z ovnk_database_store/ovnkube-node-bf6q2_sbdb
2026-05-29T17:33:46.295565520Z ovnk_database_store/ovnkube-node-bf6q2_nbdb
2026-05-29T17:33:46.309821555Z INFO: Network log collection complete.
2026-05-29T17:33:46.723821666Z Wrote inspect data to must-gather.
2026-05-29T17:33:47.257263340Z Wrote inspect data to must-gather.
2026-05-29T17:33:47.349354074Z Gathering data for ns/openshift-network-console...
2026-05-29T17:33:47.401486777Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-29T17:33:47.853327457Z Wrote inspect data to must-gather.
2026-05-29T17:33:47.934028836Z Gathering data for ns/openshift-console-operator...
2026-05-29T17:33:48.426266568Z Gathering data for ns/openshift-console...
2026-05-29T17:33:48.442554295Z Wrote inspect data to must-gather.
2026-05-29T17:33:48.738188019Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-29T17:33:48.742249134Z INFO: Fallback to identify the container image from release info
2026-05-29T17:33:49.000562905Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-29T17:33:49.445658447Z Gathering data for ns/openshift-dns-operator...
2026-05-29T17:33:49.447342216Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d8381cb85e479760bd6b601e111b39e6c56a632bfa298867f4197ec4421e9f5
2026-05-29T17:33:49.606729007Z daemonset.apps/perf-node-gather-daemonset created
2026-05-29T17:33:49.760184528Z Waiting for performance profile collector pods to become ready: 1
2026-05-29T17:33:49.883890624Z Gathering data for ns/openshift-dns...
2026-05-29T17:33:50.257467688Z Wrote inspect data to must-gather.
2026-05-29T17:33:50.545633471Z Gathering data for ns/openshift-image-registry...
2026-05-29T17:33:50.882197314Z Waiting for performance profile collector pods to become ready: 2
2026-05-29T17:33:51.049856545Z Gathering data for ns/openshift-ingress-operator...
2026-05-29T17:33:51.404476303Z Gathering data for ns/openshift-ingress...
2026-05-29T17:33:52.011007787Z Waiting for performance profile collector pods to become ready: 3
2026-05-29T17:33:52.178506055Z Gathering data for ns/openshift-ingress-canary...
2026-05-29T17:33:52.408754822Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-29T17:33:52.633665735Z Gathering data for ns/openshift-insights...
2026-05-29T17:33:53.141235779Z Waiting for performance profile collector pods to become ready: 4
2026-05-29T17:33:54.259060466Z Waiting for performance profile collector pods to become ready: 5
2026-05-29T17:33:54.746566803Z Gathering data for ns/opendatahub...
2026-05-29T17:33:55.379796026Z Waiting for performance profile collector pods to become ready: 6
2026-05-29T17:33:56.500748970Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-29T17:33:56.594423749Z Gathering data for ns/openshift-lws-operator...
2026-05-29T17:33:56.765002396Z Collecting performance related data for node ip-10-0-141-197.ec2.internal
2026-05-29T17:33:56.774113036Z Collecting performance related data for node ip-10-0-137-133.ec2.internal
2026-05-29T17:33:56.781385329Z Collecting performance related data for node ip-10-0-128-169.ec2.internal
2026-05-29T17:33:57.414913547Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-29T17:33:58.463137612Z Gathering data for ns/openshift-config...
2026-05-29T17:33:58.829125497Z Gathering data for ns/openshift-config-managed...
2026-05-29T17:33:59.321631825Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-29T17:33:59.653888144Z Gathering data for ns/openshift-kube-apiserver...
2026-05-29T17:34:00.012281638Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-29T17:34:00.336186045Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-29T17:34:00.698189961Z Gathering data for ns/openshift-kube-scheduler...
2026-05-29T17:34:01.011224939Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-29T17:34:01.405203460Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-29T17:34:01.775609367Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-29T17:34:02.141991526Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-29T17:34:02.421534920Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-29T17:34:02.850920874Z Gathering data for ns/openshift-multus...
2026-05-29T17:34:04.379550661Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-29T17:34:04.835487376Z Collecting kubelet logs for node ip-10-0-141-197.ec2.internal
2026-05-29T17:34:04.837522803Z Collecting kubelet logs for node ip-10-0-137-133.ec2.internal
2026-05-29T17:34:04.839693896Z Collecting kubelet logs for node ip-10-0-128-169.ec2.internal
2026-05-29T17:34:05.199244021Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-29T17:34:05.207612178Z INFO: Node performance data collection complete.
2026-05-29T17:34:05.850778399Z Gathering data for ns/openshift-host-network...
2026-05-29T17:34:06.198740923Z Gathering data for ns/openshift-network-diagnostics...
2026-05-29T17:34:06.753793343Z Gathering data for ns/openshift-network-node-identity...
2026-05-29T17:34:07.217780429Z Gathering data for ns/openshift-network-operator...
2026-05-29T17:34:07.426730439Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-29T17:34:07.609878175Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-29T17:34:07.968852879Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-29T17:34:08.449758633Z Gathering data for ns/openshift-apiserver-operator...
2026-05-29T17:34:08.771030917Z Gathering data for ns/openshift-apiserver...
2026-05-29T17:34:09.200200743Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-29T17:34:09.515968092Z Gathering data for ns/openshift-controller-manager...
2026-05-29T17:34:09.840533641Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-29T17:34:10.516885903Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-29T17:34:10.839645745Z Gathering data for ns/openshift-service-ca-operator...
2026-05-29T17:34:11.179726734Z Gathering data for ns/openshift-service-ca...
2026-05-29T17:34:11.565420601Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-29T17:34:12.431214952Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-29T17:34:12.676866478Z Wrote inspect data to must-gather.
2026-05-29T17:34:12.676914385Z error: inspection completed with the errors occurred while gathering data:
2026-05-29T17:34:12.676914385Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-29T17:34:12.815539373Z error: the server doesn't have a resource type "clusters"
2026-05-29T17:34:13.004918181Z Caches written to disk