2026-05-29T21:07:36.624307615Z [disk usage checker] Started
2026-05-29T21:07:36.627766594Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-29T21:07:36.852590105Z Gathering data for ns/openshift-cluster-version...
2026-05-29T21:07:37.048220664Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-29T21:07:37.168299397Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-29T21:07:37.280233295Z Gathering data for ns/default...
2026-05-29T21:07:37.800989425Z Gathering data for ns/openshift...
2026-05-29T21:07:38.321479174Z Gathering data for ns/kube-system...
2026-05-29T21:07:39.063950467Z Waiting on subprocesses to finish execution.
2026-05-29T21:07:39.067891271Z INFO: Gathering HAProxy config files
2026-05-29T21:07:39.072339576Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-29T21:07:39.092649482Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-29T21:07:39.097254116Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-29T21:07:39.102038206Z INFO: Collecting host service logs for crio
2026-05-29T21:07:39.102430330Z INFO: Collecting host service logs for kubelet
2026-05-29T21:07:39.102765745Z INFO: Collecting host service logs for rpm-ostreed
2026-05-29T21:07:39.103144183Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-29T21:07:39.103524352Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-29T21:07:39.103893529Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-29T21:07:39.104269196Z INFO: Collecting host service logs for NetworkManager
2026-05-29T21:07:39.104617128Z INFO: Collecting host service logs for openvswitch
2026-05-29T21:07:39.104947618Z INFO: Collecting host service logs for ovs-configuration
2026-05-29T21:07:39.105335518Z INFO: Collecting host service logs for ovsdb-server
2026-05-29T21:07:39.105671783Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-29T21:07:39.106077813Z INFO: Waiting for worker host service log collection to complete ...
2026-05-29T21:07:39.239504813Z INFO: Waiting for node performance related collection to complete ...
2026-05-29T21:07:39.567364041Z Gathering data for ns/openshift-etcd...
2026-05-29T21:07:39.711037278Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-29T21:07:40.033647353Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-29T21:07:40.033774308Z INFO: on-disk MachineConfig config collection complete.
2026-05-29T21:07:40.082594604Z INFO: Collecting Insights Archives from insights-operator-788bf58df5-96npl
2026-05-29T21:07:40.082594604Z insights-runtime-extractor-2vg2m
2026-05-29T21:07:40.082594604Z insights-runtime-extractor-jswkh
2026-05-29T21:07:40.082594604Z insights-runtime-extractor-vd4gs
2026-05-29T21:07:40.101390284Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-29T21:07:40.173611373Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-29T21:07:40.226597721Z INFO: "metallb-operator" not detected. Skipping.
2026-05-29T21:07:40.260367315Z No resources found in openshift-etcd namespace.
2026-05-29T21:07:40.416265749Z error: the server doesn't have a resource type "performanceprofile"
2026-05-29T21:07:40.417675886Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-29T21:07:40.517104254Z No resources found
2026-05-29T21:07:40.639001855Z INFO: Worker host service log collection to complete.
2026-05-29T21:07:40.641243021Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-29T21:07:40.641243021Z See 'oc rsync -h' for help and examples
2026-05-29T21:07:40.915505832Z ERROR: No running kube-apiserver pods found
2026-05-29T21:07:41.156985514Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-29T21:07:41.298739478Z Wrote inspect data to must-gather.
2026-05-29T21:07:41.302703240Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-29T21:07:41.535879528Z Wrote inspect data to must-gather.
2026-05-29T21:07:41.606509845Z No resources found
2026-05-29T21:07:41.643673996Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-29T21:07:41.711767153Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-29T21:07:41.751558818Z INFO: 'previous-logs' folder not found on ip-10-0-129-129.ec2.internal, skipping...
2026-05-29T21:07:41.822024885Z Wrote inspect data to must-gather.
2026-05-29T21:07:42.098631930Z Wrote inspect data to must-gather.
2026-05-29T21:07:42.126375041Z Wrote inspect data to must-gather.
2026-05-29T21:07:42.126469157Z error: inspection completed with the errors occurred while gathering data:
2026-05-29T21:07:42.126469157Z     namespaces "assisted-installer" not found
2026-05-29T21:07:42.160847794Z Wrote inspect data to must-gather.
2026-05-29T21:07:42.198239033Z Wrote inspect data to must-gather.
2026-05-29T21:07:42.227833310Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-29T21:07:42.291573436Z Gathering data for ns/kuadrant-system...
2026-05-29T21:07:42.346319571Z INFO: Getting rules from prometheus-k8s-0
2026-05-29T21:07:42.390471477Z tar: Removing leading `/' from member names
2026-05-29T21:07:42.398780342Z INFO: 'previous-logs' folder not found on ip-10-0-133-224.ec2.internal, skipping...
2026-05-29T21:07:42.399848010Z INFO: HAProxy config collection complete.
2026-05-29T21:07:42.445019636Z error: the server doesn't have a resource type "machineconfigs"
2026-05-29T21:07:42.675266140Z Wrote inspect data to must-gather.
2026-05-29T21:07:42.716796879Z INFO: 'previous-logs' folder not found on ip-10-0-137-57.ec2.internal, skipping...
2026-05-29T21:07:42.716887402Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-29T21:07:42.716917178Z INFO: Machine Config Daemon termination log collection complete.
2026-05-29T21:07:42.847996023Z Wrote inspect data to must-gather.
2026-05-29T21:07:42.874522205Z INFO: Getting status/config from prometheus-k8s-0
2026-05-29T21:07:43.155332586Z Wrote inspect data to must-gather.
2026-05-29T21:07:43.242807914Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-29T21:07:43.250066931Z Wrote inspect data to must-gather.
2026-05-29T21:07:43.327301610Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-29T21:07:43.523288510Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-29T21:07:43.576345188Z Wrote inspect data to must-gather.
2026-05-29T21:07:43.623118522Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-29T21:07:43.873736899Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-29T21:07:43.924473903Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-29T21:07:43.977289745Z Wrote inspect data to must-gather.
2026-05-29T21:07:44.034937238Z INFO: INTERCONNECT MODE
2026-05-29T21:07:44.035026104Z INFO: Gathering ovn-kubernetes DBs
2026-05-29T21:07:44.144880103Z Gathering data for ns/openshift-monitoring...
2026-05-29T21:07:44.241780055Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-29T21:07:44.247851904Z INFO: Gathering OVN_Northbound from ovnkube-node-5h2kj...
2026-05-29T21:07:44.248086887Z INFO: Gathering OVN_Northbound from ovnkube-node-5r4nz...
2026-05-29T21:07:44.248395926Z INFO: Gathering OVN_Northbound from ovnkube-node-v4ngh...
2026-05-29T21:07:44.248707357Z INFO: Gathering OVN_Southbound from ovnkube-node-5h2kj...
2026-05-29T21:07:44.248994908Z INFO: Gathering OVN_Southbound from ovnkube-node-5r4nz...
2026-05-29T21:07:44.249309385Z INFO: Gathering OVN_Southbound from ovnkube-node-v4ngh...
2026-05-29T21:07:44.569643545Z tar: Removing leading `/' from member names
2026-05-29T21:07:44.573327104Z tar: Removing leading `/' from member names
2026-05-29T21:07:44.644226000Z tar: Removing leading `/' from member names
2026-05-29T21:07:44.653971913Z Wrote inspect data to must-gather.
2026-05-29T21:07:44.671247450Z tar: Removing leading `/' from member names
2026-05-29T21:07:44.675014799Z tar: Removing leading `/' from member names
2026-05-29T21:07:44.690136416Z tar: Removing leading `/' from member names
2026-05-29T21:07:44.886068899Z INFO: Getting status from alertmanager-main-0
2026-05-29T21:07:45.154651562Z Wrote inspect data to must-gather.
2026-05-29T21:07:45.294116654Z INFO: Waiting for network log collection to complete ...
2026-05-29T21:07:45.294278547Z INFO: Waiting for ovnk database copies to complete ...
2026-05-29T21:07:45.294779072Z INFO: Copying ovnk databases complete.
2026-05-29T21:07:45.296721816Z 4.3M	must-gather/network_logs/ovnk_database_store
2026-05-29T21:07:45.299267069Z ovnk_database_store/
2026-05-29T21:07:45.299318968Z ovnk_database_store/ovnkube-node-5r4nz_sbdb
2026-05-29T21:07:45.325334829Z ovnk_database_store/ovnkube-node-5r4nz_nbdb
2026-05-29T21:07:45.337640455Z ovnk_database_store/ovnkube-node-5h2kj_sbdb
2026-05-29T21:07:45.366720645Z ovnk_database_store/ovnkube-node-v4ngh_nbdb
2026-05-29T21:07:45.380304711Z ovnk_database_store/ovnkube-node-v4ngh_sbdb
2026-05-29T21:07:45.408759544Z ovnk_database_store/ovnkube-node-5h2kj_nbdb
2026-05-29T21:07:45.424904459Z INFO: Network log collection complete.
2026-05-29T21:07:45.550574970Z Wrote inspect data to must-gather.
2026-05-29T21:07:45.931481725Z Wrote inspect data to must-gather.
2026-05-29T21:07:46.257481619Z W0529 21:07:46.257433    1421 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-05-29T21:07:46.273627789Z Wrote inspect data to must-gather.
2026-05-29T21:07:46.579700478Z Gathering data for ns/openshift-network-console...
2026-05-29T21:07:46.650592086Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-29T21:07:46.766896569Z Wrote inspect data to must-gather.
2026-05-29T21:07:47.054725288Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-29T21:07:47.057563560Z INFO: Fallback to identify the container image from release info
2026-05-29T21:07:47.178411184Z Gathering data for ns/openshift-console-operator...
2026-05-29T21:07:47.664975371Z Gathering data for ns/openshift-console...
2026-05-29T21:07:47.904477825Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d8381cb85e479760bd6b601e111b39e6c56a632bfa298867f4197ec4421e9f5
2026-05-29T21:07:48.056722367Z daemonset.apps/perf-node-gather-daemonset created
2026-05-29T21:07:48.233612489Z Waiting for performance profile collector pods to become ready: 1
2026-05-29T21:07:48.272971583Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-29T21:07:48.659025449Z Wrote inspect data to must-gather.
2026-05-29T21:07:48.757568541Z Gathering data for ns/openshift-dns-operator...
2026-05-29T21:07:49.209105743Z Gathering data for ns/openshift-dns...
2026-05-29T21:07:49.349363182Z Waiting for performance profile collector pods to become ready: 2
2026-05-29T21:07:50.108725112Z Gathering data for ns/openshift-image-registry...
2026-05-29T21:07:50.452849492Z Waiting for performance profile collector pods to become ready: 3
2026-05-29T21:07:50.698678916Z Gathering data for ns/openshift-ingress-operator...
2026-05-29T21:07:51.193063183Z Gathering data for ns/openshift-ingress...
2026-05-29T21:07:51.557826682Z Waiting for performance profile collector pods to become ready: 4
2026-05-29T21:07:51.656655045Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-29T21:07:52.165248168Z Gathering data for ns/openshift-ingress-canary...
2026-05-29T21:07:52.688235152Z Waiting for performance profile collector pods to become ready: 5
2026-05-29T21:07:52.751408770Z Gathering data for ns/openshift-insights...
2026-05-29T21:07:53.825869698Z Waiting for performance profile collector pods to become ready: 6
2026-05-29T21:07:54.962763426Z Waiting for performance profile collector pods to become ready: 7
2026-05-29T21:07:55.303298417Z Gathering data for ns/opendatahub...
2026-05-29T21:07:56.081999142Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-29T21:07:56.328488130Z Collecting performance related data for node ip-10-0-129-129.ec2.internal
2026-05-29T21:07:56.335429338Z Collecting performance related data for node ip-10-0-137-57.ec2.internal
2026-05-29T21:07:56.343623321Z Collecting performance related data for node ip-10-0-133-224.ec2.internal
2026-05-29T21:07:56.662241694Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-29T21:07:57.343371243Z Gathering data for ns/openshift-lws-operator...
2026-05-29T21:07:59.083347573Z Gathering data for ns/openshift-config...
2026-05-29T21:07:59.503932510Z Gathering data for ns/openshift-config-managed...
2026-05-29T21:08:00.037246900Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-29T21:08:00.388598004Z Gathering data for ns/openshift-kube-apiserver...
2026-05-29T21:08:00.761906692Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-29T21:08:01.170249226Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-29T21:08:01.574259129Z Gathering data for ns/openshift-kube-scheduler...
2026-05-29T21:08:01.669937905Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-29T21:08:01.933243360Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-29T21:08:02.337369362Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-29T21:08:02.789545081Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-29T21:08:03.174397954Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-29T21:08:03.920248404Z Gathering data for ns/openshift-multus...
2026-05-29T21:08:04.649571787Z Collecting kubelet logs for node ip-10-0-129-129.ec2.internal
2026-05-29T21:08:04.651639400Z Collecting kubelet logs for node ip-10-0-137-57.ec2.internal
2026-05-29T21:08:04.653834976Z Collecting kubelet logs for node ip-10-0-133-224.ec2.internal
2026-05-29T21:08:05.064463772Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-29T21:08:05.078251767Z INFO: Node performance data collection complete.
2026-05-29T21:08:05.895317478Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-29T21:08:06.674537292Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-29T21:08:07.662838645Z Gathering data for ns/openshift-host-network...
2026-05-29T21:08:08.058086266Z Gathering data for ns/openshift-network-diagnostics...
2026-05-29T21:08:08.755300495Z Gathering data for ns/openshift-network-node-identity...
2026-05-29T21:08:09.406121992Z Gathering data for ns/openshift-network-operator...
2026-05-29T21:08:09.967055962Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-29T21:08:10.356097512Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-29T21:08:10.909959221Z Gathering data for ns/openshift-apiserver-operator...
2026-05-29T21:08:11.274235305Z Gathering data for ns/openshift-apiserver...
2026-05-29T21:08:11.680794854Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-29T21:08:11.818941370Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-29T21:08:12.169295444Z Gathering data for ns/openshift-controller-manager...
2026-05-29T21:08:12.508780710Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-29T21:08:13.216746006Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-29T21:08:13.626044387Z Gathering data for ns/openshift-service-ca-operator...
2026-05-29T21:08:13.988145325Z Gathering data for ns/openshift-service-ca...
2026-05-29T21:08:14.448843305Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-29T21:08:15.589161876Z Wrote inspect data to must-gather.
2026-05-29T21:08:15.589230227Z error: inspection completed with the errors occurred while gathering data:
2026-05-29T21:08:15.589230227Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-29T21:08:15.728122248Z error: the server doesn't have a resource type "clusters"
2026-05-29T21:08:15.872888115Z Caches written to disk