2026-06-02T18:59:31.054104609Z [disk usage checker] Started
2026-06-02T18:59:31.057786111Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-02T18:59:31.298787322Z Gathering data for ns/openshift-cluster-version...
2026-06-02T18:59:31.482293999Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-02T18:59:31.593821693Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-02T18:59:31.705596225Z Gathering data for ns/default...
2026-06-02T18:59:32.191547706Z Gathering data for ns/openshift...
2026-06-02T18:59:32.676374465Z Gathering data for ns/kube-system...
2026-06-02T18:59:33.420600175Z Gathering data for ns/openshift-etcd...
2026-06-02T18:59:33.533545252Z Waiting on subprocesses to finish execution.
2026-06-02T18:59:33.535539476Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-02T18:59:33.538104254Z INFO: Gathering HAProxy config files
2026-06-02T18:59:33.547273932Z INFO: Collecting host service logs for crio
2026-06-02T18:59:33.547602752Z INFO: Collecting host service logs for kubelet
2026-06-02T18:59:33.547941953Z INFO: Collecting host service logs for rpm-ostreed
2026-06-02T18:59:33.548304134Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-02T18:59:33.548651477Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-02T18:59:33.548984735Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-02T18:59:33.549389331Z INFO: Collecting host service logs for NetworkManager
2026-06-02T18:59:33.549712277Z INFO: Collecting host service logs for openvswitch
2026-06-02T18:59:33.550043721Z INFO: Collecting host service logs for ovs-configuration
2026-06-02T18:59:33.550396000Z INFO: Collecting host service logs for ovsdb-server
2026-06-02T18:59:33.550808213Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-02T18:59:33.551295021Z INFO: Waiting for worker host service log collection to complete ...
2026-06-02T18:59:33.565022530Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-02T18:59:33.582198390Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-02T18:59:33.645255442Z INFO: Waiting for node performance related collection to complete ...
2026-06-02T18:59:33.987630273Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-02T18:59:34.436827084Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-02T18:59:34.436987748Z INFO: on-disk MachineConfig config collection complete.
2026-06-02T18:59:34.464289218Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-02T18:59:34.573091184Z No resources found
2026-06-02T18:59:34.648013734Z No resources found in openshift-etcd namespace.
2026-06-02T18:59:35.200020778Z Wrote inspect data to must-gather.
2026-06-02T18:59:35.200069129Z error: inspection completed with the errors occurred while gathering data:
2026-06-02T18:59:35.200069129Z     namespaces "assisted-installer" not found
2026-06-02T18:59:35.247464380Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-02T18:59:35.275269276Z INFO: Worker host service log collection to complete.
2026-06-02T18:59:35.422512542Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-02T18:59:35.597526102Z No resources found
2026-06-02T18:59:35.653973191Z INFO: "metallb-operator" not detected. Skipping.
2026-06-02T18:59:35.662906370Z INFO: Collecting Insights Archives from insights-operator-788bf58df5-n9nc2
2026-06-02T18:59:35.662956042Z insights-runtime-extractor-27kp4
2026-06-02T18:59:35.662977824Z insights-runtime-extractor-bqz9r
2026-06-02T18:59:35.662997480Z insights-runtime-extractor-l6jck
2026-06-02T18:59:35.789403773Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-02T18:59:35.891139197Z Wrote inspect data to must-gather.
2026-06-02T18:59:35.909386739Z Wrote inspect data to must-gather.
2026-06-02T18:59:36.099851212Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-02T18:59:36.108345000Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-02T18:59:36.113400942Z ERROR: No running kube-apiserver pods found
2026-06-02T18:59:36.186403195Z error: the server doesn't have a resource type "performanceprofile"
2026-06-02T18:59:36.209864395Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-02T18:59:36.243865532Z Wrote inspect data to must-gather.
2026-06-02T18:59:36.247453618Z Wrote inspect data to must-gather.
2026-06-02T18:59:36.290392573Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-02T18:59:36.290392573Z See 'oc rsync -h' for help and examples
2026-06-02T18:59:36.338962244Z Wrote inspect data to must-gather.
2026-06-02T18:59:36.394359098Z INFO: 'previous-logs' folder not found on ip-10-0-128-123.ec2.internal, skipping...
2026-06-02T18:59:36.405256594Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-02T18:59:36.623125910Z Gathering data for ns/kuadrant-system...
2026-06-02T18:59:36.698362051Z tar: Removing leading `/' from member names
2026-06-02T18:59:36.704861494Z INFO: HAProxy config collection complete.
2026-06-02T18:59:36.740790876Z Wrote inspect data to must-gather.
2026-06-02T18:59:36.781723423Z INFO: 'previous-logs' folder not found on ip-10-0-130-211.ec2.internal, skipping...
2026-06-02T18:59:36.786315751Z INFO: Getting rules from prometheus-k8s-0
2026-06-02T18:59:36.886553070Z Wrote inspect data to must-gather.
2026-06-02T18:59:36.965111323Z error: the server doesn't have a resource type "machineconfigs"
2026-06-02T18:59:37.125196044Z INFO: 'previous-logs' folder not found on ip-10-0-140-185.ec2.internal, skipping...
2026-06-02T18:59:37.125306593Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-02T18:59:37.125362687Z INFO: Machine Config Daemon termination log collection complete.
2026-06-02T18:59:37.314886791Z INFO: Getting status/config from prometheus-k8s-0
2026-06-02T18:59:37.393169194Z Wrote inspect data to must-gather.
2026-06-02T18:59:37.403157813Z Wrote inspect data to must-gather.
2026-06-02T18:59:37.608445353Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-02T18:59:37.835963156Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-02T18:59:37.869194165Z Wrote inspect data to must-gather.
2026-06-02T18:59:37.874290850Z Wrote inspect data to must-gather.
2026-06-02T18:59:38.097440958Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-02T18:59:38.131146401Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-02T18:59:38.238115852Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-02T18:59:38.301957048Z INFO: INTERCONNECT MODE
2026-06-02T18:59:38.302050980Z INFO: Gathering ovn-kubernetes DBs
2026-06-02T18:59:38.471681337Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-02T18:59:38.529137611Z Gathering data for ns/openshift-monitoring...
2026-06-02T18:59:38.547445575Z INFO: Gathering OVN_Northbound from ovnkube-node-86wdb...
2026-06-02T18:59:38.547697306Z INFO: Gathering OVN_Northbound from ovnkube-node-bzl2v...
2026-06-02T18:59:38.547926458Z INFO: Gathering OVN_Northbound from ovnkube-node-sftvp...
2026-06-02T18:59:38.548254325Z INFO: Gathering OVN_Southbound from ovnkube-node-86wdb...
2026-06-02T18:59:38.548692981Z INFO: Gathering OVN_Southbound from ovnkube-node-bzl2v...
2026-06-02T18:59:38.548980139Z INFO: Gathering OVN_Southbound from ovnkube-node-sftvp...
2026-06-02T18:59:38.724164591Z Wrote inspect data to must-gather.
2026-06-02T18:59:38.848667576Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-02T18:59:38.910834709Z tar: Removing leading `/' from member names
2026-06-02T18:59:38.951322701Z tar: Removing leading `/' from member names
2026-06-02T18:59:38.955941090Z tar: Removing leading `/' from member names
2026-06-02T18:59:38.960100451Z tar: Removing leading `/' from member names
2026-06-02T18:59:38.984500765Z tar: Removing leading `/' from member names
2026-06-02T18:59:39.025423133Z tar: Removing leading `/' from member names
2026-06-02T18:59:39.301967741Z Wrote inspect data to must-gather.
2026-06-02T18:59:39.372426946Z INFO: Getting status from alertmanager-main-0
2026-06-02T18:59:39.651714469Z INFO: Waiting for network log collection to complete ...
2026-06-02T18:59:39.651768464Z INFO: Waiting for ovnk database copies to complete ...
2026-06-02T18:59:39.652207645Z INFO: Copying ovnk databases complete.
2026-06-02T18:59:39.654421120Z 4.2M	must-gather/network_logs/ovnk_database_store
2026-06-02T18:59:39.657352630Z ovnk_database_store/
2026-06-02T18:59:39.657352630Z ovnk_database_store/ovnkube-node-sftvp_nbdb
2026-06-02T18:59:39.668840812Z ovnk_database_store/ovnkube-node-86wdb_nbdb
2026-06-02T18:59:39.679046637Z ovnk_database_store/ovnkube-node-bzl2v_nbdb
2026-06-02T18:59:39.688878364Z ovnk_database_store/ovnkube-node-bzl2v_sbdb
2026-06-02T18:59:39.712966856Z ovnk_database_store/ovnkube-node-sftvp_sbdb
2026-06-02T18:59:39.738523255Z ovnk_database_store/ovnkube-node-86wdb_sbdb
2026-06-02T18:59:39.762374203Z INFO: Network log collection complete.
2026-06-02T18:59:39.792750763Z Wrote inspect data to must-gather.
2026-06-02T18:59:40.190163617Z Wrote inspect data to must-gather.
2026-06-02T18:59:40.561227651Z Wrote inspect data to must-gather.
2026-06-02T18:59:41.007619949Z Wrote inspect data to must-gather.
2026-06-02T18:59:41.053201758Z Gathering data for ns/openshift-network-console...
2026-06-02T18:59:41.127049971Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-02T18:59:41.466455772Z Wrote inspect data to must-gather.
2026-06-02T18:59:41.628498369Z Gathering data for ns/openshift-console-operator...
2026-06-02T18:59:41.792359466Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-02T18:59:41.812037953Z INFO: Fallback to identify the container image from release info
2026-06-02T18:59:42.106148413Z Gathering data for ns/openshift-console...
2026-06-02T18:59:42.397431865Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d8381cb85e479760bd6b601e111b39e6c56a632bfa298867f4197ec4421e9f5
2026-06-02T18:59:42.532752186Z daemonset.apps/perf-node-gather-daemonset created
2026-06-02T18:59:42.711205052Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-02T18:59:42.711308364Z Waiting for performance profile collector pods to become ready: 1
2026-06-02T18:59:43.208512906Z Gathering data for ns/openshift-dns-operator...
2026-06-02T18:59:43.299577154Z Wrote inspect data to must-gather.
2026-06-02T18:59:43.541356469Z Gathering data for ns/openshift-dns...
2026-06-02T18:59:43.852682000Z Waiting for performance profile collector pods to become ready: 2
2026-06-02T18:59:44.208448931Z Gathering data for ns/openshift-image-registry...
2026-06-02T18:59:44.746234172Z Gathering data for ns/openshift-ingress-operator...
2026-06-02T18:59:44.975256348Z Waiting for performance profile collector pods to become ready: 3
2026-06-02T18:59:45.119235719Z Gathering data for ns/openshift-ingress...
2026-06-02T18:59:45.815213544Z Gathering data for ns/openshift-ingress-canary...
2026-06-02T18:59:46.091614085Z Waiting for performance profile collector pods to become ready: 4
2026-06-02T18:59:46.137934165Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-02T18:59:46.292732935Z Gathering data for ns/openshift-insights...
2026-06-02T18:59:47.210585388Z Waiting for performance profile collector pods to become ready: 5
2026-06-02T18:59:48.321598246Z Waiting for performance profile collector pods to become ready: 6
2026-06-02T18:59:48.497228158Z Gathering data for ns/opendatahub...
2026-06-02T18:59:49.445261030Z Waiting for performance profile collector pods to become ready: 7
2026-06-02T18:59:50.250449883Z Gathering data for ns/openshift-lws-operator...
2026-06-02T18:59:50.570961072Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-02T18:59:50.821231876Z Collecting performance related data for node ip-10-0-140-185.ec2.internal
2026-06-02T18:59:50.828067411Z Collecting performance related data for node ip-10-0-130-211.ec2.internal
2026-06-02T18:59:50.836444300Z Collecting performance related data for node ip-10-0-128-123.ec2.internal
2026-06-02T18:59:51.143601035Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-02T18:59:52.120718374Z Gathering data for ns/openshift-config...
2026-06-02T18:59:52.479098733Z Gathering data for ns/openshift-config-managed...
2026-06-02T18:59:52.952502246Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-02T18:59:53.302515301Z Gathering data for ns/openshift-kube-apiserver...
2026-06-02T18:59:53.673039653Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-02T18:59:54.070988223Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-02T18:59:54.454417644Z Gathering data for ns/openshift-kube-scheduler...
2026-06-02T18:59:54.782332627Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-02T18:59:55.122472164Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-02T18:59:55.515439179Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-02T18:59:55.923006754Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-02T18:59:56.150443465Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-02T18:59:56.540143333Z Gathering data for ns/openshift-multus...
2026-06-02T18:59:58.003867198Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-02T18:59:59.643099082Z Collecting kubelet logs for node ip-10-0-140-185.ec2.internal
2026-06-02T18:59:59.645042221Z Collecting kubelet logs for node ip-10-0-130-211.ec2.internal
2026-06-02T18:59:59.646898110Z Collecting kubelet logs for node ip-10-0-128-123.ec2.internal
2026-06-02T18:59:59.695775662Z Gathering data for ns/openshift-host-network...
2026-06-02T19:00:00.037246367Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-06-02T19:00:00.042811071Z INFO: Node performance data collection complete.
2026-06-02T19:00:00.148955947Z Gathering data for ns/openshift-network-diagnostics...
2026-06-02T19:00:00.753127768Z Gathering data for ns/openshift-network-node-identity...
2026-06-02T19:00:01.156112558Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-02T19:00:01.217570742Z Gathering data for ns/openshift-network-operator...
2026-06-02T19:00:01.630417801Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-02T19:00:01.926763914Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-02T19:00:02.417938000Z Gathering data for ns/openshift-apiserver-operator...
2026-06-02T19:00:02.764489474Z Gathering data for ns/openshift-apiserver...
2026-06-02T19:00:03.203443407Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-02T19:00:03.571855651Z Gathering data for ns/openshift-controller-manager...
2026-06-02T19:00:03.901821490Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-02T19:00:04.594033387Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-02T19:00:04.913318552Z Gathering data for ns/openshift-service-ca-operator...
2026-06-02T19:00:05.257018758Z Gathering data for ns/openshift-service-ca...
2026-06-02T19:00:05.663293911Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-02T19:00:06.161372025Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-02T19:00:06.761250097Z Wrote inspect data to must-gather.
2026-06-02T19:00:06.761293943Z error: inspection completed with the errors occurred while gathering data:
2026-06-02T19:00:06.761293943Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-02T19:00:06.894080331Z error: the server doesn't have a resource type "clusters"
2026-06-02T19:00:07.078881119Z Caches written to disk