2026-05-25T14:49:06.867888377Z [disk usage checker] Started
2026-05-25T14:49:06.871373260Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-25T14:49:07.090593365Z Gathering data for ns/openshift-cluster-version...
2026-05-25T14:49:07.250042995Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-25T14:49:07.359096377Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-25T14:49:07.462548774Z Gathering data for ns/default...
2026-05-25T14:49:07.847645405Z Gathering data for ns/openshift...
2026-05-25T14:49:08.356652483Z Gathering data for ns/kube-system...
2026-05-25T14:49:09.102992688Z Gathering data for ns/openshift-etcd...
2026-05-25T14:49:09.336452364Z Waiting on subprocesses to finish execution.
2026-05-25T14:49:09.352089026Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-25T14:49:09.356048548Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-25T14:49:09.369233947Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-25T14:49:09.381204465Z INFO: Gathering HAProxy config files
2026-05-25T14:49:09.386918452Z INFO: Waiting for node performance related collection to complete ...
2026-05-25T14:49:09.405615885Z INFO: Collecting host service logs for crio
2026-05-25T14:49:09.406088963Z INFO: Collecting host service logs for kubelet
2026-05-25T14:49:09.406471867Z INFO: Collecting host service logs for rpm-ostreed
2026-05-25T14:49:09.406930297Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-25T14:49:09.407303925Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-25T14:49:09.407656447Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-25T14:49:09.408020220Z INFO: Collecting host service logs for NetworkManager
2026-05-25T14:49:09.408382945Z INFO: Collecting host service logs for openvswitch
2026-05-25T14:49:09.408759263Z INFO: Collecting host service logs for ovs-configuration
2026-05-25T14:49:09.409124020Z INFO: Collecting host service logs for ovsdb-server
2026-05-25T14:49:09.409521193Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-25T14:49:09.414244605Z INFO: Waiting for worker host service log collection to complete ...
2026-05-25T14:49:09.802175018Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-25T14:49:09.864486745Z Wrote inspect data to must-gather.
2026-05-25T14:49:09.865512676Z error: inspection completed with the errors occurred while gathering data:
2026-05-25T14:49:09.865512676Z     namespaces "assisted-installer" not found
2026-05-25T14:49:09.965165348Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-25T14:49:10.038114245Z No resources found in openshift-etcd namespace.
2026-05-25T14:49:10.051790331Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-25T14:49:10.069243296Z No resources found
2026-05-25T14:49:10.238424509Z INFO: "metallb-operator" not detected. Skipping.
2026-05-25T14:49:10.447905050Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-25T14:49:10.559004545Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-ns884
2026-05-25T14:49:10.559004545Z insights-runtime-extractor-9wljr
2026-05-25T14:49:10.559004545Z insights-runtime-extractor-d8shq
2026-05-25T14:49:10.559004545Z insights-runtime-extractor-v4m7f
2026-05-25T14:49:10.600562078Z error: the server doesn't have a resource type "performanceprofile"
2026-05-25T14:49:10.852492820Z ERROR: No running kube-apiserver pods found
2026-05-25T14:49:11.125443411Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-25T14:49:11.125528245Z INFO: on-disk MachineConfig config collection complete.
2026-05-25T14:49:11.145681735Z Wrote inspect data to must-gather.
2026-05-25T14:49:11.428811996Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-25T14:49:11.437624007Z No resources found
2026-05-25T14:49:11.639689272Z INFO: 'previous-logs' folder not found on ip-10-0-131-223.ec2.internal, skipping...
2026-05-25T14:49:11.644631911Z INFO: Worker host service log collection to complete.
2026-05-25T14:49:11.651044021Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-25T14:49:11.721856832Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-25T14:49:11.729798462Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-25T14:49:11.729798462Z See 'oc rsync -h' for help and examples
2026-05-25T14:49:11.837044927Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-25T14:49:11.870039900Z Wrote inspect data to must-gather.
2026-05-25T14:49:11.895122342Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-25T14:49:11.912189690Z Wrote inspect data to must-gather.
2026-05-25T14:49:11.952505123Z Wrote inspect data to must-gather.
2026-05-25T14:49:12.062364088Z Wrote inspect data to must-gather.
2026-05-25T14:49:12.149678257Z Wrote inspect data to must-gather.
2026-05-25T14:49:12.244894007Z INFO: 'previous-logs' folder not found on ip-10-0-134-152.ec2.internal, skipping...
2026-05-25T14:49:12.260792196Z tar: Removing leading `/' from member names
2026-05-25T14:49:12.267387435Z INFO: HAProxy config collection complete.
2026-05-25T14:49:12.353200228Z Gathering data for ns/kuadrant-system...
2026-05-25T14:49:12.416687734Z error: the server doesn't have a resource type "machineconfigs"
2026-05-25T14:49:12.471522579Z Wrote inspect data to must-gather.
2026-05-25T14:49:12.508647956Z INFO: Getting rules from prometheus-k8s-0
2026-05-25T14:49:12.610947468Z INFO: 'previous-logs' folder not found on ip-10-0-143-67.ec2.internal, skipping...
2026-05-25T14:49:12.610980632Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-25T14:49:12.611019398Z INFO: Machine Config Daemon termination log collection complete.
2026-05-25T14:49:12.704478092Z Wrote inspect data to must-gather.
2026-05-25T14:49:12.819893519Z Wrote inspect data to must-gather.
2026-05-25T14:49:12.992839733Z INFO: Getting status/config from prometheus-k8s-0
2026-05-25T14:49:13.001170484Z Wrote inspect data to must-gather.
2026-05-25T14:49:13.203975415Z Wrote inspect data to must-gather.
2026-05-25T14:49:13.247880890Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-25T14:49:13.453100974Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-25T14:49:13.500068771Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-25T14:49:13.560338930Z Wrote inspect data to must-gather.
2026-05-25T14:49:13.614835260Z INFO: INTERCONNECT MODE
2026-05-25T14:49:13.614925121Z INFO: Gathering ovn-kubernetes DBs
2026-05-25T14:49:13.766518496Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-25T14:49:13.862703636Z INFO: Gathering OVN_Northbound from ovnkube-node-kbjzs...
2026-05-25T14:49:13.863049056Z INFO: Gathering OVN_Northbound from ovnkube-node-rzslr...
2026-05-25T14:49:13.863313439Z INFO: Gathering OVN_Northbound from ovnkube-node-trqqx...
2026-05-25T14:49:13.863613783Z INFO: Gathering OVN_Southbound from ovnkube-node-kbjzs...
2026-05-25T14:49:13.863892976Z INFO: Gathering OVN_Southbound from ovnkube-node-rzslr...
2026-05-25T14:49:13.864150094Z INFO: Gathering OVN_Southbound from ovnkube-node-trqqx...
2026-05-25T14:49:13.882179816Z Wrote inspect data to must-gather.
2026-05-25T14:49:14.157048766Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-25T14:49:14.221185939Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-25T14:49:14.291018052Z tar: Removing leading `/' from member names
2026-05-25T14:49:14.377298441Z tar: Removing leading `/' from member names
2026-05-25T14:49:14.391533939Z tar: Removing leading `/' from member names
2026-05-25T14:49:14.424237041Z tar: Removing leading `/' from member names
2026-05-25T14:49:14.436880965Z tar: Removing leading `/' from member names
2026-05-25T14:49:14.457012115Z tar: Removing leading `/' from member names
2026-05-25T14:49:14.572613298Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-25T14:49:14.576759525Z Gathering data for ns/openshift-monitoring...
2026-05-25T14:49:14.664689050Z Wrote inspect data to must-gather.
2026-05-25T14:49:15.052471799Z INFO: Getting status from alertmanager-main-0
2026-05-25T14:49:15.072966300Z Wrote inspect data to must-gather.
2026-05-25T14:49:15.120229881Z INFO: Waiting for network log collection to complete ...
2026-05-25T14:49:15.120261481Z INFO: Waiting for ovnk database copies to complete ...
2026-05-25T14:49:15.120638219Z INFO: Copying ovnk databases complete.
2026-05-25T14:49:15.122475144Z 4.2M	must-gather/network_logs/ovnk_database_store
2026-05-25T14:49:15.125419311Z ovnk_database_store/
2026-05-25T14:49:15.125419311Z ovnk_database_store/ovnkube-node-kbjzs_nbdb
2026-05-25T14:49:15.137749531Z ovnk_database_store/ovnkube-node-kbjzs_sbdb
2026-05-25T14:49:15.164042565Z ovnk_database_store/ovnkube-node-trqqx_nbdb
2026-05-25T14:49:15.175072403Z ovnk_database_store/ovnkube-node-rzslr_nbdb
2026-05-25T14:49:15.190862735Z ovnk_database_store/ovnkube-node-trqqx_sbdb
2026-05-25T14:49:15.218659726Z ovnk_database_store/ovnkube-node-rzslr_sbdb
2026-05-25T14:49:15.265256309Z INFO: Network log collection complete.
2026-05-25T14:49:15.369945046Z Wrote inspect data to must-gather.
2026-05-25T14:49:15.673496959Z Wrote inspect data to must-gather.
2026-05-25T14:49:16.023388922Z Wrote inspect data to must-gather.
2026-05-25T14:49:16.340049552Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-25T14:49:16.343131926Z INFO: Fallback to identify the container image from release info
2026-05-25T14:49:16.806902917Z Gathering data for ns/openshift-network-console...
2026-05-25T14:49:16.901780731Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-25T14:49:16.943112695Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda
2026-05-25T14:49:17.079252462Z daemonset.apps/perf-node-gather-daemonset created
2026-05-25T14:49:17.238300467Z Waiting for performance profile collector pods to become ready: 1
2026-05-25T14:49:17.391946071Z Gathering data for ns/openshift-console-operator...
2026-05-25T14:49:17.915067262Z Gathering data for ns/openshift-console...
2026-05-25T14:49:18.405606214Z Waiting for performance profile collector pods to become ready: 2
2026-05-25T14:49:18.520824830Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-25T14:49:18.958263144Z Gathering data for ns/openshift-dns-operator...
2026-05-25T14:49:19.191318184Z Wrote inspect data to must-gather.
2026-05-25T14:49:19.306084738Z Gathering data for ns/openshift-dns...
2026-05-25T14:49:19.513802109Z Waiting for performance profile collector pods to become ready: 3
2026-05-25T14:49:19.934021245Z Gathering data for ns/openshift-image-registry...
2026-05-25T14:49:20.447878857Z Gathering data for ns/openshift-ingress-operator...
2026-05-25T14:49:20.621112529Z Waiting for performance profile collector pods to become ready: 4
2026-05-25T14:49:20.858559495Z Gathering data for ns/openshift-ingress...
2026-05-25T14:49:21.678220928Z Gathering data for ns/openshift-ingress-canary...
2026-05-25T14:49:21.738249230Z Waiting for performance profile collector pods to become ready: 5
2026-05-25T14:49:21.906714395Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-25T14:49:22.147223366Z Gathering data for ns/openshift-insights...
2026-05-25T14:49:22.860525708Z Waiting for performance profile collector pods to become ready: 6
2026-05-25T14:49:23.972880282Z Waiting for performance profile collector pods to become ready: 7
2026-05-25T14:49:24.131646795Z Gathering data for ns/opendatahub...
2026-05-25T14:49:25.093070935Z Waiting for performance profile collector pods to become ready: 8
2026-05-25T14:49:25.567925242Z Gathering data for ns/openshift-lws-operator...
2026-05-25T14:49:26.204157863Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-25T14:49:26.452680257Z Collecting performance related data for node ip-10-0-143-67.ec2.internal
2026-05-25T14:49:26.459553032Z Collecting performance related data for node ip-10-0-131-223.ec2.internal
2026-05-25T14:49:26.467005048Z Collecting performance related data for node ip-10-0-134-152.ec2.internal
2026-05-25T14:49:26.880825400Z Gathering data for ns/openshift-config...
2026-05-25T14:49:26.913309033Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-25T14:49:27.248190138Z Gathering data for ns/openshift-config-managed...
2026-05-25T14:49:27.865871387Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-25T14:49:28.222383960Z Gathering data for ns/openshift-kube-apiserver...
2026-05-25T14:49:28.542003591Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-25T14:49:28.932383304Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-25T14:49:29.364256834Z Gathering data for ns/openshift-kube-scheduler...
2026-05-25T14:49:29.708658536Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-25T14:49:30.114600206Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-25T14:49:30.477164723Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-25T14:49:30.852060419Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-25T14:49:31.533842110Z Gathering data for ns/openshift-multus...
2026-05-25T14:49:31.919045722Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-25T14:49:33.033940667Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-25T14:49:34.701628489Z Gathering data for ns/openshift-host-network...
2026-05-25T14:49:34.964812151Z Collecting kubelet logs for node ip-10-0-143-67.ec2.internal
2026-05-25T14:49:34.968841772Z Collecting kubelet logs for node ip-10-0-131-223.ec2.internal
2026-05-25T14:49:34.971175289Z Collecting kubelet logs for node ip-10-0-134-152.ec2.internal
2026-05-25T14:49:35.230260019Z Gathering data for ns/openshift-network-diagnostics...
2026-05-25T14:49:35.331303913Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-25T14:49:35.336812562Z INFO: Node performance data collection complete.
2026-05-25T14:49:35.856073983Z Gathering data for ns/openshift-network-node-identity...
2026-05-25T14:49:36.315119493Z Gathering data for ns/openshift-network-operator...
2026-05-25T14:49:36.728908872Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-25T14:49:36.925538440Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-25T14:49:37.052795043Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-25T14:49:37.542756448Z Gathering data for ns/openshift-apiserver-operator...
2026-05-25T14:49:37.869480646Z Gathering data for ns/openshift-apiserver...
2026-05-25T14:49:38.326801069Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-25T14:49:38.657871885Z Gathering data for ns/openshift-controller-manager...
2026-05-25T14:49:38.986410069Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-25T14:49:39.666344462Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-25T14:49:39.992081358Z Gathering data for ns/openshift-service-ca-operator...
2026-05-25T14:49:40.335882773Z Gathering data for ns/openshift-service-ca...
2026-05-25T14:49:40.759093656Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-25T14:49:41.788841403Z Wrote inspect data to must-gather.
2026-05-25T14:49:41.788884462Z error: inspection completed with the errors occurred while gathering data:
2026-05-25T14:49:41.788884462Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-25T14:49:41.919220696Z error: the server doesn't have a resource type "clusters"
2026-05-25T14:49:41.930294832Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-25T14:49:42.135488124Z Caches written to disk