2026-05-21T21:55:11.354029220Z [disk usage checker] Started
2026-05-21T21:55:11.357260206Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-21T21:55:11.592361625Z Gathering data for ns/openshift-cluster-version...
2026-05-21T21:55:11.745846985Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-21T21:55:11.838822579Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-21T21:55:11.939935655Z Gathering data for ns/default...
2026-05-21T21:55:12.322058924Z Gathering data for ns/openshift...
2026-05-21T21:55:12.760762483Z Gathering data for ns/kube-system...
2026-05-21T21:55:13.625725211Z Gathering data for ns/openshift-etcd...
2026-05-21T21:55:13.824603574Z Waiting on subprocesses to finish execution.
2026-05-21T21:55:13.828695776Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-21T21:55:13.836067976Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-21T21:55:13.840308451Z INFO: Collecting host service logs for crio
2026-05-21T21:55:13.840686970Z INFO: Collecting host service logs for kubelet
2026-05-21T21:55:13.841071451Z INFO: Collecting host service logs for rpm-ostreed
2026-05-21T21:55:13.841399527Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-21T21:55:13.842071557Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-21T21:55:13.842406768Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-21T21:55:13.842741929Z INFO: Collecting host service logs for NetworkManager
2026-05-21T21:55:13.843117877Z INFO: Collecting host service logs for openvswitch
2026-05-21T21:55:13.845734243Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-21T21:55:13.846277307Z INFO: Gathering HAProxy config files
2026-05-21T21:55:13.846607727Z INFO: Collecting host service logs for ovs-configuration
2026-05-21T21:55:13.848432693Z INFO: Collecting host service logs for ovsdb-server
2026-05-21T21:55:13.848844817Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-21T21:55:13.849249949Z INFO: Waiting for worker host service log collection to complete ...
2026-05-21T21:55:13.968485086Z INFO: Waiting for node performance related collection to complete ...
2026-05-21T21:55:14.425007574Z Wrote inspect data to must-gather.
2026-05-21T21:55:14.425052508Z error: inspection completed with the errors occurred while gathering data:
2026-05-21T21:55:14.425052508Z     namespaces "assisted-installer" not found
2026-05-21T21:55:14.430946694Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-21T21:55:14.431068476Z INFO: on-disk MachineConfig config collection complete.
2026-05-21T21:55:14.574476475Z Wrote inspect data to must-gather.
2026-05-21T21:55:14.770530081Z error: the server doesn't have a resource type "performanceprofile"
2026-05-21T21:55:14.875987888Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-21T21:55:15.029235787Z No resources found in openshift-etcd namespace.
2026-05-21T21:55:15.072693650Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-21T21:55:15.170567069Z INFO: "metallb-operator" not detected. Skipping.
2026-05-21T21:55:15.171233464Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-tns2g
2026-05-21T21:55:15.171359282Z insights-runtime-extractor-9xfxl
2026-05-21T21:55:15.171380876Z insights-runtime-extractor-c4996
2026-05-21T21:55:15.171412953Z insights-runtime-extractor-s7z78
2026-05-21T21:55:15.204406253Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-21T21:55:15.268687870Z INFO: Worker host service log collection to complete.
2026-05-21T21:55:15.355507078Z No resources found
2026-05-21T21:55:15.384780388Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-21T21:55:15.541939584Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-21T21:55:15.541939584Z See 'oc rsync -h' for help and examples
2026-05-21T21:55:15.571804747Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-21T21:55:16.143792517Z ERROR: No running kube-apiserver pods found
2026-05-21T21:55:16.186628865Z Wrote inspect data to must-gather.
2026-05-21T21:55:16.195139764Z Gathering data for ns/kuadrant-system...
2026-05-21T21:55:16.204605508Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-21T21:55:16.350485610Z No resources found
2026-05-21T21:55:16.373649172Z Wrote inspect data to must-gather.
2026-05-21T21:55:16.377750756Z Wrote inspect data to must-gather.
2026-05-21T21:55:16.406521046Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-21T21:55:16.449111718Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-21T21:55:16.510760397Z INFO: 'previous-logs' folder not found on ip-10-0-129-28.ec2.internal, skipping...
2026-05-21T21:55:16.548001675Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-21T21:55:16.593550988Z Wrote inspect data to must-gather.
2026-05-21T21:55:16.693775763Z tar: Removing leading `/' from member names
2026-05-21T21:55:16.700037251Z INFO: HAProxy config collection complete.
2026-05-21T21:55:16.740609783Z Wrote inspect data to must-gather.
2026-05-21T21:55:16.797000251Z error: the server doesn't have a resource type "machineconfigs"
2026-05-21T21:55:16.991742581Z Wrote inspect data to must-gather.
2026-05-21T21:55:16.996621249Z INFO: 'previous-logs' folder not found on ip-10-0-133-99.ec2.internal, skipping...
2026-05-21T21:55:17.134223821Z Wrote inspect data to must-gather.
2026-05-21T21:55:17.208335784Z INFO: Getting rules from prometheus-k8s-0
2026-05-21T21:55:17.361546465Z INFO: 'previous-logs' folder not found on ip-10-0-135-166.ec2.internal, skipping...
2026-05-21T21:55:17.361674732Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-21T21:55:17.361731882Z INFO: Machine Config Daemon termination log collection complete.
2026-05-21T21:55:17.379782807Z Wrote inspect data to must-gather.
2026-05-21T21:55:17.477199955Z Wrote inspect data to must-gather.
2026-05-21T21:55:17.608781638Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-21T21:55:17.666979776Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-21T21:55:17.748832354Z Wrote inspect data to must-gather.
2026-05-21T21:55:17.753210397Z INFO: Getting status/config from prometheus-k8s-0
2026-05-21T21:55:18.005872702Z Wrote inspect data to must-gather.
2026-05-21T21:55:18.144950948Z INFO: INTERCONNECT MODE
2026-05-21T21:55:18.145058059Z INFO: Gathering ovn-kubernetes DBs
2026-05-21T21:55:18.230683457Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-21T21:55:18.281735399Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-21T21:55:18.387852652Z INFO: Gathering OVN_Northbound from ovnkube-node-cs2zq...
2026-05-21T21:55:18.392362896Z INFO: Gathering OVN_Northbound from ovnkube-node-fdmjr...
2026-05-21T21:55:18.392674388Z INFO: Gathering OVN_Northbound from ovnkube-node-x2klb...
2026-05-21T21:55:18.392979564Z INFO: Gathering OVN_Southbound from ovnkube-node-cs2zq...
2026-05-21T21:55:18.393249239Z INFO: Gathering OVN_Southbound from ovnkube-node-fdmjr...
2026-05-21T21:55:18.393500557Z INFO: Gathering OVN_Southbound from ovnkube-node-x2klb...
2026-05-21T21:55:18.432640971Z Wrote inspect data to must-gather.
2026-05-21T21:55:18.638878478Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-21T21:55:18.644872047Z Gathering data for ns/openshift-monitoring...
2026-05-21T21:55:18.799510167Z tar: Removing leading `/' from member names
2026-05-21T21:55:18.822517284Z tar: Removing leading `/' from member names
2026-05-21T21:55:18.883729419Z tar: Removing leading `/' from member names
2026-05-21T21:55:18.889493808Z tar: Removing leading `/' from member names
2026-05-21T21:55:18.948725342Z tar: Removing leading `/' from member names
2026-05-21T21:55:18.949473796Z tar: Removing leading `/' from member names
2026-05-21T21:55:18.990057166Z Wrote inspect data to must-gather.
2026-05-21T21:55:19.020602696Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-21T21:55:19.351150670Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-21T21:55:19.414398412Z Wrote inspect data to must-gather.
2026-05-21T21:55:19.572966621Z INFO: Waiting for network log collection to complete ...
2026-05-21T21:55:19.573000417Z INFO: Waiting for ovnk database copies to complete ...
2026-05-21T21:55:19.573369384Z INFO: Copying ovnk databases complete.
2026-05-21T21:55:19.574918737Z 4.2M	must-gather/network_logs/ovnk_database_store
2026-05-21T21:55:19.577031514Z ovnk_database_store/
2026-05-21T21:55:19.577031514Z ovnk_database_store/ovnkube-node-x2klb_nbdb
2026-05-21T21:55:19.587749097Z ovnk_database_store/ovnkube-node-cs2zq_nbdb
2026-05-21T21:55:19.597432672Z ovnk_database_store/ovnkube-node-cs2zq_sbdb
2026-05-21T21:55:19.626107072Z ovnk_database_store/ovnkube-node-x2klb_sbdb
2026-05-21T21:55:19.667729673Z ovnk_database_store/ovnkube-node-fdmjr_nbdb
2026-05-21T21:55:19.678486928Z ovnk_database_store/ovnkube-node-fdmjr_sbdb
2026-05-21T21:55:19.702876569Z INFO: Network log collection complete.
2026-05-21T21:55:19.777508315Z INFO: Getting status from alertmanager-main-0
2026-05-21T21:55:19.781161447Z Wrote inspect data to must-gather.
2026-05-21T21:55:20.153121769Z Wrote inspect data to must-gather.
2026-05-21T21:55:20.429199898Z Wrote inspect data to must-gather.
2026-05-21T21:55:20.751797573Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-21T21:55:20.754171046Z INFO: Fallback to identify the container image from release info
2026-05-21T21:55:21.057343680Z Gathering data for ns/openshift-network-console...
2026-05-21T21:55:21.378560424Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda
2026-05-21T21:55:21.449151644Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-21T21:55:21.531640427Z daemonset.apps/perf-node-gather-daemonset created
2026-05-21T21:55:21.640187674Z Gathering data for ns/openshift-console-operator...
2026-05-21T21:55:21.678723911Z Waiting for performance profile collector pods to become ready: 1
2026-05-21T21:55:22.121039087Z Gathering data for ns/openshift-console...
2026-05-21T21:55:22.655048256Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-21T21:55:22.836762632Z Waiting for performance profile collector pods to become ready: 2
2026-05-21T21:55:23.125273710Z Gathering data for ns/openshift-dns-operator...
2026-05-21T21:55:23.510689447Z Gathering data for ns/openshift-dns...
2026-05-21T21:55:23.551724569Z Wrote inspect data to must-gather.
2026-05-21T21:55:23.953536976Z Waiting for performance profile collector pods to become ready: 3
2026-05-21T21:55:24.147781573Z Gathering data for ns/openshift-image-registry...
2026-05-21T21:55:24.656107929Z Gathering data for ns/openshift-ingress-operator...
2026-05-21T21:55:25.062997081Z Gathering data for ns/openshift-ingress...
2026-05-21T21:55:25.078369123Z Waiting for performance profile collector pods to become ready: 4
2026-05-21T21:55:25.871101016Z Gathering data for ns/openshift-ingress-canary...
2026-05-21T21:55:26.217162966Z Waiting for performance profile collector pods to become ready: 5
2026-05-21T21:55:26.351069089Z Gathering data for ns/openshift-insights...
2026-05-21T21:55:26.453832389Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-21T21:55:27.329924807Z Waiting for performance profile collector pods to become ready: 6
2026-05-21T21:55:28.351940678Z Gathering data for ns/opendatahub...
2026-05-21T21:55:28.437994369Z Waiting for performance profile collector pods to become ready: 7
2026-05-21T21:55:29.561734807Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-21T21:55:29.810685592Z Collecting performance related data for node ip-10-0-129-28.ec2.internal
2026-05-21T21:55:29.817575198Z Collecting performance related data for node ip-10-0-133-99.ec2.internal
2026-05-21T21:55:29.825765340Z Collecting performance related data for node ip-10-0-135-166.ec2.internal
2026-05-21T21:55:29.938134854Z Gathering data for ns/openshift-lws-operator...
2026-05-21T21:55:31.367907975Z Gathering data for ns/openshift-config...
2026-05-21T21:55:31.461204279Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-21T21:55:31.728255775Z Gathering data for ns/openshift-config-managed...
2026-05-21T21:55:32.263186554Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-21T21:55:32.626004713Z Gathering data for ns/openshift-kube-apiserver...
2026-05-21T21:55:32.966570124Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-21T21:55:33.343125322Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-21T21:55:33.687900658Z Gathering data for ns/openshift-kube-scheduler...
2026-05-21T21:55:34.000610350Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-21T21:55:34.369505120Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-21T21:55:34.744411487Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-21T21:55:35.117188785Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-21T21:55:35.730675875Z Gathering data for ns/openshift-multus...
2026-05-21T21:55:36.466180240Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-21T21:55:37.226455816Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-21T21:55:38.592641144Z Collecting kubelet logs for node ip-10-0-129-28.ec2.internal
2026-05-21T21:55:38.594927813Z Collecting kubelet logs for node ip-10-0-133-99.ec2.internal
2026-05-21T21:55:38.597113723Z Collecting kubelet logs for node ip-10-0-135-166.ec2.internal
2026-05-21T21:55:38.907172545Z Gathering data for ns/openshift-host-network...
2026-05-21T21:55:38.999475232Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-21T21:55:39.005479368Z INFO: Node performance data collection complete.
2026-05-21T21:55:39.336368218Z Gathering data for ns/openshift-network-diagnostics...
2026-05-21T21:55:39.926754274Z Gathering data for ns/openshift-network-node-identity...
2026-05-21T21:55:40.472153124Z Gathering data for ns/openshift-network-operator...
2026-05-21T21:55:40.949450408Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-21T21:55:41.271781008Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-21T21:55:41.471055395Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-21T21:55:41.800177567Z Gathering data for ns/openshift-apiserver-operator...
2026-05-21T21:55:42.157275566Z Gathering data for ns/openshift-apiserver...
2026-05-21T21:55:42.669948524Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-21T21:55:42.962024341Z Gathering data for ns/openshift-controller-manager...
2026-05-21T21:55:43.283633099Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-21T21:55:43.957483041Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-21T21:55:44.316514688Z Gathering data for ns/openshift-service-ca-operator...
2026-05-21T21:55:44.655293183Z Gathering data for ns/openshift-service-ca...
2026-05-21T21:55:45.072290549Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-21T21:55:46.069706716Z Wrote inspect data to must-gather.
2026-05-21T21:55:46.069748467Z error: inspection completed with the errors occurred while gathering data:
2026-05-21T21:55:46.069748467Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-21T21:55:46.203704806Z error: the server doesn't have a resource type "clusters"
2026-05-21T21:55:46.475779215Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-05-21T21:55:46.486702361Z Caches written to disk