2026-05-21T15:15:40.001840481Z [disk usage checker] Started
2026-05-21T15:15:40.005600284Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-21T15:15:40.232519369Z Gathering data for ns/openshift-cluster-version...
2026-05-21T15:15:40.384792642Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-21T15:15:40.472676592Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-21T15:15:40.569976960Z Gathering data for ns/default...
2026-05-21T15:15:40.914366715Z Gathering data for ns/openshift...
2026-05-21T15:15:41.384291824Z Gathering data for ns/kube-system...
2026-05-21T15:15:42.078773461Z Gathering data for ns/openshift-etcd...
2026-05-21T15:15:42.437584889Z Wrote inspect data to must-gather.
2026-05-21T15:15:42.437801856Z error: inspection completed with the errors occurred while gathering data:
2026-05-21T15:15:42.437801856Z     namespaces "assisted-installer" not found
2026-05-21T15:15:42.525304064Z Waiting on subprocesses to finish execution.
2026-05-21T15:15:42.531989287Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-21T15:15:42.533267545Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-21T15:15:42.536053098Z INFO: Collecting host service logs for crio
2026-05-21T15:15:42.536316160Z INFO: Collecting host service logs for kubelet
2026-05-21T15:15:42.536362388Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-21T15:15:42.536668010Z INFO: Collecting host service logs for rpm-ostreed
2026-05-21T15:15:42.537014770Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-21T15:15:42.537354420Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-21T15:15:42.537717752Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-21T15:15:42.538074208Z INFO: Collecting host service logs for NetworkManager
2026-05-21T15:15:42.538408046Z INFO: Collecting host service logs for openvswitch
2026-05-21T15:15:42.538742435Z INFO: Collecting host service logs for ovs-configuration
2026-05-21T15:15:42.539138523Z INFO: Collecting host service logs for ovsdb-server
2026-05-21T15:15:42.539467552Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-21T15:15:42.539871345Z INFO: Waiting for worker host service log collection to complete ...
2026-05-21T15:15:42.557990246Z INFO: Gathering HAProxy config files
2026-05-21T15:15:42.883684252Z INFO: Waiting for node performance related collection to complete ...
2026-05-21T15:15:43.141284165Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-zgjn8
2026-05-21T15:15:43.141284165Z insights-runtime-extractor-24hvv
2026-05-21T15:15:43.141284165Z insights-runtime-extractor-j4wwd
2026-05-21T15:15:43.141284165Z insights-runtime-extractor-zzmjv
2026-05-21T15:15:43.241937105Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-21T15:15:43.648212399Z No resources found in openshift-etcd namespace.
2026-05-21T15:15:43.658695763Z No resources found
2026-05-21T15:15:43.667700930Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-21T15:15:43.755010205Z INFO: "metallb-operator" not detected. Skipping.
2026-05-21T15:15:43.881173637Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-21T15:15:43.888935705Z Wrote inspect data to must-gather.
2026-05-21T15:15:43.996196284Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-21T15:15:43.996479809Z INFO: on-disk MachineConfig config collection complete.
2026-05-21T15:15:44.053913519Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-21T15:15:44.089639925Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-21T15:15:44.159869275Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-21T15:15:44.159869275Z See 'oc rsync -h' for help and examples
2026-05-21T15:15:44.238253177Z INFO: Worker host service log collection to complete.
2026-05-21T15:15:44.337041213Z error: the server doesn't have a resource type "performanceprofile"
2026-05-21T15:15:44.593230192Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-21T15:15:44.669979734Z Wrote inspect data to must-gather.
2026-05-21T15:15:44.675776737Z No resources found
2026-05-21T15:15:44.810403430Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-21T15:15:44.903201627Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-21T15:15:45.073976662Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-21T15:15:45.075349592Z ERROR: No running kube-apiserver pods found
2026-05-21T15:15:45.103876907Z Wrote inspect data to must-gather.
2026-05-21T15:15:45.176286375Z Wrote inspect data to must-gather.
2026-05-21T15:15:45.191115543Z Wrote inspect data to must-gather.
2026-05-21T15:15:45.295279315Z INFO: 'previous-logs' folder not found on ip-10-0-130-146.ec2.internal, skipping...
2026-05-21T15:15:45.369780404Z Wrote inspect data to must-gather.
2026-05-21T15:15:45.429865771Z tar: Removing leading `/' from member names
2026-05-21T15:15:45.442649555Z INFO: HAProxy config collection complete.
2026-05-21T15:15:45.482878323Z Gathering data for ns/kuadrant-system...
2026-05-21T15:15:45.509539061Z INFO: Getting rules from prometheus-k8s-0
2026-05-21T15:15:45.615907507Z Wrote inspect data to must-gather.
2026-05-21T15:15:45.642710903Z error: the server doesn't have a resource type "machineconfigs"
2026-05-21T15:15:45.672221997Z INFO: 'previous-logs' folder not found on ip-10-0-132-243.ec2.internal, skipping...
2026-05-21T15:15:45.946846310Z Wrote inspect data to must-gather.
2026-05-21T15:15:45.972207646Z Wrote inspect data to must-gather.
2026-05-21T15:15:45.995635365Z INFO: Getting status/config from prometheus-k8s-0
2026-05-21T15:15:46.007784897Z INFO: 'previous-logs' folder not found on ip-10-0-134-244.ec2.internal, skipping...
2026-05-21T15:15:46.007824893Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-21T15:15:46.007852574Z INFO: Machine Config Daemon termination log collection complete.
2026-05-21T15:15:46.243103246Z Wrote inspect data to must-gather.
2026-05-21T15:15:46.291954291Z Wrote inspect data to must-gather.
2026-05-21T15:15:46.451653862Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-21T15:15:46.522299982Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-21T15:15:46.675219702Z INFO: INTERCONNECT MODE
2026-05-21T15:15:46.675332358Z INFO: Gathering ovn-kubernetes DBs
2026-05-21T15:15:46.784363727Z Wrote inspect data to must-gather.
2026-05-21T15:15:46.855609790Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-21T15:15:46.884368069Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-21T15:15:46.893944218Z INFO: Gathering OVN_Northbound from ovnkube-node-5t966...
2026-05-21T15:15:46.894213744Z INFO: Gathering OVN_Northbound from ovnkube-node-hb8kt...
2026-05-21T15:15:46.894474055Z INFO: Gathering OVN_Northbound from ovnkube-node-xz5bc...
2026-05-21T15:15:46.894980189Z INFO: Gathering OVN_Southbound from ovnkube-node-5t966...
2026-05-21T15:15:46.895259763Z INFO: Gathering OVN_Southbound from ovnkube-node-hb8kt...
2026-05-21T15:15:46.895504412Z INFO: Gathering OVN_Southbound from ovnkube-node-xz5bc...
2026-05-21T15:15:47.211405615Z Wrote inspect data to must-gather.
2026-05-21T15:15:47.214294799Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-21T15:15:47.232739925Z tar: Removing leading `/' from member names
2026-05-21T15:15:47.286946894Z tar: Removing leading `/' from member names
2026-05-21T15:15:47.346750321Z tar: Removing leading `/' from member names
2026-05-21T15:15:47.354251800Z tar: Removing leading `/' from member names
2026-05-21T15:15:47.372778090Z tar: Removing leading `/' from member names
2026-05-21T15:15:47.388953622Z tar: Removing leading `/' from member names
2026-05-21T15:15:47.572571375Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-21T15:15:47.591259637Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-21T15:15:47.733589552Z Wrote inspect data to must-gather.
2026-05-21T15:15:47.875722605Z Gathering data for ns/openshift-monitoring...
2026-05-21T15:15:48.120312847Z INFO: Getting status from alertmanager-main-0
2026-05-21T15:15:48.161519939Z INFO: Waiting for network log collection to complete ...
2026-05-21T15:15:48.161556818Z INFO: Waiting for ovnk database copies to complete ...
2026-05-21T15:15:48.161942201Z INFO: Copying ovnk databases complete.
2026-05-21T15:15:48.163603550Z 4.2M	must-gather/network_logs/ovnk_database_store
2026-05-21T15:15:48.166663500Z ovnk_database_store/
2026-05-21T15:15:48.166711918Z ovnk_database_store/ovnkube-node-5t966_sbdb
2026-05-21T15:15:48.223542136Z Wrote inspect data to must-gather.
2026-05-21T15:15:48.232955019Z ovnk_database_store/ovnkube-node-hb8kt_nbdb
2026-05-21T15:15:48.244494002Z ovnk_database_store/ovnkube-node-hb8kt_sbdb
2026-05-21T15:15:48.272114264Z ovnk_database_store/ovnkube-node-xz5bc_nbdb
2026-05-21T15:15:48.282491272Z ovnk_database_store/ovnkube-node-5t966_nbdb
2026-05-21T15:15:48.291862326Z ovnk_database_store/ovnkube-node-xz5bc_sbdb
2026-05-21T15:15:48.319988111Z INFO: Network log collection complete.
2026-05-21T15:15:48.561306310Z Wrote inspect data to must-gather.
2026-05-21T15:15:48.850203381Z Wrote inspect data to must-gather.
2026-05-21T15:15:49.180358259Z Wrote inspect data to must-gather.
2026-05-21T15:15:49.485367368Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-21T15:15:49.489191262Z INFO: Fallback to identify the container image from release info
2026-05-21T15:15:49.988368911Z Gathering data for ns/openshift-network-console...
2026-05-21T15:15:50.075649973Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda
2026-05-21T15:15:50.089364840Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-21T15:15:50.219920228Z daemonset.apps/perf-node-gather-daemonset created
2026-05-21T15:15:50.387429287Z Waiting for performance profile collector pods to become ready: 1
2026-05-21T15:15:50.512617588Z Gathering data for ns/openshift-console-operator...
2026-05-21T15:15:51.018641178Z Gathering data for ns/openshift-console...
2026-05-21T15:15:51.511373434Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-21T15:15:51.532602302Z Waiting for performance profile collector pods to become ready: 2
2026-05-21T15:15:51.947288389Z Gathering data for ns/openshift-dns-operator...
2026-05-21T15:15:52.287509642Z Gathering data for ns/openshift-dns...
2026-05-21T15:15:52.294165387Z Wrote inspect data to must-gather.
2026-05-21T15:15:52.652087310Z Waiting for performance profile collector pods to become ready: 3
2026-05-21T15:15:52.872358955Z Gathering data for ns/openshift-image-registry...
2026-05-21T15:15:53.363851903Z Gathering data for ns/openshift-ingress-operator...
2026-05-21T15:15:53.739214173Z Gathering data for ns/openshift-ingress...
2026-05-21T15:15:53.784542782Z Waiting for performance profile collector pods to become ready: 4
2026-05-21T15:15:54.509494733Z Gathering data for ns/openshift-ingress-canary...
2026-05-21T15:15:54.928621424Z Waiting for performance profile collector pods to become ready: 5
2026-05-21T15:15:54.972026710Z Gathering data for ns/openshift-insights...
2026-05-21T15:15:55.096894886Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-21T15:15:56.038656299Z Waiting for performance profile collector pods to become ready: 6
2026-05-21T15:15:56.899342531Z Gathering data for ns/opendatahub...
2026-05-21T15:15:57.151449919Z Waiting for performance profile collector pods to become ready: 7
2026-05-21T15:15:58.265795594Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-21T15:15:58.308566855Z Gathering data for ns/openshift-lws-operator...
2026-05-21T15:15:58.553786277Z Collecting performance related data for node ip-10-0-132-243.ec2.internal
2026-05-21T15:15:58.560465549Z Collecting performance related data for node ip-10-0-130-146.ec2.internal
2026-05-21T15:15:58.567809015Z Collecting performance related data for node ip-10-0-134-244.ec2.internal
2026-05-21T15:15:59.860349417Z Gathering data for ns/openshift-config...
2026-05-21T15:16:00.105895324Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-21T15:16:00.227802930Z Gathering data for ns/openshift-config-managed...
2026-05-21T15:16:00.774795954Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-21T15:16:01.116519956Z Gathering data for ns/openshift-kube-apiserver...
2026-05-21T15:16:01.465672731Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-21T15:16:01.807102574Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-21T15:16:02.166855494Z Gathering data for ns/openshift-kube-scheduler...
2026-05-21T15:16:02.447301096Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-21T15:16:02.774862993Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-21T15:16:03.113621168Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-21T15:16:03.461692689Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-21T15:16:04.102272363Z Gathering data for ns/openshift-multus...
2026-05-21T15:16:05.112870933Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-21T15:16:05.461623928Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-21T15:16:07.147177168Z Collecting kubelet logs for node ip-10-0-132-243.ec2.internal
2026-05-21T15:16:07.149461060Z Collecting kubelet logs for node ip-10-0-130-146.ec2.internal
2026-05-21T15:16:07.151715410Z Collecting kubelet logs for node ip-10-0-134-244.ec2.internal
2026-05-21T15:16:07.219785725Z Gathering data for ns/openshift-host-network...
2026-05-21T15:16:07.579222899Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-21T15:16:07.585532076Z INFO: Node performance data collection complete.
2026-05-21T15:16:07.639503370Z Gathering data for ns/openshift-network-diagnostics...
2026-05-21T15:16:08.195291509Z Gathering data for ns/openshift-network-node-identity...
2026-05-21T15:16:08.631698238Z Gathering data for ns/openshift-network-operator...
2026-05-21T15:16:08.998065029Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-21T15:16:09.314817848Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-21T15:16:09.771288442Z Gathering data for ns/openshift-apiserver-operator...
2026-05-21T15:16:10.089900177Z Gathering data for ns/openshift-apiserver...
2026-05-21T15:16:10.120459267Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-05-21T15:16:10.520399638Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-21T15:16:10.791270659Z Gathering data for ns/openshift-controller-manager...
2026-05-21T15:16:11.081541457Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-21T15:16:11.745144638Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-21T15:16:12.060931504Z Gathering data for ns/openshift-service-ca-operator...
2026-05-21T15:16:12.375575465Z Gathering data for ns/openshift-service-ca...
2026-05-21T15:16:12.749770097Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-21T15:16:13.759748978Z Wrote inspect data to must-gather.
2026-05-21T15:16:13.760000477Z error: inspection completed with the errors occurred while gathering data:
2026-05-21T15:16:13.760000477Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-21T15:16:13.892142384Z error: the server doesn't have a resource type "clusters"
2026-05-21T15:16:14.249824590Z Caches written to disk