2026-06-03T15:21:56.989023814Z [disk usage checker] Started
2026-06-03T15:21:56.992670052Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-03T15:21:57.214856629Z Gathering data for ns/openshift-cluster-version...
2026-06-03T15:21:57.391488737Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-03T15:21:57.505174034Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-03T15:21:57.661395094Z Gathering data for ns/default...
2026-06-03T15:21:58.153828609Z Gathering data for ns/openshift...
2026-06-03T15:21:58.646107246Z Gathering data for ns/kube-system...
2026-06-03T15:21:59.418611771Z Waiting on subprocesses to finish execution.
2026-06-03T15:21:59.427140637Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-03T15:21:59.436422289Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-03T15:21:59.438465696Z INFO: Gathering HAProxy config files
2026-06-03T15:21:59.441167623Z INFO: Collecting host service logs for crio
2026-06-03T15:21:59.441462035Z INFO: Collecting host service logs for kubelet
2026-06-03T15:21:59.441769753Z INFO: Collecting host service logs for rpm-ostreed
2026-06-03T15:21:59.442101998Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-03T15:21:59.442441190Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-03T15:21:59.442747777Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-03T15:21:59.443290310Z INFO: Collecting host service logs for NetworkManager
2026-06-03T15:21:59.443623433Z INFO: Collecting host service logs for openvswitch
2026-06-03T15:21:59.443927860Z INFO: Collecting host service logs for ovs-configuration
2026-06-03T15:21:59.444343742Z INFO: Collecting host service logs for ovsdb-server
2026-06-03T15:21:59.444667752Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-03T15:21:59.445036068Z INFO: Waiting for worker host service log collection to complete ...
2026-06-03T15:21:59.458138224Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-03T15:21:59.495828161Z Gathering data for ns/openshift-etcd...
2026-06-03T15:21:59.735533205Z INFO: Waiting for node performance related collection to complete ...
2026-06-03T15:22:00.384056468Z INFO: Collecting Insights Archives from insights-operator-788bf58df5-lkk8x
2026-06-03T15:22:00.384109672Z insights-runtime-extractor-dr6lc
2026-06-03T15:22:00.384130641Z insights-runtime-extractor-qnccz
2026-06-03T15:22:00.384150684Z insights-runtime-extractor-wrzg5
2026-06-03T15:22:00.414764793Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-03T15:22:00.414797537Z INFO: on-disk MachineConfig config collection complete.
2026-06-03T15:22:00.430643687Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-03T15:22:00.481286226Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-03T15:22:00.562717002Z No resources found
2026-06-03T15:22:00.656613070Z No resources found in openshift-etcd namespace.
2026-06-03T15:22:00.829082088Z error: the server doesn't have a resource type "performanceprofile"
2026-06-03T15:22:00.881992453Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-03T15:22:00.918386939Z ERROR: No running kube-apiserver pods found
2026-06-03T15:22:00.971223791Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-03T15:22:01.234830649Z INFO: Worker host service log collection to complete.
2026-06-03T15:22:01.251788071Z INFO: "metallb-operator" not detected. Skipping.
2026-06-03T15:22:01.332248805Z Wrote inspect data to must-gather.
2026-06-03T15:22:01.664929395Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-03T15:22:01.728436977Z Wrote inspect data to must-gather.
2026-06-03T15:22:01.728535822Z error: inspection completed with the errors occurred while gathering data:
2026-06-03T15:22:01.728535822Z     namespaces "assisted-installer" not found
2026-06-03T15:22:01.869962441Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-03T15:22:01.942799110Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-03T15:22:02.126630026Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-03T15:22:02.126630026Z See 'oc rsync -h' for help and examples
2026-06-03T15:22:02.129643599Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-03T15:22:02.183447276Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-03T15:22:02.272457247Z No resources found
2026-06-03T15:22:02.361496515Z Wrote inspect data to must-gather.
2026-06-03T15:22:02.412458010Z Wrote inspect data to must-gather.
2026-06-03T15:22:02.414254645Z INFO: 'previous-logs' folder not found on ip-10-0-130-78.ec2.internal, skipping...
2026-06-03T15:22:02.535113949Z Wrote inspect data to must-gather.
2026-06-03T15:22:02.546553074Z tar: Removing leading `/' from member names
2026-06-03T15:22:02.553223918Z INFO: HAProxy config collection complete.
2026-06-03T15:22:02.729130818Z error: the server doesn't have a resource type "machineconfigs"
2026-06-03T15:22:02.792422955Z INFO: Getting rules from prometheus-k8s-0
2026-06-03T15:22:02.801408375Z Wrote inspect data to must-gather.
2026-06-03T15:22:02.832485009Z Wrote inspect data to must-gather.
2026-06-03T15:22:02.834704795Z Gathering data for ns/kuadrant-system...
2026-06-03T15:22:02.846236048Z INFO: 'previous-logs' folder not found on ip-10-0-133-0.ec2.internal, skipping...
2026-06-03T15:22:03.114036681Z Wrote inspect data to must-gather.
2026-06-03T15:22:03.169676371Z INFO: 'previous-logs' folder not found on ip-10-0-133-25.ec2.internal, skipping...
2026-06-03T15:22:03.169790426Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-03T15:22:03.169845792Z INFO: Machine Config Daemon termination log collection complete.
2026-06-03T15:22:03.213688083Z Wrote inspect data to must-gather.
2026-06-03T15:22:03.306172814Z INFO: Getting status/config from prometheus-k8s-0
2026-06-03T15:22:03.525165687Z Wrote inspect data to must-gather.
2026-06-03T15:22:03.606843384Z Wrote inspect data to must-gather.
2026-06-03T15:22:03.851351517Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-03T15:22:03.866144007Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-03T15:22:03.961417692Z Wrote inspect data to must-gather.
2026-06-03T15:22:04.060572070Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-03T15:22:04.170797271Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-03T15:22:04.228125146Z Wrote inspect data to must-gather.
2026-06-03T15:22:04.387121667Z INFO: INTERCONNECT MODE
2026-06-03T15:22:04.387163266Z INFO: Gathering ovn-kubernetes DBs
2026-06-03T15:22:04.435191871Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-03T15:22:04.617529864Z INFO: Gathering OVN_Northbound from ovnkube-node-7h852...
2026-06-03T15:22:04.617829463Z INFO: Gathering OVN_Northbound from ovnkube-node-bfwpp...
2026-06-03T15:22:04.618070409Z INFO: Gathering OVN_Northbound from ovnkube-node-wcfq7...
2026-06-03T15:22:04.618398106Z INFO: Gathering OVN_Southbound from ovnkube-node-7h852...
2026-06-03T15:22:04.618717242Z INFO: Gathering OVN_Southbound from ovnkube-node-bfwpp...
2026-06-03T15:22:04.618999702Z INFO: Gathering OVN_Southbound from ovnkube-node-wcfq7...
2026-06-03T15:22:04.654690801Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-03T15:22:04.777896265Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-03T15:22:04.936147775Z tar: Removing leading `/' from member names
2026-06-03T15:22:05.012324134Z Wrote inspect data to must-gather.
2026-06-03T15:22:05.068268715Z tar: Removing leading `/' from member names
2026-06-03T15:22:05.069092769Z tar: Removing leading `/' from member names
2026-06-03T15:22:05.094614724Z Gathering data for ns/openshift-monitoring...
2026-06-03T15:22:05.119887530Z tar: Removing leading `/' from member names
2026-06-03T15:22:05.180020324Z tar: Removing leading `/' from member names
2026-06-03T15:22:05.220698284Z tar: Removing leading `/' from member names
2026-06-03T15:22:05.454000446Z INFO: Getting status from alertmanager-main-0
2026-06-03T15:22:05.568589753Z Wrote inspect data to must-gather.
2026-06-03T15:22:05.755283294Z INFO: Waiting for network log collection to complete ...
2026-06-03T15:22:05.755319871Z INFO: Waiting for ovnk database copies to complete ...
2026-06-03T15:22:05.755860164Z INFO: Copying ovnk databases complete.
2026-06-03T15:22:05.757522607Z 4.3M	must-gather/network_logs/ovnk_database_store
2026-06-03T15:22:05.759974460Z ovnk_database_store/
2026-06-03T15:22:05.759974460Z ovnk_database_store/ovnkube-node-bfwpp_nbdb
2026-06-03T15:22:05.782496389Z ovnk_database_store/ovnkube-node-wcfq7_sbdb
2026-06-03T15:22:05.814173192Z ovnk_database_store/ovnkube-node-7h852_sbdb
2026-06-03T15:22:05.837462419Z ovnk_database_store/ovnkube-node-wcfq7_nbdb
2026-06-03T15:22:05.848784002Z ovnk_database_store/ovnkube-node-bfwpp_sbdb
2026-06-03T15:22:05.870281226Z ovnk_database_store/ovnkube-node-7h852_nbdb
2026-06-03T15:22:05.885349151Z INFO: Network log collection complete.
2026-06-03T15:22:05.992949146Z Wrote inspect data to must-gather.
2026-06-03T15:22:06.339788203Z Wrote inspect data to must-gather.
2026-06-03T15:22:06.694956574Z Wrote inspect data to must-gather.
2026-06-03T15:22:07.009759592Z Wrote inspect data to must-gather.
2026-06-03T15:22:07.169899090Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-03T15:22:07.321644370Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-03T15:22:07.324474626Z INFO: Fallback to identify the container image from release info
2026-06-03T15:22:07.495336729Z Gathering data for ns/openshift-network-console...
2026-06-03T15:22:07.883511491Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d8381cb85e479760bd6b601e111b39e6c56a632bfa298867f4197ec4421e9f5
2026-06-03T15:22:08.045855143Z daemonset.apps/perf-node-gather-daemonset created
2026-06-03T15:22:08.049314961Z Gathering data for ns/openshift-console-operator...
2026-06-03T15:22:08.206014861Z Waiting for performance profile collector pods to become ready: 1
2026-06-03T15:22:08.650958030Z Gathering data for ns/openshift-console...
2026-06-03T15:22:09.095348639Z W0603 15:22:09.095298     333 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-03T15:22:09.213645648Z Wrote inspect data to must-gather.
2026-06-03T15:22:09.236326394Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-03T15:22:09.344834134Z Waiting for performance profile collector pods to become ready: 2
2026-06-03T15:22:09.662446418Z Gathering data for ns/openshift-dns-operator...
2026-06-03T15:22:10.003820102Z Gathering data for ns/openshift-dns...
2026-06-03T15:22:10.458291909Z Waiting for performance profile collector pods to become ready: 3
2026-06-03T15:22:10.699906859Z Gathering data for ns/openshift-image-registry...
2026-06-03T15:22:11.285050835Z Gathering data for ns/openshift-ingress-operator...
2026-06-03T15:22:11.582601614Z Waiting for performance profile collector pods to become ready: 4
2026-06-03T15:22:11.750885918Z Gathering data for ns/openshift-ingress...
2026-06-03T15:22:12.172421177Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-03T15:22:12.564806814Z Gathering data for ns/openshift-ingress-canary...
2026-06-03T15:22:12.690849286Z Waiting for performance profile collector pods to become ready: 5
2026-06-03T15:22:13.086443410Z Gathering data for ns/openshift-insights...
2026-06-03T15:22:13.818786097Z Waiting for performance profile collector pods to become ready: 6
2026-06-03T15:22:14.925561032Z Waiting for performance profile collector pods to become ready: 7
2026-06-03T15:22:15.390690238Z Gathering data for ns/opendatahub...
2026-06-03T15:22:16.062186039Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-03T15:22:16.309261550Z Collecting performance related data for node ip-10-0-130-78.ec2.internal
2026-06-03T15:22:16.319030343Z Collecting performance related data for node ip-10-0-133-25.ec2.internal
2026-06-03T15:22:16.331602735Z Collecting performance related data for node ip-10-0-133-0.ec2.internal
2026-06-03T15:22:17.178152433Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-03T15:22:17.453347883Z Gathering data for ns/openshift-lws-operator...
2026-06-03T15:22:19.228754361Z Gathering data for ns/openshift-config...
2026-06-03T15:22:19.604648490Z Gathering data for ns/openshift-config-managed...
2026-06-03T15:22:20.123874681Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-03T15:22:20.471590650Z Gathering data for ns/openshift-kube-apiserver...
2026-06-03T15:22:20.854488937Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-03T15:22:21.238206854Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-03T15:22:21.620799399Z Gathering data for ns/openshift-kube-scheduler...
2026-06-03T15:22:21.988594698Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-03T15:22:22.183616637Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-03T15:22:22.375231136Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-03T15:22:22.883886682Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-03T15:22:23.286494400Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-03T15:22:23.990509404Z Gathering data for ns/openshift-multus...
2026-06-03T15:22:24.745018261Z Collecting kubelet logs for node ip-10-0-130-78.ec2.internal
2026-06-03T15:22:24.749501648Z Collecting kubelet logs for node ip-10-0-133-25.ec2.internal
2026-06-03T15:22:24.752928331Z Collecting kubelet logs for node ip-10-0-133-0.ec2.internal
2026-06-03T15:22:25.101565155Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-06-03T15:22:25.107133855Z INFO: Node performance data collection complete.
2026-06-03T15:22:25.549005322Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-03T15:22:27.189520625Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-03T15:22:27.229784304Z Gathering data for ns/openshift-host-network...
2026-06-03T15:22:27.630652104Z Gathering data for ns/openshift-network-diagnostics...
2026-06-03T15:22:28.245712714Z Gathering data for ns/openshift-network-node-identity...
2026-06-03T15:22:28.752695458Z Gathering data for ns/openshift-network-operator...
2026-06-03T15:22:29.181005903Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-03T15:22:29.519393019Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-03T15:22:30.028384117Z Gathering data for ns/openshift-apiserver-operator...
2026-06-03T15:22:30.389184238Z Gathering data for ns/openshift-apiserver...
2026-06-03T15:22:30.885595445Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-03T15:22:31.238217127Z Gathering data for ns/openshift-controller-manager...
2026-06-03T15:22:31.575795318Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-03T15:22:32.194239556Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-03T15:22:32.268302568Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-03T15:22:32.603415966Z Gathering data for ns/openshift-service-ca-operator...
2026-06-03T15:22:32.995357469Z Gathering data for ns/openshift-service-ca...
2026-06-03T15:22:33.431098678Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-03T15:22:34.529230545Z Wrote inspect data to must-gather.
2026-06-03T15:22:34.529274849Z error: inspection completed with the errors occurred while gathering data:
2026-06-03T15:22:34.529274849Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-03T15:22:34.658665488Z error: the server doesn't have a resource type "clusters"
2026-06-03T15:22:34.862064443Z Caches written to disk