2026-06-03T16:40:49.831975120Z [disk usage checker] Started
2026-06-03T16:40:49.835539341Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-06-03T16:40:50.067254170Z Gathering data for ns/openshift-cluster-version...
2026-06-03T16:40:50.229040612Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-03T16:40:50.334569442Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-03T16:40:50.438188520Z Gathering data for ns/default...
2026-06-03T16:40:50.911322318Z Gathering data for ns/openshift...
2026-06-03T16:40:51.394196195Z Gathering data for ns/kube-system...
2026-06-03T16:40:52.206403110Z Gathering data for ns/openshift-etcd...
2026-06-03T16:40:52.284863103Z Waiting on subprocesses to finish execution.
2026-06-03T16:40:52.289861260Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-03T16:40:52.299280087Z INFO: Collecting host service logs for crio
2026-06-03T16:40:52.299608092Z INFO: Collecting host service logs for kubelet
2026-06-03T16:40:52.299956677Z INFO: Collecting host service logs for rpm-ostreed
2026-06-03T16:40:52.300388641Z INFO: Gathering HAProxy config files
2026-06-03T16:40:52.300628718Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-03T16:40:52.300746277Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-03T16:40:52.301044063Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-03T16:40:52.301377259Z INFO: Collecting host service logs for NetworkManager
2026-06-03T16:40:52.301680236Z INFO: Collecting host service logs for openvswitch
2026-06-03T16:40:52.301970948Z INFO: Collecting host service logs for ovs-configuration
2026-06-03T16:40:52.302927229Z INFO: Collecting host service logs for ovsdb-server
2026-06-03T16:40:52.303259740Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-03T16:40:52.303652741Z INFO: Waiting for worker host service log collection to complete ...
2026-06-03T16:40:52.305258634Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-03T16:40:52.318667219Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-03T16:40:52.436477981Z INFO: Waiting for node performance related collection to complete ...
2026-06-03T16:40:53.123835428Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-03T16:40:53.194185268Z No resources found in openshift-etcd namespace.
2026-06-03T16:40:53.332496467Z INFO: "metallb-operator" not detected. Skipping.
2026-06-03T16:40:53.377655812Z INFO: Collecting Insights Archives from insights-operator-788bf58df5-lkzx5
2026-06-03T16:40:53.377711643Z insights-runtime-extractor-797v2
2026-06-03T16:40:53.377733380Z insights-runtime-extractor-scd2b
2026-06-03T16:40:53.377754475Z insights-runtime-extractor-tlf2x
2026-06-03T16:40:53.428766859Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-03T16:40:53.579244293Z No resources found
2026-06-03T16:40:53.679545594Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-03T16:40:53.679581325Z INFO: on-disk MachineConfig config collection complete.
2026-06-03T16:40:53.730667756Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-03T16:40:53.853819317Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-03T16:40:53.853819317Z See 'oc rsync -h' for help and examples
2026-06-03T16:40:53.855551566Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-03T16:40:53.865788187Z Wrote inspect data to must-gather.
2026-06-03T16:40:53.865877461Z error: inspection completed with the errors occurred while gathering data:
2026-06-03T16:40:53.865877461Z     namespaces "assisted-installer" not found
2026-06-03T16:40:53.877310820Z error: the server doesn't have a resource type "performanceprofile"
2026-06-03T16:40:53.941899537Z INFO: Worker host service log collection to complete.
2026-06-03T16:40:53.971779668Z Wrote inspect data to must-gather.
2026-06-03T16:40:54.050287971Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-03T16:40:54.675012052Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-03T16:40:54.752517178Z Wrote inspect data to must-gather.
2026-06-03T16:40:54.969720286Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-06-03T16:40:55.000540973Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-03T16:40:55.131593587Z No resources found
2026-06-03T16:40:55.154399155Z INFO: 'previous-logs' folder not found on ip-10-0-128-141.ec2.internal, skipping...
2026-06-03T16:40:55.219321014Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-03T16:40:55.236731383Z Wrote inspect data to must-gather.
2026-06-03T16:40:55.238054755Z ERROR: No running kube-apiserver pods found
2026-06-03T16:40:55.321492735Z Wrote inspect data to must-gather.
2026-06-03T16:40:55.323073046Z Wrote inspect data to must-gather.
2026-06-03T16:40:55.419334854Z tar: Removing leading `/' from member names
2026-06-03T16:40:55.430122664Z INFO: HAProxy config collection complete.
2026-06-03T16:40:55.449662814Z error: the server doesn't have a resource type "machineconfigs"
2026-06-03T16:40:55.506725954Z Gathering data for ns/kuadrant-system...
2026-06-03T16:40:55.578803177Z INFO: 'previous-logs' folder not found on ip-10-0-139-173.ec2.internal, skipping...
2026-06-03T16:40:55.621534676Z Wrote inspect data to must-gather.
2026-06-03T16:40:55.787309543Z INFO: Getting rules from prometheus-k8s-0
2026-06-03T16:40:55.849663434Z Wrote inspect data to must-gather.
2026-06-03T16:40:55.932224563Z INFO: 'previous-logs' folder not found on ip-10-0-143-16.ec2.internal, skipping...
2026-06-03T16:40:55.932266499Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-03T16:40:55.932279401Z INFO: Machine Config Daemon termination log collection complete.
2026-06-03T16:40:56.117265202Z Wrote inspect data to must-gather.
2026-06-03T16:40:56.268646187Z Wrote inspect data to must-gather.
2026-06-03T16:40:56.291107583Z INFO: Getting status/config from prometheus-k8s-0
2026-06-03T16:40:56.376933005Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-03T16:40:56.477043678Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-03T16:40:56.595118613Z Wrote inspect data to must-gather.
2026-06-03T16:40:56.808339614Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-03T16:40:56.864203107Z W0603 16:40:56.864154    1128 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-03T16:40:56.900777615Z Wrote inspect data to must-gather.
2026-06-03T16:40:56.978853221Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-03T16:40:57.108483194Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-03T16:40:57.195041745Z Wrote inspect data to must-gather.
2026-06-03T16:40:57.222097765Z Gathering data for ns/openshift-monitoring...
2026-06-03T16:40:57.388036679Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-03T16:40:57.422593746Z Wrote inspect data to must-gather.
2026-06-03T16:40:57.538611490Z INFO: INTERCONNECT MODE
2026-06-03T16:40:57.538650450Z INFO: Gathering ovn-kubernetes DBs
2026-06-03T16:40:57.640475322Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-03T16:40:57.741853010Z INFO: Gathering OVN_Northbound from ovnkube-node-5qlvf...
2026-06-03T16:40:57.742072724Z INFO: Gathering OVN_Northbound from ovnkube-node-68jfx...
2026-06-03T16:40:57.742334742Z INFO: Gathering OVN_Northbound from ovnkube-node-ht2hb...
2026-06-03T16:40:57.742678821Z INFO: Gathering OVN_Southbound from ovnkube-node-5qlvf...
2026-06-03T16:40:57.742983508Z INFO: Gathering OVN_Southbound from ovnkube-node-68jfx...
2026-06-03T16:40:57.743304669Z INFO: Gathering OVN_Southbound from ovnkube-node-ht2hb...
2026-06-03T16:40:57.986144753Z Wrote inspect data to must-gather.
2026-06-03T16:40:58.053637250Z tar: Removing leading `/' from member names
2026-06-03T16:40:58.095789338Z tar: Removing leading `/' from member names
2026-06-03T16:40:58.181071852Z tar: Removing leading `/' from member names
2026-06-03T16:40:58.181071852Z tar: Removing leading `/' from member names
2026-06-03T16:40:58.185500274Z tar: Removing leading `/' from member names
2026-06-03T16:40:58.231013357Z tar: Removing leading `/' from member names
2026-06-03T16:40:58.288911608Z INFO: Getting status from alertmanager-main-0
2026-06-03T16:40:58.497152417Z Wrote inspect data to must-gather.
2026-06-03T16:40:58.873759583Z INFO: Waiting for network log collection to complete ...
2026-06-03T16:40:58.873812591Z INFO: Waiting for ovnk database copies to complete ...
2026-06-03T16:40:58.874336082Z INFO: Copying ovnk databases complete.
2026-06-03T16:40:58.876412121Z 4.2M	must-gather/network_logs/ovnk_database_store
2026-06-03T16:40:58.878521213Z ovnk_database_store/
2026-06-03T16:40:58.878521213Z ovnk_database_store/ovnkube-node-68jfx_nbdb
2026-06-03T16:40:58.890237092Z ovnk_database_store/ovnkube-node-5qlvf_sbdb
2026-06-03T16:40:58.913401698Z ovnk_database_store/ovnkube-node-ht2hb_sbdb
2026-06-03T16:40:58.941956453Z ovnk_database_store/ovnkube-node-ht2hb_nbdb
2026-06-03T16:40:58.945668840Z Wrote inspect data to must-gather.
2026-06-03T16:40:58.951490715Z ovnk_database_store/ovnkube-node-5qlvf_nbdb
2026-06-03T16:40:58.960993773Z ovnk_database_store/ovnkube-node-68jfx_sbdb
2026-06-03T16:40:58.992257277Z INFO: Network log collection complete.
2026-06-03T16:40:59.279728589Z W0603 16:40:59.279676    1440 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-03T16:40:59.304910701Z Wrote inspect data to must-gather.
2026-06-03T16:40:59.563841279Z Gathering data for ns/openshift-network-console...
2026-06-03T16:40:59.769597605Z Wrote inspect data to must-gather.
2026-06-03T16:41:00.006623522Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-06-03T16:41:00.089997032Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-03T16:41:00.092960349Z INFO: Fallback to identify the container image from release info
2026-06-03T16:41:00.127504279Z Gathering data for ns/openshift-console-operator...
2026-06-03T16:41:00.585629049Z Gathering data for ns/openshift-console...
2026-06-03T16:41:00.790775057Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d8381cb85e479760bd6b601e111b39e6c56a632bfa298867f4197ec4421e9f5
2026-06-03T16:41:00.932767376Z daemonset.apps/perf-node-gather-daemonset created
2026-06-03T16:41:01.088163813Z Waiting for performance profile collector pods to become ready: 1
2026-06-03T16:41:01.179641682Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-03T16:41:01.655977893Z Gathering data for ns/openshift-dns-operator...
2026-06-03T16:41:02.010085147Z Gathering data for ns/openshift-dns...
2026-06-03T16:41:02.082124666Z Wrote inspect data to must-gather.
2026-06-03T16:41:02.195548373Z Waiting for performance profile collector pods to become ready: 2
2026-06-03T16:41:02.641055314Z Gathering data for ns/openshift-image-registry...
2026-06-03T16:41:03.116292137Z Gathering data for ns/openshift-ingress-operator...
2026-06-03T16:41:03.308716035Z Waiting for performance profile collector pods to become ready: 3
2026-06-03T16:41:03.484785503Z Gathering data for ns/openshift-ingress...
2026-06-03T16:41:04.184403089Z Gathering data for ns/openshift-ingress-canary...
2026-06-03T16:41:04.427385874Z Waiting for performance profile collector pods to become ready: 4
2026-06-03T16:41:04.641496014Z Gathering data for ns/openshift-insights...
2026-06-03T16:41:05.011475649Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-06-03T16:41:05.541499094Z Waiting for performance profile collector pods to become ready: 5
2026-06-03T16:41:06.666579595Z Waiting for performance profile collector pods to become ready: 6
2026-06-03T16:41:06.750497883Z Gathering data for ns/opendatahub...
2026-06-03T16:41:07.792561231Z Waiting for performance profile collector pods to become ready: 7
2026-06-03T16:41:08.434810840Z Gathering data for ns/openshift-lws-operator...
2026-06-03T16:41:08.929855436Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-03T16:41:09.188527205Z Collecting performance related data for node ip-10-0-128-141.ec2.internal
2026-06-03T16:41:09.195843964Z Collecting performance related data for node ip-10-0-139-173.ec2.internal
2026-06-03T16:41:09.203088532Z Collecting performance related data for node ip-10-0-143-16.ec2.internal
2026-06-03T16:41:10.019610851Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-06-03T16:41:10.175721386Z Gathering data for ns/openshift-config...
2026-06-03T16:41:10.592125081Z Gathering data for ns/openshift-config-managed...
2026-06-03T16:41:11.100752857Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-03T16:41:11.417066360Z Gathering data for ns/openshift-kube-apiserver...
2026-06-03T16:41:11.769136774Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-03T16:41:12.197161092Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-03T16:41:12.555001537Z Gathering data for ns/openshift-kube-scheduler...
2026-06-03T16:41:12.850621472Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-03T16:41:13.205703580Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-03T16:41:13.571828427Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-03T16:41:13.926607181Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-03T16:41:14.545339963Z Gathering data for ns/openshift-multus...
2026-06-03T16:41:15.025104541Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-06-03T16:41:15.982327563Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-03T16:41:17.529799381Z Gathering data for ns/openshift-host-network...
2026-06-03T16:41:17.752252506Z Collecting kubelet logs for node ip-10-0-128-141.ec2.internal
2026-06-03T16:41:17.754132660Z Collecting kubelet logs for node ip-10-0-139-173.ec2.internal
2026-06-03T16:41:17.756307632Z Collecting kubelet logs for node ip-10-0-143-16.ec2.internal
2026-06-03T16:41:17.965144640Z Gathering data for ns/openshift-network-diagnostics...
2026-06-03T16:41:18.136796000Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-06-03T16:41:18.143789360Z INFO: Node performance data collection complete.
2026-06-03T16:41:18.635866257Z Gathering data for ns/openshift-network-node-identity...
2026-06-03T16:41:19.109634642Z Gathering data for ns/openshift-network-operator...
2026-06-03T16:41:19.517414700Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-03T16:41:19.818644836Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-03T16:41:20.029595143Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-06-03T16:41:20.317569946Z Gathering data for ns/openshift-apiserver-operator...
2026-06-03T16:41:20.633701813Z Gathering data for ns/openshift-apiserver...
2026-06-03T16:41:21.092468210Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-03T16:41:21.402082594Z Gathering data for ns/openshift-controller-manager...
2026-06-03T16:41:21.714847210Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-03T16:41:22.387291945Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-03T16:41:22.720509688Z Gathering data for ns/openshift-service-ca-operator...
2026-06-03T16:41:23.074712330Z Gathering data for ns/openshift-service-ca...
2026-06-03T16:41:23.504751752Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-03T16:41:24.543834079Z Wrote inspect data to must-gather.
2026-06-03T16:41:24.543881537Z error: inspection completed with the errors occurred while gathering data:
2026-06-03T16:41:24.543881537Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-03T16:41:24.681651735Z error: the server doesn't have a resource type "clusters"
2026-06-03T16:41:24.857150698Z Caches written to disk