2026-06-02T20:16:27.201664825Z [disk usage checker] Started
2026-06-02T20:16:27.205260777Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-06-02T20:16:27.438941860Z Gathering data for ns/openshift-cluster-version...
2026-06-02T20:16:27.631402135Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-02T20:16:27.763473317Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-02T20:16:27.884086838Z Gathering data for ns/default...
2026-06-02T20:16:28.570292072Z Gathering data for ns/openshift...
2026-06-02T20:16:29.073190804Z Gathering data for ns/kube-system...
2026-06-02T20:16:29.620540574Z Waiting on subprocesses to finish execution.
2026-06-02T20:16:29.625725426Z INFO: Collecting host service logs for crio
2026-06-02T20:16:29.626039954Z INFO: Collecting host service logs for kubelet
2026-06-02T20:16:29.626388886Z INFO: Collecting host service logs for rpm-ostreed
2026-06-02T20:16:29.626741111Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-02T20:16:29.627081062Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-02T20:16:29.627399683Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-02T20:16:29.627754435Z INFO: Collecting host service logs for NetworkManager
2026-06-02T20:16:29.628084614Z INFO: Collecting host service logs for openvswitch
2026-06-02T20:16:29.628400412Z INFO: Collecting host service logs for ovs-configuration
2026-06-02T20:16:29.628763049Z INFO: Collecting host service logs for ovsdb-server
2026-06-02T20:16:29.629094679Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-02T20:16:29.629492050Z INFO: Waiting for worker host service log collection to complete ...
2026-06-02T20:16:29.631927399Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-02T20:16:29.636424129Z INFO: Gathering HAProxy config files
2026-06-02T20:16:29.663044222Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-02T20:16:29.704330752Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-02T20:16:29.916295192Z INFO: Waiting for node performance related collection to complete ...
2026-06-02T20:16:30.474370768Z No resources found in openshift-etcd namespace.
2026-06-02T20:16:30.645079058Z INFO: Collecting Insights Archives from insights-operator-788bf58df5-4t95b
2026-06-02T20:16:30.645143782Z insights-runtime-extractor-b5c54
2026-06-02T20:16:30.645165561Z insights-runtime-extractor-njcx7
2026-06-02T20:16:30.645185447Z insights-runtime-extractor-ppm2l
2026-06-02T20:16:30.678973757Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-02T20:16:30.679066926Z INFO: on-disk MachineConfig config collection complete.
2026-06-02T20:16:30.736861248Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-02T20:16:30.767239587Z No resources found
2026-06-02T20:16:30.768288928Z INFO: Worker host service log collection to complete.
2026-06-02T20:16:30.828094135Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-02T20:16:30.828910508Z INFO: "metallb-operator" not detected. Skipping.
2026-06-02T20:16:30.830934236Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-02T20:16:31.113465598Z error: the server doesn't have a resource type "performanceprofile"
2026-06-02T20:16:31.122848755Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-02T20:16:31.203786695Z ERROR: No running kube-apiserver pods found
2026-06-02T20:16:31.250386928Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-02T20:16:31.250386928Z See 'oc rsync -h' for help and examples
2026-06-02T20:16:31.302135888Z Gathering data for ns/openshift-etcd...
2026-06-02T20:16:31.462333588Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-02T20:16:31.543027766Z Wrote inspect data to must-gather.
2026-06-02T20:16:31.626432056Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-02T20:16:31.782929534Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-02T20:16:32.100707363Z tar: Removing leading `/' from member names
2026-06-02T20:16:32.120242571Z INFO: HAProxy config collection complete.
2026-06-02T20:16:32.268336296Z Gathering data for ns/kuadrant-system...
2026-06-02T20:16:32.289545575Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-06-02T20:16:32.322210832Z INFO: 'previous-logs' folder not found on ip-10-0-130-103.ec2.internal, skipping...
2026-06-02T20:16:32.517318930Z No resources found
2026-06-02T20:16:32.637891911Z Wrote inspect data to must-gather.
2026-06-02T20:16:32.807227191Z Wrote inspect data to must-gather.
2026-06-02T20:16:32.827346326Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-02T20:16:32.831242892Z Wrote inspect data to must-gather.
2026-06-02T20:16:32.852948619Z Wrote inspect data to must-gather.
2026-06-02T20:16:33.027473469Z INFO: Getting rules from prometheus-k8s-0
2026-06-02T20:16:33.163320621Z Wrote inspect data to must-gather.
2026-06-02T20:16:33.228850453Z INFO: 'previous-logs' folder not found on ip-10-0-137-56.ec2.internal, skipping...
2026-06-02T20:16:33.240410706Z error: the server doesn't have a resource type "machineconfigs"
2026-06-02T20:16:33.478247575Z Wrote inspect data to must-gather.
2026-06-02T20:16:33.478335578Z error: inspection completed with the errors occurred while gathering data:
2026-06-02T20:16:33.478335578Z     namespaces "assisted-installer" not found
2026-06-02T20:16:33.704231134Z INFO: 'previous-logs' folder not found on ip-10-0-142-203.ec2.internal, skipping...
2026-06-02T20:16:33.704348891Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-02T20:16:33.704408520Z INFO: Machine Config Daemon termination log collection complete.
2026-06-02T20:16:33.723706634Z Wrote inspect data to must-gather.
2026-06-02T20:16:33.747181775Z INFO: Getting status/config from prometheus-k8s-0
2026-06-02T20:16:33.771768650Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-02T20:16:33.861556182Z Wrote inspect data to must-gather.
2026-06-02T20:16:34.260007127Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-02T20:16:34.271779284Z Wrote inspect data to must-gather.
2026-06-02T20:16:34.349949875Z W0602 20:16:34.349903    1087 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-02T20:16:34.393770781Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-02T20:16:34.398479577Z Wrote inspect data to must-gather.
2026-06-02T20:16:34.580810410Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-02T20:16:34.634115814Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-02T20:16:34.651945825Z Gathering data for ns/openshift-monitoring...
2026-06-02T20:16:34.889302852Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-02T20:16:34.950813369Z Wrote inspect data to must-gather.
2026-06-02T20:16:35.158324215Z Wrote inspect data to must-gather.
2026-06-02T20:16:35.182664458Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-02T20:16:35.327636505Z INFO: INTERCONNECT MODE
2026-06-02T20:16:35.327728414Z INFO: Gathering ovn-kubernetes DBs
2026-06-02T20:16:35.521835997Z INFO: Gathering OVN_Northbound from ovnkube-node-79xzx...
2026-06-02T20:16:35.522102748Z INFO: Gathering OVN_Northbound from ovnkube-node-mhvx7...
2026-06-02T20:16:35.522352176Z INFO: Gathering OVN_Northbound from ovnkube-node-xlrd9...
2026-06-02T20:16:35.522659208Z INFO: Gathering OVN_Southbound from ovnkube-node-79xzx...
2026-06-02T20:16:35.522923616Z INFO: Gathering OVN_Southbound from ovnkube-node-mhvx7...
2026-06-02T20:16:35.523170427Z INFO: Gathering OVN_Southbound from ovnkube-node-xlrd9...
2026-06-02T20:16:35.772463654Z INFO: Getting status from alertmanager-main-0
2026-06-02T20:16:35.781495958Z Wrote inspect data to must-gather.
2026-06-02T20:16:35.900745670Z tar: Removing leading `/' from member names
2026-06-02T20:16:35.972626543Z tar: Removing leading `/' from member names
2026-06-02T20:16:35.995816484Z tar: Removing leading `/' from member names
2026-06-02T20:16:36.015777881Z tar: Removing leading `/' from member names
2026-06-02T20:16:36.015777881Z tar: Removing leading `/' from member names
2026-06-02T20:16:36.053039801Z tar: Removing leading `/' from member names
2026-06-02T20:16:36.515271264Z Wrote inspect data to must-gather.
2026-06-02T20:16:36.618488536Z INFO: Waiting for network log collection to complete ...
2026-06-02T20:16:36.618532644Z INFO: Waiting for ovnk database copies to complete ...
2026-06-02T20:16:36.619134738Z INFO: Copying ovnk databases complete.
2026-06-02T20:16:36.621186966Z 4.4M	must-gather/network_logs/ovnk_database_store
2026-06-02T20:16:36.623778355Z ovnk_database_store/
2026-06-02T20:16:36.623931312Z ovnk_database_store/ovnkube-node-79xzx_nbdb
2026-06-02T20:16:36.636693768Z ovnk_database_store/ovnkube-node-mhvx7_sbdb
2026-06-02T20:16:36.671373635Z ovnk_database_store/ovnkube-node-79xzx_sbdb
2026-06-02T20:16:36.703928990Z ovnk_database_store/ovnkube-node-xlrd9_sbdb
2026-06-02T20:16:36.733818897Z ovnk_database_store/ovnkube-node-xlrd9_nbdb
2026-06-02T20:16:36.756822888Z ovnk_database_store/ovnkube-node-mhvx7_nbdb
2026-06-02T20:16:36.769603579Z INFO: Network log collection complete.
2026-06-02T20:16:36.973646423Z W0602 20:16:36.973598    1391 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-02T20:16:36.999792986Z Wrote inspect data to must-gather.
2026-06-02T20:16:37.320859682Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-06-02T20:16:37.323474669Z Gathering data for ns/openshift-network-console...
2026-06-02T20:16:37.553289169Z Wrote inspect data to must-gather.
2026-06-02T20:16:37.924557889Z Gathering data for ns/openshift-console-operator...
2026-06-02T20:16:38.134093511Z Wrote inspect data to must-gather.
2026-06-02T20:16:38.450241407Z Gathering data for ns/openshift-console...
2026-06-02T20:16:38.746553561Z Wrote inspect data to must-gather.
2026-06-02T20:16:39.028923312Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-02T20:16:39.032460198Z INFO: Fallback to identify the container image from release info
2026-06-02T20:16:39.060738571Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-02T20:16:39.552217354Z Gathering data for ns/openshift-dns-operator...
2026-06-02T20:16:39.708805467Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d8381cb85e479760bd6b601e111b39e6c56a632bfa298867f4197ec4421e9f5
2026-06-02T20:16:39.838278036Z daemonset.apps/perf-node-gather-daemonset created
2026-06-02T20:16:39.989958892Z Gathering data for ns/openshift-dns...
2026-06-02T20:16:39.992857852Z Waiting for performance profile collector pods to become ready: 1
2026-06-02T20:16:40.083776811Z Wrote inspect data to must-gather.
2026-06-02T20:16:40.765210530Z Gathering data for ns/openshift-image-registry...
2026-06-02T20:16:41.106753221Z Waiting for performance profile collector pods to become ready: 2
2026-06-02T20:16:41.357216581Z Gathering data for ns/openshift-ingress-operator...
2026-06-02T20:16:41.789074178Z Gathering data for ns/openshift-ingress...
2026-06-02T20:16:42.245260290Z Waiting for performance profile collector pods to become ready: 3
2026-06-02T20:16:42.325508264Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-06-02T20:16:42.617469586Z Gathering data for ns/openshift-ingress-canary...
2026-06-02T20:16:43.131355451Z Gathering data for ns/openshift-insights...
2026-06-02T20:16:43.353491757Z Waiting for performance profile collector pods to become ready: 4
2026-06-02T20:16:44.471790399Z Waiting for performance profile collector pods to become ready: 5
2026-06-02T20:16:45.386755662Z Gathering data for ns/opendatahub...
2026-06-02T20:16:45.597534171Z Waiting for performance profile collector pods to become ready: 6
2026-06-02T20:16:46.714346570Z Waiting for performance profile collector pods to become ready: 7
2026-06-02T20:16:47.283839910Z Gathering data for ns/openshift-lws-operator...
2026-06-02T20:16:47.329962115Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-06-02T20:16:47.852113756Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-02T20:16:48.120903229Z Collecting performance related data for node ip-10-0-130-103.ec2.internal
2026-06-02T20:16:48.127772830Z Collecting performance related data for node ip-10-0-137-56.ec2.internal
2026-06-02T20:16:48.135834137Z Collecting performance related data for node ip-10-0-142-203.ec2.internal
2026-06-02T20:16:49.149381578Z Gathering data for ns/openshift-config...
2026-06-02T20:16:49.595061717Z Gathering data for ns/openshift-config-managed...
2026-06-02T20:16:50.171517694Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-02T20:16:50.527987943Z Gathering data for ns/openshift-kube-apiserver...
2026-06-02T20:16:50.896635093Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-02T20:16:51.398899176Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-02T20:16:51.790631407Z Gathering data for ns/openshift-kube-scheduler...
2026-06-02T20:16:52.201511833Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-02T20:16:52.335426016Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-06-02T20:16:52.614388205Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-02T20:16:53.052435900Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-02T20:16:53.456402200Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-02T20:16:54.096312372Z Gathering data for ns/openshift-multus...
2026-06-02T20:16:55.924520830Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-02T20:16:57.041780635Z Collecting kubelet logs for node ip-10-0-130-103.ec2.internal
2026-06-02T20:16:57.044000007Z Collecting kubelet logs for node ip-10-0-137-56.ec2.internal
2026-06-02T20:16:57.045927914Z Collecting kubelet logs for node ip-10-0-142-203.ec2.internal
2026-06-02T20:16:57.342745245Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-06-02T20:16:57.409881316Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-06-02T20:16:57.415013861Z INFO: Node performance data collection complete.
2026-06-02T20:16:57.783007142Z Gathering data for ns/openshift-host-network...
2026-06-02T20:16:58.158634050Z Gathering data for ns/openshift-network-diagnostics...
2026-06-02T20:16:58.778879319Z Gathering data for ns/openshift-network-node-identity...
2026-06-02T20:16:59.274021882Z Gathering data for ns/openshift-network-operator...
2026-06-02T20:16:59.729076495Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-02T20:17:00.073291826Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-02T20:17:00.604173570Z Gathering data for ns/openshift-apiserver-operator...
2026-06-02T20:17:01.003063558Z Gathering data for ns/openshift-apiserver...
2026-06-02T20:17:01.578745731Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-02T20:17:01.951939345Z Gathering data for ns/openshift-controller-manager...
2026-06-02T20:17:02.334966823Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-02T20:17:02.347653097Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-06-02T20:17:03.131096912Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-02T20:17:03.494153558Z Gathering data for ns/openshift-service-ca-operator...
2026-06-02T20:17:03.874365343Z Gathering data for ns/openshift-service-ca...
2026-06-02T20:17:04.300250070Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-02T20:17:05.508136340Z Wrote inspect data to must-gather.
2026-06-02T20:17:05.508180685Z error: inspection completed with the errors occurred while gathering data:
2026-06-02T20:17:05.508180685Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-02T20:17:05.640829125Z error: the server doesn't have a resource type "clusters"
2026-06-02T20:17:05.826367874Z Caches written to disk