2026-06-03T17:06:22.615687029Z [disk usage checker] Started
2026-06-03T17:06:22.619867530Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-06-03T17:06:22.845238902Z Gathering data for ns/openshift-cluster-version...
2026-06-03T17:06:23.083672786Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-03T17:06:23.193843940Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-03T17:06:23.307334715Z Gathering data for ns/default...
2026-06-03T17:06:23.821521347Z Gathering data for ns/openshift...
2026-06-03T17:06:24.311762951Z Gathering data for ns/kube-system...
2026-06-03T17:06:25.061025748Z Waiting on subprocesses to finish execution.
2026-06-03T17:06:25.066986580Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-03T17:06:25.071482453Z INFO: Collecting host service logs for crio
2026-06-03T17:06:25.071810295Z INFO: Collecting host service logs for kubelet
2026-06-03T17:06:25.072161579Z INFO: Collecting host service logs for rpm-ostreed
2026-06-03T17:06:25.072507488Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-03T17:06:25.072851408Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-03T17:06:25.073175037Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-03T17:06:25.073533210Z INFO: Collecting host service logs for NetworkManager
2026-06-03T17:06:25.073882873Z INFO: Collecting host service logs for openvswitch
2026-06-03T17:06:25.074212644Z INFO: Collecting host service logs for ovs-configuration
2026-06-03T17:06:25.074608419Z INFO: Collecting host service logs for ovsdb-server
2026-06-03T17:06:25.075137184Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-03T17:06:25.075516144Z INFO: Waiting for worker host service log collection to complete ...
2026-06-03T17:06:25.085673713Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-03T17:06:25.105828429Z INFO: Gathering HAProxy config files
2026-06-03T17:06:25.109593645Z Gathering data for ns/openshift-etcd...
2026-06-03T17:06:25.119870274Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-03T17:06:25.500176124Z INFO: Waiting for node performance related collection to complete ...
2026-06-03T17:06:25.966937433Z INFO: "metallb-operator" not detected. Skipping.
2026-06-03T17:06:25.967494759Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-03T17:06:26.000194361Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-03T17:06:26.048351409Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-03T17:06:26.048434417Z INFO: on-disk MachineConfig config collection complete.
2026-06-03T17:06:26.106314151Z No resources found in openshift-etcd namespace.
2026-06-03T17:06:26.188939033Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-03T17:06:26.229769603Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-03T17:06:26.450326195Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-03T17:06:26.613034678Z No resources found
2026-06-03T17:06:26.636145234Z INFO: Collecting Insights Archives from insights-operator-788bf58df5-5n57j
2026-06-03T17:06:26.636195691Z insights-runtime-extractor-9n2sx
2026-06-03T17:06:26.636216263Z insights-runtime-extractor-9tlxs
2026-06-03T17:06:26.636235788Z insights-runtime-extractor-cqnsv
2026-06-03T17:06:26.808761105Z error: the server doesn't have a resource type "performanceprofile"
2026-06-03T17:06:26.952817493Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-03T17:06:26.968430970Z tar: Removing leading `/' from member names
2026-06-03T17:06:26.974520284Z INFO: HAProxy config collection complete.
2026-06-03T17:06:27.092930884Z INFO: Worker host service log collection to complete.
2026-06-03T17:06:27.353213098Z Wrote inspect data to must-gather.
2026-06-03T17:06:27.353310244Z error: inspection completed with the errors occurred while gathering data:
2026-06-03T17:06:27.353310244Z     namespaces "assisted-installer" not found
2026-06-03T17:06:27.363369464Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-03T17:06:27.575252202Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-03T17:06:27.838139734Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-06-03T17:06:27.873681648Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-03T17:06:27.873681648Z See 'oc rsync -h' for help and examples
2026-06-03T17:06:27.958278249Z Wrote inspect data to must-gather.
2026-06-03T17:06:27.970688578Z Wrote inspect data to must-gather.
2026-06-03T17:06:27.986191858Z Wrote inspect data to must-gather.
2026-06-03T17:06:27.987382046Z Wrote inspect data to must-gather.
2026-06-03T17:06:28.003634234Z ERROR: No running kube-apiserver pods found
2026-06-03T17:06:28.022993381Z No resources found
2026-06-03T17:06:28.100077822Z Wrote inspect data to must-gather.
2026-06-03T17:06:28.137199394Z INFO: 'previous-logs' folder not found on ip-10-0-136-251.ec2.internal, skipping...
2026-06-03T17:06:28.242960016Z Wrote inspect data to must-gather.
2026-06-03T17:06:28.376492867Z Gathering data for ns/kuadrant-system...
2026-06-03T17:06:28.457516829Z Wrote inspect data to must-gather.
2026-06-03T17:06:28.463570045Z error: the server doesn't have a resource type "machineconfigs"
2026-06-03T17:06:28.486863023Z INFO: 'previous-logs' folder not found on ip-10-0-141-18.ec2.internal, skipping...
2026-06-03T17:06:28.501841914Z INFO: Getting rules from prometheus-k8s-0
2026-06-03T17:06:28.790092694Z INFO: 'previous-logs' folder not found on ip-10-0-143-30.ec2.internal, skipping...
2026-06-03T17:06:28.790125924Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-03T17:06:28.790136688Z INFO: Machine Config Daemon termination log collection complete.
2026-06-03T17:06:28.851928661Z Wrote inspect data to must-gather.
2026-06-03T17:06:28.924665687Z Wrote inspect data to must-gather.
2026-06-03T17:06:28.990501203Z INFO: Getting status/config from prometheus-k8s-0
2026-06-03T17:06:29.282937884Z Wrote inspect data to must-gather.
2026-06-03T17:06:29.384235154Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-03T17:06:29.402303828Z Wrote inspect data to must-gather.
2026-06-03T17:06:29.486476319Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-03T17:06:29.494564087Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-03T17:06:29.780625323Z INFO: INTERCONNECT MODE
2026-06-03T17:06:29.780654418Z INFO: Gathering ovn-kubernetes DBs
2026-06-03T17:06:29.784126776Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-03T17:06:29.819047827Z W0603 17:06:29.819000    1133 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-03T17:06:29.850622967Z Wrote inspect data to must-gather.
2026-06-03T17:06:29.973971208Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-03T17:06:29.987670281Z INFO: Gathering OVN_Northbound from ovnkube-node-27p8j...
2026-06-03T17:06:29.988029293Z INFO: Gathering OVN_Northbound from ovnkube-node-lp8cj...
2026-06-03T17:06:29.988350268Z INFO: Gathering OVN_Northbound from ovnkube-node-qxv8r...
2026-06-03T17:06:29.988693840Z INFO: Gathering OVN_Southbound from ovnkube-node-27p8j...
2026-06-03T17:06:29.989013827Z INFO: Gathering OVN_Southbound from ovnkube-node-lp8cj...
2026-06-03T17:06:29.989289211Z INFO: Gathering OVN_Southbound from ovnkube-node-qxv8r...
2026-06-03T17:06:30.101330626Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-03T17:06:30.296580308Z tar: Removing leading `/' from member names
2026-06-03T17:06:30.378303950Z tar: Removing leading `/' from member names
2026-06-03T17:06:30.378889206Z tar: Removing leading `/' from member names
2026-06-03T17:06:30.415059169Z tar: Removing leading `/' from member names
2026-06-03T17:06:30.428872923Z tar: Removing leading `/' from member names
2026-06-03T17:06:30.502892010Z Gathering data for ns/openshift-monitoring...
2026-06-03T17:06:30.522094644Z Wrote inspect data to must-gather.
2026-06-03T17:06:30.539106755Z tar: Removing leading `/' from member names
2026-06-03T17:06:30.563480191Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-03T17:06:30.944873382Z Wrote inspect data to must-gather.
2026-06-03T17:06:31.050143073Z INFO: Waiting for network log collection to complete ...
2026-06-03T17:06:31.050197092Z INFO: Waiting for ovnk database copies to complete ...
2026-06-03T17:06:31.050681953Z INFO: Copying ovnk databases complete.
2026-06-03T17:06:31.052497373Z 4.2M	must-gather/network_logs/ovnk_database_store
2026-06-03T17:06:31.054624796Z ovnk_database_store/
2026-06-03T17:06:31.054624796Z ovnk_database_store/ovnkube-node-27p8j_nbdb
2026-06-03T17:06:31.069750412Z ovnk_database_store/ovnkube-node-lp8cj_sbdb
2026-06-03T17:06:31.075925064Z INFO: Getting status from alertmanager-main-0
2026-06-03T17:06:31.100575766Z ovnk_database_store/ovnkube-node-qxv8r_sbdb
2026-06-03T17:06:31.123497660Z ovnk_database_store/ovnkube-node-lp8cj_nbdb
2026-06-03T17:06:31.136757224Z ovnk_database_store/ovnkube-node-qxv8r_nbdb
2026-06-03T17:06:31.149295000Z ovnk_database_store/ovnkube-node-27p8j_sbdb
2026-06-03T17:06:31.180754358Z INFO: Network log collection complete.
2026-06-03T17:06:31.354560384Z Wrote inspect data to must-gather.
2026-06-03T17:06:31.674961771Z Wrote inspect data to must-gather.
2026-06-03T17:06:32.070966074Z Wrote inspect data to must-gather.
2026-06-03T17:06:32.411192149Z Wrote inspect data to must-gather.
2026-06-03T17:06:32.697062286Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-03T17:06:32.699682809Z INFO: Fallback to identify the container image from release info
2026-06-03T17:06:32.807637500Z Gathering data for ns/openshift-network-console...
2026-06-03T17:06:32.867816064Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-06-03T17:06:33.380228099Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d8381cb85e479760bd6b601e111b39e6c56a632bfa298867f4197ec4421e9f5
2026-06-03T17:06:33.404680060Z Gathering data for ns/openshift-console-operator...
2026-06-03T17:06:33.514967246Z daemonset.apps/perf-node-gather-daemonset created
2026-06-03T17:06:33.654777903Z Waiting for performance profile collector pods to become ready: 1
2026-06-03T17:06:33.981412617Z Gathering data for ns/openshift-console...
2026-06-03T17:06:34.587284359Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-03T17:06:34.656999020Z Wrote inspect data to must-gather.
2026-06-03T17:06:34.782973804Z Waiting for performance profile collector pods to become ready: 2
2026-06-03T17:06:35.000286386Z Gathering data for ns/openshift-dns-operator...
2026-06-03T17:06:35.375758457Z Gathering data for ns/openshift-dns...
2026-06-03T17:06:35.914676424Z Waiting for performance profile collector pods to become ready: 3
2026-06-03T17:06:36.043710975Z Gathering data for ns/openshift-image-registry...
2026-06-03T17:06:36.596398981Z Gathering data for ns/openshift-ingress-operator...
2026-06-03T17:06:37.032271876Z Gathering data for ns/openshift-ingress...
2026-06-03T17:06:37.042974255Z Waiting for performance profile collector pods to become ready: 4
2026-06-03T17:06:37.746670877Z Gathering data for ns/openshift-ingress-canary...
2026-06-03T17:06:37.875058604Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-06-03T17:06:38.174235621Z Waiting for performance profile collector pods to become ready: 5
2026-06-03T17:06:38.267434322Z Gathering data for ns/openshift-insights...
2026-06-03T17:06:39.290123161Z Waiting for performance profile collector pods to become ready: 6
2026-06-03T17:06:40.424765921Z Waiting for performance profile collector pods to become ready: 7
2026-06-03T17:06:40.659920064Z Gathering data for ns/opendatahub...
2026-06-03T17:06:41.552377797Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-03T17:06:41.822040065Z Collecting performance related data for node ip-10-0-143-30.ec2.internal
2026-06-03T17:06:41.829195300Z Collecting performance related data for node ip-10-0-136-251.ec2.internal
2026-06-03T17:06:41.836537801Z Collecting performance related data for node ip-10-0-141-18.ec2.internal
2026-06-03T17:06:42.612010151Z Gathering data for ns/openshift-lws-operator...
2026-06-03T17:06:42.879983761Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-06-03T17:06:44.471175846Z Gathering data for ns/openshift-config...
2026-06-03T17:06:44.904224658Z Gathering data for ns/openshift-config-managed...
2026-06-03T17:06:45.493785371Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-03T17:06:45.877847587Z Gathering data for ns/openshift-kube-apiserver...
2026-06-03T17:06:46.236488766Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-03T17:06:46.587043392Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-03T17:06:46.957370762Z Gathering data for ns/openshift-kube-scheduler...
2026-06-03T17:06:47.327081835Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-03T17:06:47.858058496Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-03T17:06:47.884717815Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-06-03T17:06:48.346898842Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-03T17:06:48.722027260Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-03T17:06:49.441183476Z Gathering data for ns/openshift-multus...
2026-06-03T17:06:50.037269702Z Collecting kubelet logs for node ip-10-0-143-30.ec2.internal
2026-06-03T17:06:50.039001258Z Collecting kubelet logs for node ip-10-0-136-251.ec2.internal
2026-06-03T17:06:50.040781862Z Collecting kubelet logs for node ip-10-0-141-18.ec2.internal
2026-06-03T17:06:50.391872002Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-06-03T17:06:50.397940241Z INFO: Node performance data collection complete.
2026-06-03T17:06:51.061339114Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-03T17:06:52.890718084Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-06-03T17:06:52.893222198Z Gathering data for ns/openshift-host-network...
2026-06-03T17:06:53.281490242Z Gathering data for ns/openshift-network-diagnostics...
2026-06-03T17:06:53.892465557Z Gathering data for ns/openshift-network-node-identity...
2026-06-03T17:06:54.387099643Z Gathering data for ns/openshift-network-operator...
2026-06-03T17:06:54.818931114Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-03T17:06:55.167716464Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-03T17:06:55.705763036Z Gathering data for ns/openshift-apiserver-operator...
2026-06-03T17:06:56.040483696Z Gathering data for ns/openshift-apiserver...
2026-06-03T17:06:56.514228141Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-03T17:06:56.823195584Z Gathering data for ns/openshift-controller-manager...
2026-06-03T17:06:57.188066538Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-03T17:06:57.873936948Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-03T17:06:57.898063602Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-06-03T17:06:58.229499367Z Gathering data for ns/openshift-service-ca-operator...
2026-06-03T17:06:58.594090505Z Gathering data for ns/openshift-service-ca...
2026-06-03T17:06:59.015875102Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-03T17:07:00.068312986Z Wrote inspect data to must-gather.
2026-06-03T17:07:00.068412823Z error: inspection completed with the errors occurred while gathering data:
2026-06-03T17:07:00.068412823Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-03T17:07:00.208662583Z error: the server doesn't have a resource type "clusters"
2026-06-03T17:07:00.532404900Z Caches written to disk