2026-04-16T19:24:25.604117820Z [disk usage checker] Started
2026-04-16T19:24:25.637230652Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T19:24:25.836581806Z Gathering data for ns/openshift-cluster-version...
2026-04-16T19:24:25.996254565Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T19:24:26.101190240Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T19:24:26.193736361Z Gathering data for ns/default...
2026-04-16T19:24:26.530318212Z Gathering data for ns/openshift...
2026-04-16T19:24:26.970291690Z Gathering data for ns/kube-system...
2026-04-16T19:24:27.783774585Z Gathering data for ns/openshift-etcd...
2026-04-16T19:24:28.127817570Z Waiting on subprocesses to finish execution.
2026-04-16T19:24:28.132065372Z INFO: Gathering HAProxy config files
2026-04-16T19:24:28.142965798Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-16T19:24:28.145863029Z INFO: Collecting host service logs for crio
2026-04-16T19:24:28.146223420Z INFO: Collecting host service logs for kubelet
2026-04-16T19:24:28.146521167Z INFO: Collecting host service logs for rpm-ostreed
2026-04-16T19:24:28.146829246Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-16T19:24:28.147164782Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-16T19:24:28.147473607Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-16T19:24:28.147790160Z INFO: Collecting host service logs for NetworkManager
2026-04-16T19:24:28.148113996Z INFO: Collecting host service logs for openvswitch
2026-04-16T19:24:28.148445532Z INFO: Collecting host service logs for ovs-configuration
2026-04-16T19:24:28.148770676Z INFO: Collecting host service logs for ovsdb-server
2026-04-16T19:24:28.149164901Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-16T19:24:28.149589029Z INFO: Waiting for worker host service log collection to complete ...
2026-04-16T19:24:28.152307376Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-16T19:24:28.157824531Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-16T19:24:28.163373813Z Wrote inspect data to must-gather.
2026-04-16T19:24:28.163462716Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T19:24:28.163462716Z     namespaces "assisted-installer" not found
2026-04-16T19:24:28.200017331Z INFO: Waiting for node performance related collection to complete ...
2026-04-16T19:24:28.597652371Z INFO: Collecting Insights Archives from insights-operator-5785d4fcdd-xmqgp
2026-04-16T19:24:28.597701966Z insights-runtime-extractor-dz7b9
2026-04-16T19:24:28.597701966Z insights-runtime-extractor-qx2kf
2026-04-16T19:24:28.597701966Z insights-runtime-extractor-rhp8h
2026-04-16T19:24:28.779251375Z No resources found in openshift-etcd namespace.
2026-04-16T19:24:28.804027053Z Wrote inspect data to must-gather.
2026-04-16T19:24:28.910497697Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-16T19:24:29.223948635Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-16T19:24:29.223948635Z See 'oc rsync -h' for help and examples
2026-04-16T19:24:29.522723392Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-16T19:24:29.541909564Z No resources found
2026-04-16T19:24:29.663616567Z INFO: "metallb-operator" not detected. Skipping.
2026-04-16T19:24:29.668236533Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-16T19:24:29.815760824Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-16T19:24:29.815855401Z INFO: on-disk MachineConfig config collection complete.
2026-04-16T19:24:29.857429394Z Gathering data for ns/openshift-monitoring...
2026-04-16T19:24:29.987294204Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-16T19:24:29.995717866Z No resources found
2026-04-16T19:24:30.046323819Z ERROR: No running kube-apiserver pods found
2026-04-16T19:24:30.122982308Z Wrote inspect data to must-gather.
2026-04-16T19:24:30.140765719Z Wrote inspect data to must-gather.
2026-04-16T19:24:30.196328756Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-16T19:24:30.228508005Z Wrote inspect data to must-gather.
2026-04-16T19:24:30.288376255Z error: the server doesn't have a resource type "performanceprofile"
2026-04-16T19:24:30.292842049Z Wrote inspect data to must-gather.
2026-04-16T19:24:30.296890142Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-16T19:24:30.312607238Z Wrote inspect data to must-gather.
2026-04-16T19:24:30.319650795Z INFO: Worker host service log collection to complete.
2026-04-16T19:24:30.327972371Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T19:24:30.356467774Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-16T19:24:30.480425009Z INFO: 'previous-logs' folder not found on ip-10-0-136-226.ec2.internal, skipping...
2026-04-16T19:24:30.591957287Z Wrote inspect data to must-gather.
2026-04-16T19:24:30.592356302Z tar: Removing leading `/' from member names
2026-04-16T19:24:30.593309262Z Wrote inspect data to must-gather.
2026-04-16T19:24:30.597049233Z INFO: HAProxy config collection complete.
2026-04-16T19:24:30.645342812Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T19:24:30.755049634Z INFO: 'previous-logs' folder not found on ip-10-0-138-175.ec2.internal, skipping...
2026-04-16T19:24:30.767306965Z error: the server doesn't have a resource type "machineconfigs"
2026-04-16T19:24:30.768248346Z INFO: Getting rules from prometheus-k8s-0
2026-04-16T19:24:30.884017453Z Wrote inspect data to must-gather.
2026-04-16T19:24:30.999721702Z Wrote inspect data to must-gather.
2026-04-16T19:24:31.021141781Z INFO: 'previous-logs' folder not found on ip-10-0-139-117.ec2.internal, skipping...
2026-04-16T19:24:31.021179415Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-16T19:24:31.021179415Z INFO: Machine Config Daemon termination log collection complete.
2026-04-16T19:24:31.140145507Z Wrote inspect data to must-gather.
2026-04-16T19:24:31.198985463Z INFO: Getting status/config from prometheus-k8s-0
2026-04-16T19:24:31.233978650Z Wrote inspect data to must-gather.
2026-04-16T19:24:31.393164410Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-16T19:24:31.429844590Z INFO: INTERCONNECT MODE
2026-04-16T19:24:31.429944930Z INFO: Gathering ovn-kubernetes DBs
2026-04-16T19:24:31.606918440Z INFO: Gathering OVN_Northbound from ovnkube-node-2frx4...
2026-04-16T19:24:31.607170560Z INFO: Gathering OVN_Northbound from ovnkube-node-8vpdh...
2026-04-16T19:24:31.607362746Z INFO: Gathering OVN_Northbound from ovnkube-node-kxwr5...
2026-04-16T19:24:31.607598482Z INFO: Gathering OVN_Southbound from ovnkube-node-2frx4...
2026-04-16T19:24:31.607859543Z INFO: Gathering OVN_Southbound from ovnkube-node-8vpdh...
2026-04-16T19:24:31.608229466Z INFO: Gathering OVN_Southbound from ovnkube-node-kxwr5...
2026-04-16T19:24:31.632169542Z Wrote inspect data to must-gather.
2026-04-16T19:24:31.634856165Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-16T19:24:31.935277427Z tar: Removing leading `/' from member names
2026-04-16T19:24:31.944545378Z tar: Removing leading `/' from member names
2026-04-16T19:24:31.953146209Z tar: Removing leading `/' from member names
2026-04-16T19:24:31.953323663Z tar: Removing leading `/' from member names
2026-04-16T19:24:31.970629203Z tar: Removing leading `/' from member names
2026-04-16T19:24:31.989544238Z tar: Removing leading `/' from member names
2026-04-16T19:24:32.046517977Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-16T19:24:32.049328867Z Wrote inspect data to must-gather.
2026-04-16T19:24:32.068070474Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T19:24:32.294185175Z Wrote inspect data to must-gather.
2026-04-16T19:24:32.303664609Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-16T19:24:32.461251946Z INFO: Waiting for network log collection to complete ...
2026-04-16T19:24:32.461292754Z INFO: Waiting for ovnk database copies to complete ...
2026-04-16T19:24:32.461876085Z INFO: Copying ovnk databases complete.
2026-04-16T19:24:32.463625194Z 4.3M	must-gather/network_logs/ovnk_database_store
2026-04-16T19:24:32.465860402Z ovnk_database_store/
2026-04-16T19:24:32.465912532Z ovnk_database_store/ovnkube-node-8vpdh_sbdb
2026-04-16T19:24:32.492145759Z ovnk_database_store/ovnkube-node-8vpdh_nbdb
2026-04-16T19:24:32.505365324Z ovnk_database_store/ovnkube-node-2frx4_sbdb
2026-04-16T19:24:32.529519001Z ovnk_database_store/ovnkube-node-kxwr5_sbdb
2026-04-16T19:24:32.540963251Z Wrote inspect data to must-gather.
2026-04-16T19:24:32.554289664Z ovnk_database_store/ovnkube-node-kxwr5_nbdb
2026-04-16T19:24:32.563189581Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-16T19:24:32.567244892Z ovnk_database_store/ovnkube-node-2frx4_nbdb
2026-04-16T19:24:32.580637746Z INFO: Network log collection complete.
2026-04-16T19:24:32.692493363Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T19:24:32.833333850Z Wrote inspect data to must-gather.
2026-04-16T19:24:32.858447252Z Gathering data for ns/openshift-network-console...
2026-04-16T19:24:32.981582794Z INFO: Getting status from alertmanager-main-0
2026-04-16T19:24:33.123821794Z Wrote inspect data to must-gather.
2026-04-16T19:24:33.348186690Z Gathering data for ns/openshift-console-operator...
2026-04-16T19:24:33.410014262Z Wrote inspect data to must-gather.
2026-04-16T19:24:33.678420056Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-16T19:24:33.682072623Z INFO: Fallback to identify the container image from release info
2026-04-16T19:24:33.747557107Z Gathering data for ns/openshift-console...
2026-04-16T19:24:34.217975162Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-16T19:24:34.267372822Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc77fc5f1546189d6b57967ccc684e794bd846ea36c3e3c1e61d83a684f0f9b9
2026-04-16T19:24:34.406342009Z daemonset.apps/perf-node-gather-daemonset created
2026-04-16T19:24:34.540639167Z Waiting for performance profile collector pods to become ready: 1
2026-04-16T19:24:34.662331936Z Gathering data for ns/openshift-dns-operator...
2026-04-16T19:24:35.005206208Z Gathering data for ns/openshift-dns...
2026-04-16T19:24:35.654559755Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T19:24:35.685367623Z Waiting for performance profile collector pods to become ready: 2
2026-04-16T19:24:35.719784480Z Gathering data for ns/openshift-image-registry...
2026-04-16T19:24:36.263580284Z Gathering data for ns/openshift-ingress-operator...
2026-04-16T19:24:36.696752777Z Gathering data for ns/openshift-ingress...
2026-04-16T19:24:36.824952966Z Waiting for performance profile collector pods to become ready: 3
2026-04-16T19:24:37.087121793Z Gathering data for ns/openshift-ingress-canary...
2026-04-16T19:24:37.536345044Z Gathering data for ns/openshift-insights...
2026-04-16T19:24:37.963851543Z Waiting for performance profile collector pods to become ready: 4
2026-04-16T19:24:39.103266331Z Waiting for performance profile collector pods to become ready: 5
2026-04-16T19:24:39.887392175Z Gathering data for ns/kserve...
2026-04-16T19:24:40.223970763Z Waiting for performance profile collector pods to become ready: 6
2026-04-16T19:24:40.661746632Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T19:24:41.371471915Z Waiting for performance profile collector pods to become ready: 7
2026-04-16T19:24:41.962594423Z Gathering data for ns/openshift-config...
2026-04-16T19:24:42.393059550Z Gathering data for ns/openshift-config-managed...
2026-04-16T19:24:42.510409680Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-16T19:24:42.797970831Z Collecting performance related data for node ip-10-0-136-226.ec2.internal
2026-04-16T19:24:42.805914464Z Collecting performance related data for node ip-10-0-138-175.ec2.internal
2026-04-16T19:24:42.817436728Z Collecting performance related data for node ip-10-0-139-117.ec2.internal
2026-04-16T19:24:42.966994857Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-16T19:24:43.331117299Z Gathering data for ns/openshift-kube-apiserver...
2026-04-16T19:24:43.714232825Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-16T19:24:44.159665467Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-16T19:24:44.644112969Z Gathering data for ns/openshift-kube-scheduler...
2026-04-16T19:24:45.003968425Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-16T19:24:45.342765278Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-16T19:24:45.670271648Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T19:24:45.773846709Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-16T19:24:46.181510074Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-16T19:24:46.773768665Z Gathering data for ns/openshift-multus...
2026-04-16T19:24:48.464773621Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-16T19:24:50.445457822Z Gathering data for ns/openshift-host-network...
2026-04-16T19:24:50.678065075Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T19:24:50.842231233Z Gathering data for ns/openshift-network-diagnostics...
2026-04-16T19:24:51.384961441Z Collecting kubelet logs for node ip-10-0-136-226.ec2.internal
2026-04-16T19:24:51.387617289Z Collecting kubelet logs for node ip-10-0-138-175.ec2.internal
2026-04-16T19:24:51.389867246Z Collecting kubelet logs for node ip-10-0-139-117.ec2.internal
2026-04-16T19:24:51.466558469Z Gathering data for ns/openshift-network-node-identity...
2026-04-16T19:24:51.814044830Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-16T19:24:51.821742498Z INFO: Node performance data collection complete.
2026-04-16T19:24:51.985934408Z Gathering data for ns/openshift-network-operator...
2026-04-16T19:24:52.437110483Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-16T19:24:52.744827522Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-16T19:24:53.248257653Z Gathering data for ns/openshift-apiserver-operator...
2026-04-16T19:24:53.582994539Z Gathering data for ns/openshift-apiserver...
2026-04-16T19:24:54.013382765Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-16T19:24:54.347847819Z Gathering data for ns/openshift-controller-manager...
2026-04-16T19:24:54.697695871Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-16T19:24:55.395238805Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-16T19:24:55.687221972Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T19:24:55.713853399Z Gathering data for ns/openshift-service-ca-operator...
2026-04-16T19:24:56.056576006Z Gathering data for ns/openshift-service-ca...
2026-04-16T19:24:56.463590344Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-16T19:24:57.455533914Z Wrote inspect data to must-gather.
2026-04-16T19:24:57.455588093Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T19:24:57.455588093Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-16T19:24:57.589935462Z error: the server doesn't have a resource type "clusters"
2026-04-16T19:24:57.740439571Z Caches written to disk