2026-04-25T00:40:06.000159758Z [disk usage checker] Started
2026-04-25T00:40:06.004162596Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-25T00:40:06.215358407Z Gathering data for ns/openshift-cluster-version...
2026-04-25T00:40:06.356960156Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-25T00:40:06.441344106Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-25T00:40:06.526818589Z Gathering data for ns/default...
2026-04-25T00:40:06.818075292Z Gathering data for ns/openshift...
2026-04-25T00:40:07.240910081Z Gathering data for ns/kube-system...
2026-04-25T00:40:07.940001216Z Gathering data for ns/openshift-etcd...
2026-04-25T00:40:08.259354011Z Wrote inspect data to must-gather.
2026-04-25T00:40:08.259388681Z error: inspection completed with the errors occurred while gathering data:
2026-04-25T00:40:08.259388681Z     namespaces "assisted-installer" not found
2026-04-25T00:40:08.468161986Z Waiting on subprocesses to finish execution.
2026-04-25T00:40:08.472961122Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-25T00:40:08.482573895Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-25T00:40:08.493096774Z INFO: Gathering HAProxy config files
2026-04-25T00:40:08.493202321Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-25T00:40:08.520357735Z INFO: Waiting for node performance related collection to complete ...
2026-04-25T00:40:08.541785075Z INFO: Collecting host service logs for crio
2026-04-25T00:40:08.542198146Z INFO: Collecting host service logs for kubelet
2026-04-25T00:40:08.542586859Z INFO: Collecting host service logs for rpm-ostreed
2026-04-25T00:40:08.542934380Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-25T00:40:08.543281791Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-25T00:40:08.543638645Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-25T00:40:08.543975294Z INFO: Collecting host service logs for NetworkManager
2026-04-25T00:40:08.544341339Z INFO: Collecting host service logs for openvswitch
2026-04-25T00:40:08.544693850Z INFO: Collecting host service logs for ovs-configuration
2026-04-25T00:40:08.545032396Z INFO: Collecting host service logs for ovsdb-server
2026-04-25T00:40:08.545348294Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-25T00:40:08.545766697Z INFO: Waiting for worker host service log collection to complete ...
2026-04-25T00:40:09.095719999Z INFO: "metallb-operator" not detected. Skipping.
2026-04-25T00:40:09.387492233Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-25T00:40:09.387579047Z INFO: on-disk MachineConfig config collection complete.
2026-04-25T00:40:09.526372935Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-25T00:40:09.607784099Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-25T00:40:09.647222940Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-lrmcq
2026-04-25T00:40:09.647268002Z insights-runtime-extractor-jcrnq
2026-04-25T00:40:09.647287915Z insights-runtime-extractor-rrznm
2026-04-25T00:40:09.647307209Z insights-runtime-extractor-vv7gt
2026-04-25T00:40:09.814523531Z No resources found in openshift-etcd namespace.
2026-04-25T00:40:09.849311959Z No resources found
2026-04-25T00:40:09.917216025Z error: the server doesn't have a resource type "performanceprofile"
2026-04-25T00:40:09.941371214Z INFO: Worker host service log collection to complete.
2026-04-25T00:40:09.956722020Z Wrote inspect data to must-gather.
2026-04-25T00:40:09.981568347Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-25T00:40:09.985245042Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-25T00:40:10.203082070Z Wrote inspect data to must-gather.
2026-04-25T00:40:10.509759203Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-25T00:40:10.531257503Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-25T00:40:10.544051865Z ERROR: No running kube-apiserver pods found
2026-04-25T00:40:10.554042971Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-25T00:40:10.569288661Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-25T00:40:10.569288661Z See 'oc rsync -h' for help and examples
2026-04-25T00:40:10.674168362Z Wrote inspect data to must-gather.
2026-04-25T00:40:10.693103653Z No resources found
2026-04-25T00:40:10.775332718Z INFO: 'previous-logs' folder not found on ip-10-0-129-98.ec2.internal, skipping...
2026-04-25T00:40:10.824062565Z Wrote inspect data to must-gather.
2026-04-25T00:40:10.841637902Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-25T00:40:10.842185882Z Wrote inspect data to must-gather.
2026-04-25T00:40:10.940692001Z Gathering data for ns/openshift-monitoring...
2026-04-25T00:40:10.941804184Z Wrote inspect data to must-gather.
2026-04-25T00:40:10.949861936Z Wrote inspect data to must-gather.
2026-04-25T00:40:10.978728947Z tar: Removing leading `/' from member names
2026-04-25T00:40:10.983888524Z INFO: HAProxy config collection complete.
2026-04-25T00:40:11.010157691Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-25T00:40:11.115483493Z Wrote inspect data to must-gather.
2026-04-25T00:40:11.133638760Z error: the server doesn't have a resource type "machineconfigs"
2026-04-25T00:40:11.169149220Z INFO: 'previous-logs' folder not found on ip-10-0-135-201.ec2.internal, skipping...
2026-04-25T00:40:11.305754724Z INFO: Getting rules from prometheus-k8s-0
2026-04-25T00:40:11.379449967Z Wrote inspect data to must-gather.
2026-04-25T00:40:11.395813007Z Wrote inspect data to must-gather.
2026-04-25T00:40:11.460273625Z INFO: 'previous-logs' folder not found on ip-10-0-142-30.ec2.internal, skipping...
2026-04-25T00:40:11.460314533Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-25T00:40:11.460314533Z INFO: Machine Config Daemon termination log collection complete.
2026-04-25T00:40:11.616636887Z Wrote inspect data to must-gather.
2026-04-25T00:40:11.633526957Z Wrote inspect data to must-gather.
2026-04-25T00:40:11.747167267Z INFO: Getting status/config from prometheus-k8s-0
2026-04-25T00:40:11.783011189Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-25T00:40:11.937854111Z INFO: INTERCONNECT MODE
2026-04-25T00:40:11.937882470Z INFO: Gathering ovn-kubernetes DBs
2026-04-25T00:40:12.004577125Z Wrote inspect data to must-gather.
2026-04-25T00:40:12.107495920Z INFO: Gathering OVN_Northbound from ovnkube-node-7hz6p...
2026-04-25T00:40:12.107716954Z INFO: Gathering OVN_Northbound from ovnkube-node-t4f4q...
2026-04-25T00:40:12.107954889Z INFO: Gathering OVN_Northbound from ovnkube-node-v9qcn...
2026-04-25T00:40:12.108307569Z INFO: Gathering OVN_Southbound from ovnkube-node-7hz6p...
2026-04-25T00:40:12.108653061Z INFO: Gathering OVN_Southbound from ovnkube-node-t4f4q...
2026-04-25T00:40:12.108959412Z INFO: Gathering OVN_Southbound from ovnkube-node-v9qcn...
2026-04-25T00:40:12.298257163Z Wrote inspect data to must-gather.
2026-04-25T00:40:12.315695752Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-25T00:40:12.402059563Z tar: Removing leading `/' from member names
2026-04-25T00:40:12.415526401Z tar: Removing leading `/' from member names
2026-04-25T00:40:12.417067201Z tar: Removing leading `/' from member names
2026-04-25T00:40:12.417203539Z tar: Removing leading `/' from member names
2026-04-25T00:40:12.428197634Z tar: Removing leading `/' from member names
2026-04-25T00:40:12.492082439Z tar: Removing leading `/' from member names
2026-04-25T00:40:12.540582522Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-25T00:40:12.576153569Z Wrote inspect data to must-gather.
2026-04-25T00:40:12.619624131Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-25T00:40:12.837875301Z Wrote inspect data to must-gather.
2026-04-25T00:40:12.883456491Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-25T00:40:12.929766953Z INFO: Waiting for network log collection to complete ...
2026-04-25T00:40:12.929800357Z INFO: Waiting for ovnk database copies to complete ...
2026-04-25T00:40:12.930176042Z INFO: Copying ovnk databases complete.
2026-04-25T00:40:12.931884378Z 3.9M	must-gather/network_logs/ovnk_database_store
2026-04-25T00:40:12.934132107Z ovnk_database_store/
2026-04-25T00:40:12.934132107Z ovnk_database_store/ovnkube-node-7hz6p_sbdb
2026-04-25T00:40:12.957473493Z ovnk_database_store/ovnkube-node-t4f4q_sbdb
2026-04-25T00:40:12.984910818Z ovnk_database_store/ovnkube-node-v9qcn_sbdb
2026-04-25T00:40:13.007977677Z ovnk_database_store/ovnkube-node-7hz6p_nbdb
2026-04-25T00:40:13.019580384Z ovnk_database_store/ovnkube-node-t4f4q_nbdb
2026-04-25T00:40:13.030364802Z ovnk_database_store/ovnkube-node-v9qcn_nbdb
2026-04-25T00:40:13.042118815Z INFO: Network log collection complete.
2026-04-25T00:40:13.058013929Z W0425 00:40:13.057968    1343 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-04-25T00:40:13.062740576Z Wrote inspect data to must-gather.
2026-04-25T00:40:13.151912079Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-25T00:40:13.158106639Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-25T00:40:13.328377672Z Gathering data for ns/openshift-network-console...
2026-04-25T00:40:13.353251051Z Wrote inspect data to must-gather.
2026-04-25T00:40:13.584534186Z INFO: Getting status from alertmanager-main-0
2026-04-25T00:40:13.667890063Z Wrote inspect data to must-gather.
2026-04-25T00:40:13.760000362Z Gathering data for ns/openshift-console-operator...
2026-04-25T00:40:13.953728048Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-25T00:40:13.956118099Z INFO: Fallback to identify the container image from release info
2026-04-25T00:40:14.129812033Z Gathering data for ns/openshift-console...
2026-04-25T00:40:14.529163842Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-25T00:40:14.532110183Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-25T00:40:14.665757595Z daemonset.apps/perf-node-gather-daemonset created
2026-04-25T00:40:14.796460398Z Waiting for performance profile collector pods to become ready: 1
2026-04-25T00:40:14.905496618Z Gathering data for ns/openshift-dns-operator...
2026-04-25T00:40:15.198887954Z Gathering data for ns/openshift-dns...
2026-04-25T00:40:15.738571310Z Gathering data for ns/openshift-image-registry...
2026-04-25T00:40:15.926564216Z Waiting for performance profile collector pods to become ready: 2
2026-04-25T00:40:16.014936257Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-25T00:40:16.221895751Z Gathering data for ns/openshift-ingress-operator...
2026-04-25T00:40:16.558504217Z Gathering data for ns/openshift-ingress...
2026-04-25T00:40:16.867486243Z Gathering data for ns/openshift-ingress-canary...
2026-04-25T00:40:17.059048698Z Waiting for performance profile collector pods to become ready: 3
2026-04-25T00:40:17.242882150Z Gathering data for ns/openshift-insights...
2026-04-25T00:40:18.177721762Z Waiting for performance profile collector pods to become ready: 4
2026-04-25T00:40:19.297041323Z Gathering data for ns/kserve...
2026-04-25T00:40:19.299710872Z Waiting for performance profile collector pods to become ready: 5
2026-04-25T00:40:20.456676694Z Waiting for performance profile collector pods to become ready: 6
2026-04-25T00:40:20.868343207Z Gathering data for ns/openshift-config...
2026-04-25T00:40:21.019693808Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-25T00:40:21.157930916Z Gathering data for ns/openshift-config-managed...
2026-04-25T00:40:21.580843325Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-25T00:40:21.603743275Z Waiting for performance profile collector pods to become ready: 7
2026-04-25T00:40:21.839907622Z Gathering data for ns/openshift-kube-apiserver...
2026-04-25T00:40:22.124129316Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-25T00:40:22.378541808Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-25T00:40:22.712550689Z Gathering data for ns/openshift-kube-scheduler...
2026-04-25T00:40:22.754464159Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-25T00:40:22.980232307Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-25T00:40:23.106594448Z Collecting performance related data for node ip-10-0-142-30.ec2.internal
2026-04-25T00:40:23.113663017Z Collecting performance related data for node ip-10-0-129-98.ec2.internal
2026-04-25T00:40:23.121333452Z Collecting performance related data for node ip-10-0-135-201.ec2.internal
2026-04-25T00:40:23.348955634Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-25T00:40:23.733678078Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-25T00:40:24.095727009Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-25T00:40:24.806003539Z Gathering data for ns/openshift-multus...
2026-04-25T00:40:26.024751286Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-25T00:40:26.255667780Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-25T00:40:27.731885643Z Gathering data for ns/openshift-host-network...
2026-04-25T00:40:28.069266224Z Gathering data for ns/openshift-network-diagnostics...
2026-04-25T00:40:28.604004256Z Gathering data for ns/openshift-network-node-identity...
2026-04-25T00:40:29.033797525Z Gathering data for ns/openshift-network-operator...
2026-04-25T00:40:29.387453065Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-25T00:40:29.625380295Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-25T00:40:30.055954695Z Gathering data for ns/openshift-apiserver-operator...
2026-04-25T00:40:30.350730577Z Gathering data for ns/openshift-apiserver...
2026-04-25T00:40:30.765964281Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-25T00:40:31.027688960Z Gathering data for ns/openshift-controller-manager...
2026-04-25T00:40:31.030920865Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-25T00:40:31.097393239Z Collecting kubelet logs for node ip-10-0-142-30.ec2.internal
2026-04-25T00:40:31.099396393Z Collecting kubelet logs for node ip-10-0-129-98.ec2.internal
2026-04-25T00:40:31.101308195Z Collecting kubelet logs for node ip-10-0-135-201.ec2.internal
2026-04-25T00:40:31.350189407Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-25T00:40:31.520086830Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-25T00:40:31.525171728Z INFO: Node performance data collection complete.
2026-04-25T00:40:32.011962991Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-25T00:40:32.278548821Z Gathering data for ns/openshift-service-ca-operator...
2026-04-25T00:40:32.584625796Z Gathering data for ns/openshift-service-ca...
2026-04-25T00:40:32.919541221Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-25T00:40:33.778281427Z Wrote inspect data to must-gather.
2026-04-25T00:40:33.778505982Z error: inspection completed with the errors occurred while gathering data:
2026-04-25T00:40:33.778505982Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-25T00:40:33.911746813Z error: the server doesn't have a resource type "clusters"
2026-04-25T00:40:34.223030822Z Caches written to disk