[must-gather ] OUT 2026-04-21T15:24:46.706754772Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 6ad81a53-3b46-46e7-8c0a-fd60d6a42aa0 ClientVersion: 4.21.4 ClusterVersion: Stable at "4.20.19" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing [must-gather ] OUT 2026-04-21T15:24:46.755959399Z namespace/openshift-must-gather-5v6kh created [must-gather ] OUT 2026-04-21T15:24:46.760509381Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-dwhft created [must-gather ] OUT 2026-04-21T15:24:46.783771607Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather-2wc9p] OUT 2026-04-21T15:25:40.626894312Z waiting for gather to complete [must-gather-2wc9p] OUT 2026-04-21T15:25:40.629863706Z downloading gather output [must-gather-2wc9p] OUT 2026-04-21T15:25:40.905399009Z receiving incremental file list [must-gather-2wc9p] OUT 2026-04-21T15:25:40.918727372Z ./ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.918845145Z aggregated-discovery-api.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.919168063Z aggregated-discovery-apis.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.919730947Z event-filter.html [must-gather-2wc9p] OUT 2026-04-21T15:25:40.919819049Z timestamp [must-gather-2wc9p] OUT 2026-04-21T15:25:40.919964973Z version [must-gather-2wc9p] OUT 2026-04-21T15:25:40.925254345Z namespaces/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.925274445Z namespaces/opendatahub/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.925334617Z namespaces/opendatahub/opendatahub.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.925432779Z namespaces/opendatahub/apps.openshift.io/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.925498951Z namespaces/opendatahub/apps.openshift.io/deploymentconfigs.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.925558842Z namespaces/opendatahub/apps/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.925623404Z namespaces/opendatahub/apps/daemonsets.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.925727876Z namespaces/opendatahub/apps/deployments.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.92587561Z namespaces/opendatahub/apps/replicasets.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.926053195Z namespaces/opendatahub/apps/statefulsets.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.926102196Z namespaces/opendatahub/autoscaling/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.926173868Z namespaces/opendatahub/autoscaling/horizontalpodautoscalers.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.926229109Z namespaces/opendatahub/batch/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.926289731Z namespaces/opendatahub/batch/cronjobs.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.926384793Z namespaces/opendatahub/batch/jobs.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.926463725Z namespaces/opendatahub/build.openshift.io/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.926503856Z namespaces/opendatahub/build.openshift.io/buildconfigs.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.926596768Z namespaces/opendatahub/build.openshift.io/builds.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.926634189Z namespaces/opendatahub/core/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.9266827Z namespaces/opendatahub/core/configmaps.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.926839604Z namespaces/opendatahub/core/endpoints.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.926904756Z namespaces/opendatahub/core/events.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.927025659Z namespaces/opendatahub/core/persistentvolumeclaims.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.927114421Z namespaces/opendatahub/core/pods.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.927242754Z namespaces/opendatahub/core/replicationcontrollers.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.927363437Z namespaces/opendatahub/core/secrets.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.927566073Z namespaces/opendatahub/core/services.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.927646744Z namespaces/opendatahub/discovery.k8s.io/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.927688825Z namespaces/opendatahub/discovery.k8s.io/endpointslices.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.927761847Z namespaces/opendatahub/image.openshift.io/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.927798558Z namespaces/opendatahub/image.openshift.io/imagestreams.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.927918061Z namespaces/opendatahub/k8s.ovn.org/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.927955962Z namespaces/opendatahub/k8s.ovn.org/egressfirewalls.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.928072795Z namespaces/opendatahub/k8s.ovn.org/egressqoses.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.928111156Z namespaces/opendatahub/monitoring.coreos.com/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.928162147Z namespaces/opendatahub/monitoring.coreos.com/servicemonitors.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.928233969Z namespaces/opendatahub/networking.k8s.io/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.92827346Z namespaces/opendatahub/networking.k8s.io/networkpolicies.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.928346412Z namespaces/opendatahub/pods/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.928365082Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-6f49596498-xxpmg/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.928403633Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-6f49596498-xxpmg/kubeflow-trainer-controller-manager-6f49596498-xxpmg.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.928525997Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-6f49596498-xxpmg/manager/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.928539317Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-6f49596498-xxpmg/manager/manager/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.928542917Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-6f49596498-xxpmg/manager/manager/logs/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.928581068Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-6f49596498-xxpmg/manager/manager/logs/current.log [must-gather-2wc9p] OUT 2026-04-21T15:25:40.930038284Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-6f49596498-xxpmg/manager/manager/logs/previous.insecure.log [must-gather-2wc9p] OUT 2026-04-21T15:25:40.930109576Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-6f49596498-xxpmg/manager/manager/logs/previous.log [must-gather-2wc9p] OUT 2026-04-21T15:25:40.930140517Z namespaces/opendatahub/policy/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.930194318Z namespaces/opendatahub/policy/poddisruptionbudgets.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.930244919Z namespaces/opendatahub/route.openshift.io/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.930309561Z namespaces/opendatahub/route.openshift.io/routes.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.930344962Z namespaces/openshift/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.930382373Z namespaces/openshift/openshift.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.930487426Z namespaces/openshift/apps.openshift.io/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.930616359Z namespaces/openshift/apps.openshift.io/deploymentconfigs.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.930640069Z namespaces/openshift/apps/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.930703501Z namespaces/openshift/apps/daemonsets.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.930766562Z namespaces/openshift/apps/deployments.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.930848904Z namespaces/openshift/apps/replicasets.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.930931757Z namespaces/openshift/apps/statefulsets.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.930982408Z namespaces/openshift/autoscaling/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.931026679Z namespaces/openshift/autoscaling/horizontalpodautoscalers.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.93107887Z namespaces/openshift/batch/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.931118801Z namespaces/openshift/batch/cronjobs.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.931199813Z namespaces/openshift/batch/jobs.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.931252045Z namespaces/openshift/build.openshift.io/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.931294366Z namespaces/openshift/build.openshift.io/buildconfigs.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.931377378Z namespaces/openshift/build.openshift.io/builds.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.931440009Z namespaces/openshift/core/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.931488211Z namespaces/openshift/core/configmaps.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.931608334Z namespaces/openshift/core/endpoints.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.931696506Z namespaces/openshift/core/events.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.931779498Z namespaces/openshift/core/persistentvolumeclaims.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.93186367Z namespaces/openshift/core/pods.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.931946342Z namespaces/openshift/core/replicationcontrollers.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.932063435Z namespaces/openshift/core/secrets.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.932228089Z namespaces/openshift/core/services.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.932288151Z namespaces/openshift/discovery.k8s.io/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.932345082Z namespaces/openshift/discovery.k8s.io/endpointslices.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.932389723Z namespaces/openshift/image.openshift.io/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.932447555Z namespaces/openshift/image.openshift.io/imagestreams.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.933913221Z namespaces/openshift/k8s.ovn.org/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.933961132Z namespaces/openshift/k8s.ovn.org/egressfirewalls.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.934046984Z namespaces/openshift/k8s.ovn.org/egressqoses.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.934109476Z namespaces/openshift/monitoring.coreos.com/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.934131617Z namespaces/openshift/monitoring.coreos.com/servicemonitors.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.934188348Z namespaces/openshift/networking.k8s.io/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.934227399Z namespaces/openshift/networking.k8s.io/networkpolicies.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.9342806Z namespaces/openshift/policy/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.934322701Z namespaces/openshift/policy/poddisruptionbudgets.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.934371512Z namespaces/openshift/route.openshift.io/ [must-gather-2wc9p] OUT 2026-04-21T15:25:40.934401873Z namespaces/openshift/route.openshift.io/routes.yaml [must-gather-2wc9p] OUT 2026-04-21T15:25:40.942705041Z [must-gather-2wc9p] OUT 2026-04-21T15:25:40.942739532Z sent 1,380 bytes received 99,153 bytes 201,066.00 bytes/sec [must-gather-2wc9p] OUT 2026-04-21T15:25:40.942756282Z total size is 1,277,360 speedup is 12.71 [must-gather ] OUT 2026-04-21T15:25:40.97711726Z namespace/openshift-must-gather-5v6kh deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 6ad81a53-3b46-46e7-8c0a-fd60d6a42aa0 ClientVersion: 4.21.4 ClusterVersion: Stable at "4.20.19" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing