[must-gather ] OUT 2026-04-21T06:40:08.74701027Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: c0a5ebb0-60cb-45c6-8dee-104d5e870d11 ClientVersion: 4.21.4 ClusterVersion: Stable at "4.20.19" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing [must-gather ] OUT 2026-04-21T06:40:08.792633557Z namespace/openshift-must-gather-56jvx created [must-gather ] OUT 2026-04-21T06:40:08.799886705Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-ltvw5 created [must-gather ] OUT 2026-04-21T06:40:08.817461205Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather-p6fpk] OUT 2026-04-21T06:41:00.374521764Z waiting for gather to complete [must-gather-p6fpk] OUT 2026-04-21T06:41:00.389744887Z downloading gather output [must-gather-p6fpk] OUT 2026-04-21T06:41:00.779451803Z receiving incremental file list [must-gather-p6fpk] OUT 2026-04-21T06:41:00.790478624Z ./ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.790523685Z aggregated-discovery-api.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.82337838Z aggregated-discovery-apis.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.823974595Z event-filter.html [must-gather-p6fpk] OUT 2026-04-21T06:41:00.824131679Z timestamp [must-gather-p6fpk] OUT 2026-04-21T06:41:00.824305593Z version [must-gather-p6fpk] OUT 2026-04-21T06:41:00.82459454Z namespaces/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.824643551Z namespaces/opendatahub/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.824699703Z namespaces/opendatahub/opendatahub.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.824783415Z namespaces/opendatahub/apps.openshift.io/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.824844007Z namespaces/opendatahub/apps.openshift.io/deploymentconfigs.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.824920878Z namespaces/opendatahub/apps/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.82497453Z namespaces/opendatahub/apps/daemonsets.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.825088942Z namespaces/opendatahub/apps/deployments.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.825272577Z namespaces/opendatahub/apps/replicasets.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.825428191Z namespaces/opendatahub/apps/statefulsets.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.825484762Z namespaces/opendatahub/autoscaling/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.825540944Z namespaces/opendatahub/autoscaling/horizontalpodautoscalers.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.825595655Z namespaces/opendatahub/batch/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.825693377Z namespaces/opendatahub/batch/cronjobs.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.825832281Z namespaces/opendatahub/batch/jobs.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.825893612Z namespaces/opendatahub/build.openshift.io/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.825948674Z namespaces/opendatahub/build.openshift.io/buildconfigs.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.826058646Z namespaces/opendatahub/build.openshift.io/builds.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.826109738Z namespaces/opendatahub/core/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.82618544Z namespaces/opendatahub/core/configmaps.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.826333823Z namespaces/opendatahub/core/endpoints.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.826445036Z namespaces/opendatahub/core/events.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.826639471Z namespaces/opendatahub/core/persistentvolumeclaims.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.826752163Z namespaces/opendatahub/core/pods.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.826910207Z namespaces/opendatahub/core/replicationcontrollers.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.827069381Z namespaces/opendatahub/core/secrets.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.827295497Z namespaces/opendatahub/core/services.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.827365818Z namespaces/opendatahub/discovery.k8s.io/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.82742191Z namespaces/opendatahub/discovery.k8s.io/endpointslices.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.827500732Z namespaces/opendatahub/image.openshift.io/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.827552223Z namespaces/opendatahub/image.openshift.io/imagestreams.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.827687026Z namespaces/opendatahub/k8s.ovn.org/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.827733867Z namespaces/opendatahub/k8s.ovn.org/egressfirewalls.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.82785052Z namespaces/opendatahub/k8s.ovn.org/egressqoses.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.827916062Z namespaces/opendatahub/monitoring.coreos.com/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.827976493Z namespaces/opendatahub/monitoring.coreos.com/servicemonitors.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.828034015Z namespaces/opendatahub/networking.k8s.io/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.828081726Z namespaces/opendatahub/networking.k8s.io/networkpolicies.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.828146088Z namespaces/opendatahub/pods/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.828160538Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-7778bbcbb8-ms24v/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.828221769Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-7778bbcbb8-ms24v/kubeflow-trainer-controller-manager-7778bbcbb8-ms24v.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.828335492Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-7778bbcbb8-ms24v/manager/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.828350753Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-7778bbcbb8-ms24v/manager/manager/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.828354533Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-7778bbcbb8-ms24v/manager/manager/logs/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.828375793Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-7778bbcbb8-ms24v/manager/manager/logs/current.log [must-gather-p6fpk] OUT 2026-04-21T06:41:00.829939711Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-7778bbcbb8-ms24v/manager/manager/logs/previous.insecure.log [must-gather-p6fpk] OUT 2026-04-21T06:41:00.830041014Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-7778bbcbb8-ms24v/manager/manager/logs/previous.log [must-gather-p6fpk] OUT 2026-04-21T06:41:00.830097285Z namespaces/opendatahub/policy/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.830133506Z namespaces/opendatahub/policy/poddisruptionbudgets.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.830202018Z namespaces/opendatahub/route.openshift.io/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.830325091Z namespaces/opendatahub/route.openshift.io/routes.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.830374342Z namespaces/openshift/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.830444034Z namespaces/openshift/openshift.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.830498895Z namespaces/openshift/apps.openshift.io/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.830573827Z namespaces/openshift/apps.openshift.io/deploymentconfigs.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.830660599Z namespaces/openshift/apps/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.830731131Z namespaces/openshift/apps/daemonsets.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.830824143Z namespaces/openshift/apps/deployments.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.830938356Z namespaces/openshift/apps/replicasets.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.831036568Z namespaces/openshift/apps/statefulsets.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.83109011Z namespaces/openshift/autoscaling/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.831159441Z namespaces/openshift/autoscaling/horizontalpodautoscalers.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.831229293Z namespaces/openshift/batch/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.831283845Z namespaces/openshift/batch/cronjobs.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.831395017Z namespaces/openshift/batch/jobs.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.831452689Z namespaces/openshift/build.openshift.io/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.83151236Z namespaces/openshift/build.openshift.io/buildconfigs.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.831629093Z namespaces/openshift/build.openshift.io/builds.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.831696935Z namespaces/openshift/core/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.831752056Z namespaces/openshift/core/configmaps.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.831883299Z namespaces/openshift/core/endpoints.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.831990262Z namespaces/openshift/core/events.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.832130885Z namespaces/openshift/core/persistentvolumeclaims.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.832231448Z namespaces/openshift/core/pods.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.83233432Z namespaces/openshift/core/replicationcontrollers.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.832517065Z namespaces/openshift/core/secrets.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.832760041Z namespaces/openshift/core/services.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.832824572Z namespaces/openshift/discovery.k8s.io/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.832883324Z namespaces/openshift/discovery.k8s.io/endpointslices.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.832947365Z namespaces/openshift/image.openshift.io/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.832998516Z namespaces/openshift/image.openshift.io/imagestreams.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.834619396Z namespaces/openshift/k8s.ovn.org/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.834693498Z namespaces/openshift/k8s.ovn.org/egressfirewalls.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.834805871Z namespaces/openshift/k8s.ovn.org/egressqoses.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.834873402Z namespaces/openshift/monitoring.coreos.com/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.834933504Z namespaces/openshift/monitoring.coreos.com/servicemonitors.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.834999836Z namespaces/openshift/networking.k8s.io/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.835059187Z namespaces/openshift/networking.k8s.io/networkpolicies.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.835112858Z namespaces/openshift/policy/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.83516526Z namespaces/openshift/policy/poddisruptionbudgets.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.835226371Z namespaces/openshift/route.openshift.io/ [must-gather-p6fpk] OUT 2026-04-21T06:41:00.835279942Z namespaces/openshift/route.openshift.io/routes.yaml [must-gather-p6fpk] OUT 2026-04-21T06:41:00.842246443Z [must-gather-p6fpk] OUT 2026-04-21T06:41:00.842266584Z sent 1,380 bytes received 97,595 bytes 197,950.00 bytes/sec [must-gather-p6fpk] OUT 2026-04-21T06:41:00.842272074Z total size is 1,252,632 speedup is 12.66 [must-gather ] OUT 2026-04-21T06:41:00.872090535Z namespace/openshift-must-gather-56jvx deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: c0a5ebb0-60cb-45c6-8dee-104d5e870d11 ClientVersion: 4.21.4 ClusterVersion: Stable at "4.20.19" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing