[must-gather ] OUT 2026-04-23T09:43:38.139059595Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: b199a4fa-ea48-497d-b1b4-df552bd216e3 ClientVersion: 4.21.4 ClusterVersion: Stable at "4.20.19" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing [must-gather ] OUT 2026-04-23T09:43:38.197637236Z namespace/openshift-must-gather-s7ptq created [must-gather ] OUT 2026-04-23T09:43:38.204813118Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-44pv2 created [must-gather ] OUT 2026-04-23T09:43:38.232941784Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather-j5wdf] OUT 2026-04-23T09:44:30.72378645Z waiting for gather to complete [must-gather-j5wdf] OUT 2026-04-23T09:44:30.727314939Z downloading gather output [must-gather-j5wdf] OUT 2026-04-23T09:44:31.019061273Z receiving incremental file list [must-gather-j5wdf] OUT 2026-04-23T09:44:31.030940304Z ./ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.031052426Z aggregated-discovery-api.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.031347614Z aggregated-discovery-apis.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.03198989Z event-filter.html [must-gather-j5wdf] OUT 2026-04-23T09:44:31.032125423Z timestamp [must-gather-j5wdf] OUT 2026-04-23T09:44:31.032253067Z version [must-gather-j5wdf] OUT 2026-04-23T09:44:31.036929745Z namespaces/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.036948115Z namespaces/opendatahub/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.036992966Z namespaces/opendatahub/opendatahub.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.037055148Z namespaces/opendatahub/apps.openshift.io/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.037095329Z namespaces/opendatahub/apps.openshift.io/deploymentconfigs.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.03715551Z namespaces/opendatahub/apps/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.037194571Z namespaces/opendatahub/apps/daemonsets.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.037274764Z namespaces/opendatahub/apps/deployments.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.037389666Z namespaces/opendatahub/apps/replicasets.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.0375246Z namespaces/opendatahub/apps/statefulsets.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.037567501Z namespaces/opendatahub/autoscaling/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.037602362Z namespaces/opendatahub/autoscaling/horizontalpodautoscalers.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.037658453Z namespaces/opendatahub/batch/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.037694294Z namespaces/opendatahub/batch/cronjobs.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.037775266Z namespaces/opendatahub/batch/jobs.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.037823457Z namespaces/opendatahub/build.openshift.io/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.037858148Z namespaces/opendatahub/build.openshift.io/buildconfigs.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.03794803Z namespaces/opendatahub/build.openshift.io/builds.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.037994022Z namespaces/opendatahub/core/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.038035113Z namespaces/opendatahub/core/configmaps.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.038160096Z namespaces/opendatahub/core/endpoints.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.038249848Z namespaces/opendatahub/core/events.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.038371741Z namespaces/opendatahub/core/persistentvolumeclaims.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.038476794Z namespaces/opendatahub/core/pods.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.038605647Z namespaces/opendatahub/core/replicationcontrollers.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.03872748Z namespaces/opendatahub/core/secrets.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.038920775Z namespaces/opendatahub/core/services.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.038975267Z namespaces/opendatahub/discovery.k8s.io/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.039005757Z namespaces/opendatahub/discovery.k8s.io/endpointslices.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.039072189Z namespaces/opendatahub/image.openshift.io/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.03911512Z namespaces/opendatahub/image.openshift.io/imagestreams.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.039199032Z namespaces/opendatahub/k8s.ovn.org/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.039237423Z namespaces/opendatahub/k8s.ovn.org/egressfirewalls.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.039330745Z namespaces/opendatahub/k8s.ovn.org/egressqoses.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.039375997Z namespaces/opendatahub/monitoring.coreos.com/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.039432558Z namespaces/opendatahub/monitoring.coreos.com/servicemonitors.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.039491209Z namespaces/opendatahub/networking.k8s.io/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.03952426Z namespaces/opendatahub/networking.k8s.io/networkpolicies.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.039572102Z namespaces/opendatahub/pods/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.039578332Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-78c9cf59d7-p7k75/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.039623923Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-78c9cf59d7-p7k75/kubeflow-trainer-controller-manager-78c9cf59d7-p7k75.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.039731586Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-78c9cf59d7-p7k75/manager/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.039738526Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-78c9cf59d7-p7k75/manager/manager/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.039741936Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-78c9cf59d7-p7k75/manager/manager/logs/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.039775377Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-78c9cf59d7-p7k75/manager/manager/logs/current.log [must-gather-j5wdf] OUT 2026-04-23T09:44:31.041245284Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-78c9cf59d7-p7k75/manager/manager/logs/previous.insecure.log [must-gather-j5wdf] OUT 2026-04-23T09:44:31.041316806Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-78c9cf59d7-p7k75/manager/manager/logs/previous.log [must-gather-j5wdf] OUT 2026-04-23T09:44:31.041345056Z namespaces/opendatahub/policy/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.041390697Z namespaces/opendatahub/policy/poddisruptionbudgets.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.04146805Z namespaces/opendatahub/route.openshift.io/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.041504831Z namespaces/opendatahub/route.openshift.io/routes.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.041565832Z namespaces/openshift/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.041614753Z namespaces/openshift/openshift.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.041664835Z namespaces/openshift/apps.openshift.io/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.041699245Z namespaces/openshift/apps.openshift.io/deploymentconfigs.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.041748257Z namespaces/openshift/apps/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.041791808Z namespaces/openshift/apps/daemonsets.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.04189594Z namespaces/openshift/apps/deployments.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.041984703Z namespaces/openshift/apps/replicasets.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.042062814Z namespaces/openshift/apps/statefulsets.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.042115966Z namespaces/openshift/autoscaling/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.042154137Z namespaces/openshift/autoscaling/horizontalpodautoscalers.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.042201668Z namespaces/openshift/batch/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.042246519Z namespaces/openshift/batch/cronjobs.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.042325291Z namespaces/openshift/batch/jobs.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.042368862Z namespaces/openshift/build.openshift.io/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.042408473Z namespaces/openshift/build.openshift.io/buildconfigs.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.042517816Z namespaces/openshift/build.openshift.io/builds.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.042561307Z namespaces/openshift/core/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.042599198Z namespaces/openshift/core/configmaps.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.042703351Z namespaces/openshift/core/endpoints.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.042781733Z namespaces/openshift/core/events.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.042856554Z namespaces/openshift/core/persistentvolumeclaims.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.042938237Z namespaces/openshift/core/pods.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.043017549Z namespaces/openshift/core/replicationcontrollers.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.043148422Z namespaces/openshift/core/secrets.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.043325146Z namespaces/openshift/core/services.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.043368638Z namespaces/openshift/discovery.k8s.io/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.043428699Z namespaces/openshift/discovery.k8s.io/endpointslices.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.043490941Z namespaces/openshift/image.openshift.io/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.043531912Z namespaces/openshift/image.openshift.io/imagestreams.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.044931647Z namespaces/openshift/k8s.ovn.org/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.044970198Z namespaces/openshift/k8s.ovn.org/egressfirewalls.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.04505803Z namespaces/openshift/k8s.ovn.org/egressqoses.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.045095721Z namespaces/openshift/monitoring.coreos.com/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.045136682Z namespaces/openshift/monitoring.coreos.com/servicemonitors.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.045189984Z namespaces/openshift/networking.k8s.io/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.045230855Z namespaces/openshift/networking.k8s.io/networkpolicies.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.045277346Z namespaces/openshift/policy/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.045323857Z namespaces/openshift/policy/poddisruptionbudgets.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.045367488Z namespaces/openshift/route.openshift.io/ [must-gather-j5wdf] OUT 2026-04-23T09:44:31.045398469Z namespaces/openshift/route.openshift.io/routes.yaml [must-gather-j5wdf] OUT 2026-04-23T09:44:31.052503178Z [must-gather-j5wdf] OUT 2026-04-23T09:44:31.052527089Z sent 1,376 bytes received 98,843 bytes 66,812.67 bytes/sec [must-gather-j5wdf] OUT 2026-04-23T09:44:31.052538139Z total size is 1,266,771 speedup is 12.64 [must-gather ] OUT 2026-04-23T09:44:31.084387684Z namespace/openshift-must-gather-s7ptq deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: b199a4fa-ea48-497d-b1b4-df552bd216e3 ClientVersion: 4.21.4 ClusterVersion: Stable at "4.20.19" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing