[must-gather ] OUT 2026-07-27T15:48:46.427898442Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: fc0f6c3b-3678-47b0-b48d-a9e083588dfc ClientVersion: 4.22.5 ClusterVersion: Stable at "4.21.26" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing [must-gather ] OUT 2026-07-27T15:48:46.471103776Z namespace/openshift-must-gather-mtfxg created [must-gather ] OUT 2026-07-27T15:48:46.476715359Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-65vww created [must-gather ] OUT 2026-07-27T15:48:46.502155836Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather-8bg8t] OUT 2026-07-27T15:49:39.523683264Z waiting for gather to complete [must-gather-8bg8t] OUT 2026-07-27T15:49:39.527351985Z downloading gather output [must-gather-8bg8t] OUT 2026-07-27T15:49:39.793064671Z receiving incremental file list [must-gather-8bg8t] OUT 2026-07-27T15:49:39.802702705Z ./ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.802763557Z aggregated-discovery-api.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.803030324Z aggregated-discovery-apis.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.803714363Z event-filter.html [must-gather-8bg8t] OUT 2026-07-27T15:49:39.803823016Z timestamp [must-gather-8bg8t] OUT 2026-07-27T15:49:39.80395989Z version [must-gather-8bg8t] OUT 2026-07-27T15:49:39.808618248Z namespaces/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.808636398Z namespaces/opendatahub/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.808645248Z namespaces/opendatahub/opendatahub.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.80872162Z namespaces/opendatahub/apps.openshift.io/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.808758921Z namespaces/opendatahub/apps.openshift.io/deploymentconfigs.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.808823133Z namespaces/opendatahub/apps/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.808869915Z namespaces/opendatahub/apps/daemonsets.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.808961137Z namespaces/opendatahub/apps/deployments.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.80908911Z namespaces/opendatahub/apps/replicasets.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.809210844Z namespaces/opendatahub/apps/statefulsets.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.809255445Z namespaces/opendatahub/autoscaling/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.809290456Z namespaces/opendatahub/autoscaling/horizontalpodautoscalers.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.809341737Z namespaces/opendatahub/batch/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.809375168Z namespaces/opendatahub/batch/cronjobs.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.809483531Z namespaces/opendatahub/batch/jobs.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.809538073Z namespaces/opendatahub/build.openshift.io/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.809559153Z namespaces/opendatahub/build.openshift.io/buildconfigs.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.809678177Z namespaces/opendatahub/build.openshift.io/builds.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.809710907Z namespaces/opendatahub/core/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.809770259Z namespaces/opendatahub/core/configmaps.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.809929893Z namespaces/opendatahub/core/endpoints.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.810025736Z namespaces/opendatahub/core/events.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.810151819Z namespaces/opendatahub/core/persistentvolumeclaims.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.810227912Z namespaces/opendatahub/core/pods.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.810372956Z namespaces/opendatahub/core/replicationcontrollers.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.810497199Z namespaces/opendatahub/core/secrets.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.810683504Z namespaces/opendatahub/core/services.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.810743176Z namespaces/opendatahub/discovery.k8s.io/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.810782017Z namespaces/opendatahub/discovery.k8s.io/endpointslices.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.810868179Z namespaces/opendatahub/image.openshift.io/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.81089007Z namespaces/opendatahub/image.openshift.io/imagestreams.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.811007503Z namespaces/opendatahub/k8s.ovn.org/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.811036354Z namespaces/opendatahub/k8s.ovn.org/egressfirewalls.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.811120606Z namespaces/opendatahub/k8s.ovn.org/egressqoses.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.811175058Z namespaces/opendatahub/monitoring.coreos.com/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.811208868Z namespaces/opendatahub/monitoring.coreos.com/servicemonitors.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.8112575Z namespaces/opendatahub/networking.k8s.io/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.811285431Z namespaces/opendatahub/networking.k8s.io/networkpolicies.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.811344652Z namespaces/opendatahub/pods/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.811351783Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-665dfd4758-6w4nw/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.811386313Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-665dfd4758-6w4nw/kubeflow-trainer-controller-manager-665dfd4758-6w4nw.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.811476246Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-665dfd4758-6w4nw/manager/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.811485386Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-665dfd4758-6w4nw/manager/manager/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.811489186Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-665dfd4758-6w4nw/manager/manager/logs/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.811516977Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-665dfd4758-6w4nw/manager/manager/logs/current.log [must-gather-8bg8t] OUT 2026-07-27T15:49:39.812785722Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-665dfd4758-6w4nw/manager/manager/logs/previous.insecure.log [must-gather-8bg8t] OUT 2026-07-27T15:49:39.812885494Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-665dfd4758-6w4nw/manager/manager/logs/previous.log [must-gather-8bg8t] OUT 2026-07-27T15:49:39.812915865Z namespaces/opendatahub/policy/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.812961717Z namespaces/opendatahub/policy/poddisruptionbudgets.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.813026188Z namespaces/opendatahub/route.openshift.io/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.813054349Z namespaces/opendatahub/route.openshift.io/routes.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.813115461Z namespaces/openshift/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.813155442Z namespaces/openshift/openshift.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.813213554Z namespaces/openshift/apps.openshift.io/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.813255475Z namespaces/openshift/apps.openshift.io/deploymentconfigs.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.813304676Z namespaces/openshift/apps/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.813347107Z namespaces/openshift/apps/daemonsets.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.813418839Z namespaces/openshift/apps/deployments.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.813512172Z namespaces/openshift/apps/replicasets.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.813594474Z namespaces/openshift/apps/statefulsets.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.813641525Z namespaces/openshift/autoscaling/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.813675456Z namespaces/openshift/autoscaling/horizontalpodautoscalers.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.813729838Z namespaces/openshift/batch/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.813760738Z namespaces/openshift/batch/cronjobs.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.813866541Z namespaces/openshift/batch/jobs.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.813907362Z namespaces/openshift/build.openshift.io/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.813959404Z namespaces/openshift/build.openshift.io/buildconfigs.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.814039776Z namespaces/openshift/build.openshift.io/builds.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.814078117Z namespaces/openshift/core/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.814121008Z namespaces/openshift/core/configmaps.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.814234542Z namespaces/openshift/core/endpoints.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.814323874Z namespaces/openshift/core/events.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.814400466Z namespaces/openshift/core/persistentvolumeclaims.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.814484318Z namespaces/openshift/core/pods.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.814561111Z namespaces/openshift/core/replicationcontrollers.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.814690214Z namespaces/openshift/core/secrets.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.81490112Z namespaces/openshift/core/services.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.81492685Z namespaces/openshift/discovery.k8s.io/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.814970432Z namespaces/openshift/discovery.k8s.io/endpointslices.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.815018943Z namespaces/openshift/image.openshift.io/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.815046774Z namespaces/openshift/image.openshift.io/imagestreams.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.816621477Z namespaces/openshift/k8s.ovn.org/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.816665598Z namespaces/openshift/k8s.ovn.org/egressfirewalls.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.816754441Z namespaces/openshift/k8s.ovn.org/egressqoses.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.816786341Z namespaces/openshift/monitoring.coreos.com/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.816861994Z namespaces/openshift/monitoring.coreos.com/servicemonitors.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.816896015Z namespaces/openshift/networking.k8s.io/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.816950476Z namespaces/openshift/networking.k8s.io/networkpolicies.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.816993277Z namespaces/openshift/policy/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.817037978Z namespaces/openshift/policy/poddisruptionbudgets.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.817081349Z namespaces/openshift/route.openshift.io/ [must-gather-8bg8t] OUT 2026-07-27T15:49:39.817126011Z namespaces/openshift/route.openshift.io/routes.yaml [must-gather-8bg8t] OUT 2026-07-27T15:49:39.822293792Z [must-gather-8bg8t] OUT 2026-07-27T15:49:39.822312393Z sent 1,376 bytes received 94,340 bytes 191,432.00 bytes/sec [must-gather-8bg8t] OUT 2026-07-27T15:49:39.822318273Z total size is 1,195,353 speedup is 12.49 [must-gather ] OUT 2026-07-27T15:49:39.850930948Z namespace/openshift-must-gather-mtfxg deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: fc0f6c3b-3678-47b0-b48d-a9e083588dfc ClientVersion: 4.22.5 ClusterVersion: Stable at "4.21.26" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing