[must-gather ] OUT 2026-07-20T11:04:18.294684263Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: a2da988e-241e-4796-a25a-a1ce93724528 ClientVersion: 4.22.3 ClusterVersion: Stable at "4.21.25" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing [must-gather ] OUT 2026-07-20T11:04:18.352925134Z namespace/openshift-must-gather-4tvw9 created [must-gather ] OUT 2026-07-20T11:04:18.361078039Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-ld9bq created [must-gather ] OUT 2026-07-20T11:04:18.381730174Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather-dztjd] OUT 2026-07-20T11:05:12.534138147Z waiting for gather to complete [must-gather-dztjd] OUT 2026-07-20T11:05:12.537684818Z downloading gather output [must-gather-dztjd] OUT 2026-07-20T11:05:12.836947444Z receiving incremental file list [must-gather-dztjd] OUT 2026-07-20T11:05:12.850539917Z ./ [must-gather-dztjd] OUT 2026-07-20T11:05:12.850649635Z aggregated-discovery-api.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.85308242Z aggregated-discovery-apis.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.853924457Z event-filter.html [must-gather-dztjd] OUT 2026-07-20T11:05:12.854036644Z timestamp [must-gather-dztjd] OUT 2026-07-20T11:05:12.854153392Z version [must-gather-dztjd] OUT 2026-07-20T11:05:12.855540147Z namespaces/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.855680706Z namespaces/opendatahub/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.855713049Z namespaces/opendatahub/opendatahub.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.855728919Z namespaces/opendatahub/apps.openshift.io/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.85573862Z namespaces/opendatahub/apps.openshift.io/deploymentconfigs.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.855747621Z namespaces/opendatahub/apps/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.855773473Z namespaces/opendatahub/apps/daemonsets.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.855858558Z namespaces/opendatahub/apps/deployments.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.855977816Z namespaces/opendatahub/apps/replicasets.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.856081263Z namespaces/opendatahub/apps/statefulsets.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.856130337Z namespaces/opendatahub/autoscaling/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.85617801Z namespaces/opendatahub/autoscaling/horizontalpodautoscalers.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.856229973Z namespaces/opendatahub/batch/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.856269256Z namespaces/opendatahub/batch/cronjobs.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.856370603Z namespaces/opendatahub/batch/jobs.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.856423967Z namespaces/opendatahub/build.openshift.io/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.856464989Z namespaces/opendatahub/build.openshift.io/buildconfigs.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.856561176Z namespaces/opendatahub/build.openshift.io/builds.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.856611979Z namespaces/opendatahub/core/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.856671903Z namespaces/opendatahub/core/configmaps.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.856798532Z namespaces/opendatahub/core/endpoints.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.856879688Z namespaces/opendatahub/core/events.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.857004666Z namespaces/opendatahub/core/persistentvolumeclaims.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.857104993Z namespaces/opendatahub/core/pods.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.857250093Z namespaces/opendatahub/core/replicationcontrollers.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.857394683Z namespaces/opendatahub/core/secrets.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.857577395Z namespaces/opendatahub/core/services.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.857620878Z namespaces/opendatahub/discovery.k8s.io/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.857671101Z namespaces/opendatahub/discovery.k8s.io/endpointslices.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.857730795Z namespaces/opendatahub/image.openshift.io/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.857782169Z namespaces/opendatahub/image.openshift.io/imagestreams.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.857878215Z namespaces/opendatahub/k8s.ovn.org/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.857919388Z namespaces/opendatahub/k8s.ovn.org/egressfirewalls.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.857996624Z namespaces/opendatahub/k8s.ovn.org/egressqoses.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.858042716Z namespaces/opendatahub/monitoring.coreos.com/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.858080119Z namespaces/opendatahub/monitoring.coreos.com/servicemonitors.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.858128842Z namespaces/opendatahub/networking.k8s.io/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.858166185Z namespaces/opendatahub/networking.k8s.io/networkpolicies.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.858212848Z namespaces/opendatahub/pods/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.858219919Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-6d47c7cb99-dhqvk/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.858281513Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-6d47c7cb99-dhqvk/kubeflow-trainer-controller-manager-6d47c7cb99-dhqvk.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.858373899Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-6d47c7cb99-dhqvk/manager/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.8583873Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-6d47c7cb99-dhqvk/manager/manager/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.85839164Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-6d47c7cb99-dhqvk/manager/manager/logs/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.858414192Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-6d47c7cb99-dhqvk/manager/manager/logs/current.log [must-gather-dztjd] OUT 2026-07-20T11:05:12.859923324Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-6d47c7cb99-dhqvk/manager/manager/logs/previous.insecure.log [must-gather-dztjd] OUT 2026-07-20T11:05:12.859997429Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-6d47c7cb99-dhqvk/manager/manager/logs/previous.log [must-gather-dztjd] OUT 2026-07-20T11:05:12.860017041Z namespaces/opendatahub/policy/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.860069024Z namespaces/opendatahub/policy/poddisruptionbudgets.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.860120138Z namespaces/opendatahub/route.openshift.io/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.86015026Z namespaces/opendatahub/route.openshift.io/routes.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.860203903Z namespaces/openshift/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.860243886Z namespaces/openshift/openshift.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.8602996Z namespaces/openshift/apps.openshift.io/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.860365334Z namespaces/openshift/apps.openshift.io/deploymentconfigs.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.860413928Z namespaces/openshift/apps/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.860465081Z namespaces/openshift/apps/daemonsets.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.860540566Z namespaces/openshift/apps/deployments.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.860620262Z namespaces/openshift/apps/replicasets.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.860701097Z namespaces/openshift/apps/statefulsets.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.86075286Z namespaces/openshift/autoscaling/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.860789853Z namespaces/openshift/autoscaling/horizontalpodautoscalers.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.860840606Z namespaces/openshift/batch/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.860868588Z namespaces/openshift/batch/cronjobs.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.860944444Z namespaces/openshift/batch/jobs.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.861002917Z namespaces/openshift/build.openshift.io/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.86103271Z namespaces/openshift/build.openshift.io/buildconfigs.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.861127116Z namespaces/openshift/build.openshift.io/builds.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.861165008Z namespaces/openshift/core/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.861210382Z namespaces/openshift/core/configmaps.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.861328379Z namespaces/openshift/core/endpoints.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.861411285Z namespaces/openshift/core/events.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.861497171Z namespaces/openshift/core/persistentvolumeclaims.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.861578227Z namespaces/openshift/core/pods.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.861657512Z namespaces/openshift/core/replicationcontrollers.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.861787751Z namespaces/openshift/core/secrets.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.861957022Z namespaces/openshift/core/services.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.862008936Z namespaces/openshift/discovery.k8s.io/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.862040428Z namespaces/openshift/discovery.k8s.io/endpointslices.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.862094862Z namespaces/openshift/image.openshift.io/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.862138214Z namespaces/openshift/image.openshift.io/imagestreams.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.863621805Z namespaces/openshift/k8s.ovn.org/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.863645617Z namespaces/openshift/k8s.ovn.org/egressfirewalls.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.863741114Z namespaces/openshift/k8s.ovn.org/egressqoses.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.863782156Z namespaces/openshift/monitoring.coreos.com/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.863876723Z namespaces/openshift/monitoring.coreos.com/servicemonitors.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.863928426Z namespaces/openshift/networking.k8s.io/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.863941627Z namespaces/openshift/networking.k8s.io/networkpolicies.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.864003401Z namespaces/openshift/policy/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.864045854Z namespaces/openshift/policy/poddisruptionbudgets.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.864093617Z namespaces/openshift/route.openshift.io/ [must-gather-dztjd] OUT 2026-07-20T11:05:12.86412486Z namespaces/openshift/route.openshift.io/routes.yaml [must-gather-dztjd] OUT 2026-07-20T11:05:12.872171606Z [must-gather-dztjd] OUT 2026-07-20T11:05:12.872197348Z sent 1,372 bytes received 99,390 bytes 201,524.00 bytes/sec [must-gather-dztjd] OUT 2026-07-20T11:05:12.872205898Z total size is 1,276,723 speedup is 12.67 [must-gather ] OUT 2026-07-20T11:05:12.904360871Z namespace/openshift-must-gather-4tvw9 deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: a2da988e-241e-4796-a25a-a1ce93724528 ClientVersion: 4.22.3 ClusterVersion: Stable at "4.21.25" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing