[must-gather ] OUT 2026-10-30T18:46:21.42305949Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 8a26615c-f79a-49ef-b992-4a1e43210add ClientVersion: 4.22.15 ClusterVersion: Stable at "4.22.20" ClusterOperators: clusteroperator/cloud-credential is not upgradeable because Upgradeable annotation cloudcredential.openshift.io/upgradeable-to on cloudcredential.operator.openshift.io/cluster object needs updating before upgrade. See Manually Creating IAM documentation for instructions on preparing a cluster for upgrade. clusteroperator/authentication is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing [must-gather ] OUT 2026-10-30T18:46:21.555265461Z namespace/openshift-must-gather-h9h72 created [must-gather ] OUT 2026-10-30T18:46:21.574295528Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-mlkpr created [must-gather ] OUT 2026-10-30T18:46:21.615171787Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather-4wxsd] OUT 2026-10-30T18:47:50.35084992Z waiting for gather to complete [must-gather-4wxsd] OUT 2026-10-30T18:47:50.365394533Z downloading gather output [must-gather-4wxsd] OUT 2026-10-30T18:47:50.904834238Z receiving incremental file list [must-gather-4wxsd] OUT 2026-10-30T18:47:50.958464638Z ./ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.958507529Z aggregated-discovery-api.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.958938337Z aggregated-discovery-apis.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.971645074Z event-filter.html [must-gather-4wxsd] OUT 2026-10-30T18:47:50.971766836Z timestamp [must-gather-4wxsd] OUT 2026-10-30T18:47:50.971902889Z version [must-gather-4wxsd] OUT 2026-10-30T18:47:50.972025811Z namespaces/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.972036361Z namespaces/opendatahub/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.972106772Z namespaces/opendatahub/opendatahub.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.972202845Z namespaces/opendatahub/apps.openshift.io/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.972267026Z namespaces/opendatahub/apps.openshift.io/deploymentconfigs.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.972306376Z namespaces/opendatahub/apps/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.972379718Z namespaces/opendatahub/apps/daemonsets.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.97247247Z namespaces/opendatahub/apps/deployments.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.972617453Z namespaces/opendatahub/apps/replicasets.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.972746575Z namespaces/opendatahub/apps/statefulsets.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.972799656Z namespaces/opendatahub/autoscaling/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.972856387Z namespaces/opendatahub/autoscaling/horizontalpodautoscalers.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.972927899Z namespaces/opendatahub/batch/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.97298336Z namespaces/opendatahub/batch/cronjobs.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.973082702Z namespaces/opendatahub/batch/jobs.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.973125382Z namespaces/opendatahub/build.openshift.io/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.973185804Z namespaces/opendatahub/build.openshift.io/buildconfigs.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.973277575Z namespaces/opendatahub/build.openshift.io/builds.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.973334756Z namespaces/opendatahub/core/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.973380717Z namespaces/opendatahub/core/configmaps.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.97352949Z namespaces/opendatahub/core/endpoints.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.973626362Z namespaces/opendatahub/core/events.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.973773895Z namespaces/opendatahub/core/persistentvolumeclaims.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.973895787Z namespaces/opendatahub/core/pods.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.974083931Z namespaces/opendatahub/core/replicationcontrollers.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.974285645Z namespaces/opendatahub/core/secrets.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.974518809Z namespaces/opendatahub/core/services.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.974609611Z namespaces/opendatahub/discovery.k8s.io/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.974689023Z namespaces/opendatahub/discovery.k8s.io/endpointslices.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.974770914Z namespaces/opendatahub/image.openshift.io/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.974845686Z namespaces/opendatahub/image.openshift.io/imagestreams.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.982656187Z namespaces/opendatahub/k8s.ovn.org/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.982728609Z namespaces/opendatahub/k8s.ovn.org/egressfirewalls.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.982863921Z namespaces/opendatahub/k8s.ovn.org/egressqoses.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.982968623Z namespaces/opendatahub/monitoring.coreos.com/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.983031955Z namespaces/opendatahub/monitoring.coreos.com/servicemonitors.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.983108156Z namespaces/opendatahub/networking.k8s.io/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.983169397Z namespaces/opendatahub/networking.k8s.io/networkpolicies.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.983265809Z namespaces/opendatahub/pods/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.983273289Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-2gtpz/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.983340101Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-2gtpz/kubeflow-trainer-controller-manager-84d6d98cc8-2gtpz.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.983465223Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-2gtpz/manager/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.983473353Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-2gtpz/manager/manager/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.983477513Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-2gtpz/manager/manager/logs/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.983535114Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-2gtpz/manager/manager/logs/current.log [must-gather-4wxsd] OUT 2026-10-30T18:47:50.983737378Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-2gtpz/manager/manager/logs/previous.insecure.log [must-gather-4wxsd] OUT 2026-10-30T18:47:50.98383445Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-2gtpz/manager/manager/logs/previous.log [must-gather-4wxsd] OUT 2026-10-30T18:47:50.983896791Z namespaces/opendatahub/policy/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.983958653Z namespaces/opendatahub/policy/poddisruptionbudgets.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.984001963Z namespaces/opendatahub/route.openshift.io/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.984065875Z namespaces/opendatahub/route.openshift.io/routes.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.984111036Z namespaces/openshift/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.984173227Z namespaces/openshift/openshift.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.984229138Z namespaces/openshift/apps.openshift.io/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.984273419Z namespaces/openshift/apps.openshift.io/deploymentconfigs.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.98433248Z namespaces/openshift/apps/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.984379251Z namespaces/openshift/apps/daemonsets.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.984481703Z namespaces/openshift/apps/deployments.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.984573025Z namespaces/openshift/apps/replicasets.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.984666506Z namespaces/openshift/apps/statefulsets.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.984711057Z namespaces/openshift/autoscaling/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.984770008Z namespaces/openshift/autoscaling/horizontalpodautoscalers.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.984833039Z namespaces/openshift/batch/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.984891441Z namespaces/openshift/batch/cronjobs.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.984999043Z namespaces/openshift/batch/jobs.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.985052334Z namespaces/openshift/build.openshift.io/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.985097205Z namespaces/openshift/build.openshift.io/buildconfigs.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.985198346Z namespaces/openshift/build.openshift.io/builds.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.985241847Z namespaces/openshift/core/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.985285138Z namespaces/openshift/core/configmaps.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.98540679Z namespaces/openshift/core/endpoints.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.985500502Z namespaces/openshift/core/events.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.985620265Z namespaces/openshift/core/persistentvolumeclaims.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.985742697Z namespaces/openshift/core/pods.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.98589167Z namespaces/openshift/core/replicationcontrollers.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.986053383Z namespaces/openshift/core/secrets.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.986231766Z namespaces/openshift/core/services.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.986289308Z namespaces/openshift/discovery.k8s.io/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.986346209Z namespaces/openshift/discovery.k8s.io/endpointslices.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.98639769Z namespaces/openshift/image.openshift.io/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.986451761Z namespaces/openshift/image.openshift.io/imagestreams.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.98847815Z namespaces/openshift/k8s.ovn.org/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.988552082Z namespaces/openshift/k8s.ovn.org/egressfirewalls.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.988691074Z namespaces/openshift/k8s.ovn.org/egressqoses.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.988747215Z namespaces/openshift/monitoring.coreos.com/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.988814577Z namespaces/openshift/monitoring.coreos.com/servicemonitors.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.988917329Z namespaces/openshift/networking.k8s.io/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.98897232Z namespaces/openshift/networking.k8s.io/networkpolicies.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.989057491Z namespaces/openshift/policy/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.989133393Z namespaces/openshift/policy/poddisruptionbudgets.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:50.989218315Z namespaces/openshift/route.openshift.io/ [must-gather-4wxsd] OUT 2026-10-30T18:47:50.989285576Z namespaces/openshift/route.openshift.io/routes.yaml [must-gather-4wxsd] OUT 2026-10-30T18:47:51.044470376Z [must-gather-4wxsd] OUT 2026-10-30T18:47:51.044503247Z sent 1,376 bytes received 72,763 bytes 49,426.00 bytes/sec [must-gather-4wxsd] OUT 2026-10-30T18:47:51.044516707Z total size is 872,394 speedup is 11.77 [must-gather ] OUT 2026-10-30T18:47:51.106078222Z namespace/openshift-must-gather-h9h72 deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 8a26615c-f79a-49ef-b992-4a1e43210add ClientVersion: 4.22.15 ClusterVersion: Stable at "4.22.20" ClusterOperators: clusteroperator/cloud-credential is not upgradeable because Upgradeable annotation cloudcredential.openshift.io/upgradeable-to on cloudcredential.operator.openshift.io/cluster object needs updating before upgrade. See Manually Creating IAM documentation for instructions on preparing a cluster for upgrade. clusteroperator/authentication is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing