[must-gather ] OUT 2026-11-03T15:44:23.344832335Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: d3ceb001-8499-43d0-af13-3d182a10224a ClientVersion: 4.22.15 ClusterVersion: Stable at "4.22.20" ClusterOperators: clusteroperator/cloud-credential is not upgradeable because Upgradeable annotation cloudcredential.openshift.io/upgradeable-to on cloudcredential.operator.openshift.io/cluster object needs updating before upgrade. See Manually Creating IAM documentation for instructions on preparing a cluster for upgrade. clusteroperator/authentication is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing [must-gather ] OUT 2026-11-03T15:44:23.496020804Z namespace/openshift-must-gather-bkwzr created [must-gather ] OUT 2026-11-03T15:44:23.514488675Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-nj7j7 created [must-gather ] OUT 2026-11-03T15:44:23.574810951Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather-qkkld] OUT 2026-11-03T15:47:42.408741554Z waiting for gather to complete [must-gather-qkkld] OUT 2026-11-03T15:47:42.423356923Z downloading gather output [must-gather-qkkld] OUT 2026-11-03T15:47:43.424888826Z receiving incremental file list [must-gather-qkkld] OUT 2026-11-03T15:47:43.604071186Z ./ [must-gather-qkkld] OUT 2026-11-03T15:47:43.604323831Z aggregated-discovery-api.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.604625847Z aggregated-discovery-apis.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.605443322Z event-filter.html [must-gather-qkkld] OUT 2026-11-03T15:47:43.605558174Z timestamp [must-gather-qkkld] OUT 2026-11-03T15:47:43.615906131Z version [must-gather-qkkld] OUT 2026-11-03T15:47:43.656549645Z namespaces/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.656572976Z namespaces/opendatahub/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.656700648Z namespaces/opendatahub/opendatahub.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.656836041Z namespaces/opendatahub/apps.openshift.io/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.656910322Z namespaces/opendatahub/apps.openshift.io/deploymentconfigs.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.656978933Z namespaces/opendatahub/apps/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.657055975Z namespaces/opendatahub/apps/daemonsets.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.657176997Z namespaces/opendatahub/apps/deployments.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.65733604Z namespaces/opendatahub/apps/replicasets.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.657479363Z namespaces/opendatahub/apps/statefulsets.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.657515603Z namespaces/opendatahub/autoscaling/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.657590365Z namespaces/opendatahub/autoscaling/horizontalpodautoscalers.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.657646326Z namespaces/opendatahub/batch/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.657721927Z namespaces/opendatahub/batch/cronjobs.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.657825609Z namespaces/opendatahub/batch/jobs.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.65787852Z namespaces/opendatahub/build.openshift.io/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.657941602Z namespaces/opendatahub/build.openshift.io/buildconfigs.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.658024223Z namespaces/opendatahub/build.openshift.io/builds.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.658080994Z namespaces/opendatahub/core/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.658136685Z namespaces/opendatahub/core/configmaps.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.658300698Z namespaces/opendatahub/core/endpoints.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.65840604Z namespaces/opendatahub/core/events.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.658544333Z namespaces/opendatahub/core/persistentvolumeclaims.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.658634035Z namespaces/opendatahub/core/pods.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.658815748Z namespaces/opendatahub/core/replicationcontrollers.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.658962861Z namespaces/opendatahub/core/secrets.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.659159415Z namespaces/opendatahub/core/services.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.659232416Z namespaces/opendatahub/discovery.k8s.io/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.659296177Z namespaces/opendatahub/discovery.k8s.io/endpointslices.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.659356698Z namespaces/opendatahub/image.openshift.io/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.659412959Z namespaces/opendatahub/image.openshift.io/imagestreams.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.668717347Z namespaces/opendatahub/k8s.ovn.org/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.668782558Z namespaces/opendatahub/k8s.ovn.org/egressfirewalls.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.66891242Z namespaces/opendatahub/k8s.ovn.org/egressqoses.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.668991142Z namespaces/opendatahub/monitoring.coreos.com/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.669070233Z namespaces/opendatahub/monitoring.coreos.com/servicemonitors.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.669171215Z namespaces/opendatahub/networking.k8s.io/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.669247537Z namespaces/opendatahub/networking.k8s.io/networkpolicies.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.669352589Z namespaces/opendatahub/pods/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.669364819Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-9vdgx/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.66941881Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-9vdgx/kubeflow-trainer-controller-manager-84d6d98cc8-9vdgx.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.669544402Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-9vdgx/manager/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.669552112Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-9vdgx/manager/manager/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.669556112Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-9vdgx/manager/manager/logs/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.669614704Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-9vdgx/manager/manager/logs/current.log [must-gather-qkkld] OUT 2026-11-03T15:47:43.669863838Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-9vdgx/manager/manager/logs/previous.insecure.log [must-gather-qkkld] OUT 2026-11-03T15:47:43.66996238Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-9vdgx/manager/manager/logs/previous.log [must-gather-qkkld] OUT 2026-11-03T15:47:43.670009771Z namespaces/opendatahub/policy/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.670067372Z namespaces/opendatahub/policy/poddisruptionbudgets.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.670121123Z namespaces/opendatahub/route.openshift.io/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.670175664Z namespaces/opendatahub/route.openshift.io/routes.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.670226545Z namespaces/openshift/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.670291507Z namespaces/openshift/openshift.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.670347518Z namespaces/openshift/apps.openshift.io/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.670401989Z namespaces/openshift/apps.openshift.io/deploymentconfigs.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.67045007Z namespaces/openshift/apps/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.670511071Z namespaces/openshift/apps/daemonsets.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.670626423Z namespaces/openshift/apps/deployments.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.670775706Z namespaces/openshift/apps/replicasets.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.670885348Z namespaces/openshift/apps/statefulsets.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.670950899Z namespaces/openshift/autoscaling/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.671015731Z namespaces/openshift/autoscaling/horizontalpodautoscalers.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.671086262Z namespaces/openshift/batch/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.671149163Z namespaces/openshift/batch/cronjobs.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.671262245Z namespaces/openshift/batch/jobs.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.671329446Z namespaces/openshift/build.openshift.io/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.671394328Z namespaces/openshift/build.openshift.io/buildconfigs.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.67150566Z namespaces/openshift/build.openshift.io/builds.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.671575721Z namespaces/openshift/core/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.671639142Z namespaces/openshift/core/configmaps.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.671790505Z namespaces/openshift/core/endpoints.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.671905067Z namespaces/openshift/core/events.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.67202003Z namespaces/openshift/core/persistentvolumeclaims.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.672128812Z namespaces/openshift/core/pods.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.672242664Z namespaces/openshift/core/replicationcontrollers.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.672397907Z namespaces/openshift/core/secrets.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.67258882Z namespaces/openshift/core/services.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.672660752Z namespaces/openshift/discovery.k8s.io/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.672760713Z namespaces/openshift/discovery.k8s.io/endpointslices.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.672820365Z namespaces/openshift/image.openshift.io/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.672884706Z namespaces/openshift/image.openshift.io/imagestreams.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.674527827Z namespaces/openshift/k8s.ovn.org/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.674590958Z namespaces/openshift/k8s.ovn.org/egressfirewalls.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.674738501Z namespaces/openshift/k8s.ovn.org/egressqoses.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.674806632Z namespaces/openshift/monitoring.coreos.com/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.674872454Z namespaces/openshift/monitoring.coreos.com/servicemonitors.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.674945635Z namespaces/openshift/networking.k8s.io/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.675006876Z namespaces/openshift/networking.k8s.io/networkpolicies.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.675077918Z namespaces/openshift/policy/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.675137939Z namespaces/openshift/policy/poddisruptionbudgets.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.6752082Z namespaces/openshift/route.openshift.io/ [must-gather-qkkld] OUT 2026-11-03T15:47:43.675268131Z namespaces/openshift/route.openshift.io/routes.yaml [must-gather-qkkld] OUT 2026-11-03T15:47:43.803085144Z [must-gather-qkkld] OUT 2026-11-03T15:47:43.803125085Z sent 1,380 bytes received 73,009 bytes 49,592.67 bytes/sec [must-gather-qkkld] OUT 2026-11-03T15:47:43.803137405Z total size is 876,300 speedup is 11.78 [must-gather ] OUT 2026-11-03T15:47:43.904592776Z namespace/openshift-must-gather-bkwzr deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: d3ceb001-8499-43d0-af13-3d182a10224a ClientVersion: 4.22.15 ClusterVersion: Stable at "4.22.20" ClusterOperators: clusteroperator/cloud-credential is not upgradeable because Upgradeable annotation cloudcredential.openshift.io/upgradeable-to on cloudcredential.operator.openshift.io/cluster object needs updating before upgrade. See Manually Creating IAM documentation for instructions on preparing a cluster for upgrade. clusteroperator/authentication is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing