[must-gather ] OUT 2026-10-31T08:10:48.275524483Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: ed1fcdb5-a2c2-4308-9b9b-93581ee4770e ClientVersion: 4.22.15 ClusterVersion: Stable at "4.22.20" ClusterOperators: clusteroperator/cloud-credential is not upgradeable because Upgradeable annotation cloudcredential.openshift.io/upgradeable-to on cloudcredential.operator.openshift.io/cluster object needs updating before upgrade. See Manually Creating IAM documentation for instructions on preparing a cluster for upgrade. clusteroperator/authentication is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing [must-gather ] OUT 2026-10-31T08:10:48.409031877Z namespace/openshift-must-gather-njhxc created [must-gather ] OUT 2026-10-31T08:10:48.427573578Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-jkrtz created [must-gather ] OUT 2026-10-31T08:10:48.470341151Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather-px7mm] OUT 2026-10-31T08:14:02.861455063Z waiting for gather to complete [must-gather-px7mm] OUT 2026-10-31T08:14:02.876081975Z downloading gather output [must-gather-px7mm] OUT 2026-10-31T08:14:03.928089566Z receiving incremental file list [must-gather-px7mm] OUT 2026-10-31T08:14:04.109263228Z ./ [must-gather-px7mm] OUT 2026-10-31T08:14:04.109350111Z aggregated-discovery-api.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.109786422Z aggregated-discovery-apis.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.110668565Z event-filter.html [must-gather-px7mm] OUT 2026-10-31T08:14:04.110797439Z timestamp [must-gather-px7mm] OUT 2026-10-31T08:14:04.12270295Z version [must-gather-px7mm] OUT 2026-10-31T08:14:04.160732683Z namespaces/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.160754284Z namespaces/opendatahub/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.160852156Z namespaces/opendatahub/opendatahub.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.160983779Z namespaces/opendatahub/apps.openshift.io/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.161038931Z namespaces/opendatahub/apps.openshift.io/deploymentconfigs.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.161121603Z namespaces/opendatahub/apps/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.161191395Z namespaces/opendatahub/apps/daemonsets.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.161309668Z namespaces/opendatahub/apps/deployments.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.161478222Z namespaces/opendatahub/apps/replicasets.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.161639817Z namespaces/opendatahub/apps/statefulsets.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.161691008Z namespaces/opendatahub/autoscaling/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.16175367Z namespaces/opendatahub/autoscaling/horizontalpodautoscalers.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.161826612Z namespaces/opendatahub/batch/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.161880883Z namespaces/opendatahub/batch/cronjobs.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.161982765Z namespaces/opendatahub/batch/jobs.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.162036647Z namespaces/opendatahub/build.openshift.io/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.162071628Z namespaces/opendatahub/build.openshift.io/buildconfigs.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.162178851Z namespaces/opendatahub/build.openshift.io/builds.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.162219742Z namespaces/opendatahub/core/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.162275893Z namespaces/opendatahub/core/configmaps.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.162438897Z namespaces/opendatahub/core/endpoints.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.1625418Z namespaces/opendatahub/core/events.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.162692114Z namespaces/opendatahub/core/persistentvolumeclaims.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.162783426Z namespaces/opendatahub/core/pods.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.162957781Z namespaces/opendatahub/core/replicationcontrollers.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.163099695Z namespaces/opendatahub/core/secrets.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.16329286Z namespaces/opendatahub/core/services.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.163364852Z namespaces/opendatahub/discovery.k8s.io/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.163418923Z namespaces/opendatahub/discovery.k8s.io/endpointslices.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.163490225Z namespaces/opendatahub/image.openshift.io/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.163529536Z namespaces/opendatahub/image.openshift.io/imagestreams.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.174556454Z namespaces/opendatahub/k8s.ovn.org/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.174622866Z namespaces/opendatahub/k8s.ovn.org/egressfirewalls.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.174753439Z namespaces/opendatahub/k8s.ovn.org/egressqoses.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.17479091Z namespaces/opendatahub/monitoring.coreos.com/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.174885853Z namespaces/opendatahub/monitoring.coreos.com/servicemonitors.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.174947674Z namespaces/opendatahub/networking.k8s.io/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.175007516Z namespaces/opendatahub/networking.k8s.io/networkpolicies.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.175072587Z namespaces/opendatahub/pods/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.175082248Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-tfc4s/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.175130279Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-tfc4s/kubeflow-trainer-controller-manager-84d6d98cc8-tfc4s.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.175238412Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-tfc4s/manager/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.175249052Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-tfc4s/manager/manager/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.175254232Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-tfc4s/manager/manager/logs/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.175288353Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-tfc4s/manager/manager/logs/current.log [must-gather-px7mm] OUT 2026-10-31T08:14:04.175482728Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-tfc4s/manager/manager/logs/previous.insecure.log [must-gather-px7mm] OUT 2026-10-31T08:14:04.17556532Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84d6d98cc8-tfc4s/manager/manager/logs/previous.log [must-gather-px7mm] OUT 2026-10-31T08:14:04.175617662Z namespaces/opendatahub/policy/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.175669433Z namespaces/opendatahub/policy/poddisruptionbudgets.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.175718134Z namespaces/opendatahub/route.openshift.io/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.175771466Z namespaces/opendatahub/route.openshift.io/routes.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.175827947Z namespaces/openshift/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.175906679Z namespaces/openshift/openshift.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.175956741Z namespaces/openshift/apps.openshift.io/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.176010732Z namespaces/openshift/apps.openshift.io/deploymentconfigs.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.176062753Z namespaces/openshift/apps/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.176106765Z namespaces/openshift/apps/daemonsets.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.176204517Z namespaces/openshift/apps/deployments.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.176288719Z namespaces/openshift/apps/replicasets.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.176383022Z namespaces/openshift/apps/statefulsets.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.176422623Z namespaces/openshift/autoscaling/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.176467674Z namespaces/openshift/autoscaling/horizontalpodautoscalers.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.176525516Z namespaces/openshift/batch/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.176579337Z namespaces/openshift/batch/cronjobs.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.176675569Z namespaces/openshift/batch/jobs.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.176715661Z namespaces/openshift/build.openshift.io/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.176777132Z namespaces/openshift/build.openshift.io/buildconfigs.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.176883985Z namespaces/openshift/build.openshift.io/builds.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.176936346Z namespaces/openshift/core/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.176976447Z namespaces/openshift/core/configmaps.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.17709108Z namespaces/openshift/core/endpoints.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.177176893Z namespaces/openshift/core/events.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.177270775Z namespaces/openshift/core/persistentvolumeclaims.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.177376098Z namespaces/openshift/core/pods.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.17745763Z namespaces/openshift/core/replicationcontrollers.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.177598524Z namespaces/openshift/core/secrets.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.177766918Z namespaces/openshift/core/services.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.17783327Z namespaces/openshift/discovery.k8s.io/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.177895791Z namespaces/openshift/discovery.k8s.io/endpointslices.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.177954703Z namespaces/openshift/image.openshift.io/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.178003124Z namespaces/openshift/image.openshift.io/imagestreams.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.179598566Z namespaces/openshift/k8s.ovn.org/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.179639047Z namespaces/openshift/k8s.ovn.org/egressfirewalls.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.17974843Z namespaces/openshift/k8s.ovn.org/egressqoses.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.179792571Z namespaces/openshift/monitoring.coreos.com/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.179868943Z namespaces/openshift/monitoring.coreos.com/servicemonitors.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.179915754Z namespaces/openshift/networking.k8s.io/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.179966215Z namespaces/openshift/networking.k8s.io/networkpolicies.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.180012347Z namespaces/openshift/policy/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.180059808Z namespaces/openshift/policy/poddisruptionbudgets.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.180118699Z namespaces/openshift/route.openshift.io/ [must-gather-px7mm] OUT 2026-10-31T08:14:04.180167971Z namespaces/openshift/route.openshift.io/routes.yaml [must-gather-px7mm] OUT 2026-10-31T08:14:04.310557987Z [must-gather-px7mm] OUT 2026-10-31T08:14:04.310593978Z sent 1,380 bytes received 72,788 bytes 49,445.33 bytes/sec [must-gather-px7mm] OUT 2026-10-31T08:14:04.310612908Z total size is 874,803 speedup is 11.79 [must-gather ] OUT 2026-10-31T08:14:04.415439606Z namespace/openshift-must-gather-njhxc deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: ed1fcdb5-a2c2-4308-9b9b-93581ee4770e ClientVersion: 4.22.15 ClusterVersion: Stable at "4.22.20" ClusterOperators: clusteroperator/cloud-credential is not upgradeable because Upgradeable annotation cloudcredential.openshift.io/upgradeable-to on cloudcredential.operator.openshift.io/cluster object needs updating before upgrade. See Manually Creating IAM documentation for instructions on preparing a cluster for upgrade. clusteroperator/authentication is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing