[must-gather ] OUT 2026-10-30T20:51:49.117987072Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: edcd47ed-099f-43a2-848c-15b4e6653454 ClientVersion: 4.22.15 ClusterVersion: Stable at "4.22.20" ClusterOperators: clusteroperator/cloud-credential is not upgradeable because Upgradeable annotation cloudcredential.openshift.io/upgradeable-to on cloudcredential.operator.openshift.io/cluster object needs updating before upgrade. See Manually Creating IAM documentation for instructions on preparing a cluster for upgrade. clusteroperator/authentication is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing [must-gather ] OUT 2026-10-30T20:51:49.242653296Z namespace/openshift-must-gather-pc87s created [must-gather ] OUT 2026-10-30T20:51:49.259857405Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-7m2mm created [must-gather ] OUT 2026-10-30T20:51:49.30002071Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather-2hm78] OUT 2026-10-30T20:52:50.757497272Z waiting for gather to complete [must-gather-2hm78] OUT 2026-10-30T20:52:50.770288286Z downloading gather output [must-gather-2hm78] OUT 2026-10-30T20:52:51.173478828Z receiving incremental file list [must-gather-2hm78] OUT 2026-10-30T20:52:51.20814672Z ./ [must-gather-2hm78] OUT 2026-10-30T20:52:51.208262282Z aggregated-discovery-api.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.208869637Z aggregated-discovery-apis.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.209732197Z event-filter.html [must-gather-2hm78] OUT 2026-10-30T20:52:51.220713588Z timestamp [must-gather-2hm78] OUT 2026-10-30T20:52:51.220864112Z version [must-gather-2hm78] OUT 2026-10-30T20:52:51.220992024Z namespaces/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.221002685Z namespaces/openshift/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.221084347Z namespaces/openshift/openshift.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.221193989Z namespaces/openshift/apps.openshift.io/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.221256831Z namespaces/openshift/apps.openshift.io/deploymentconfigs.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.221331843Z namespaces/openshift/apps/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.221392904Z namespaces/openshift/apps/daemonsets.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.221514357Z namespaces/openshift/apps/deployments.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.22166056Z namespaces/openshift/apps/replicasets.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.221782813Z namespaces/openshift/apps/statefulsets.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.221831894Z namespaces/openshift/autoscaling/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.221864155Z namespaces/openshift/autoscaling/horizontalpodautoscalers.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.221928957Z namespaces/openshift/batch/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.221973238Z namespaces/openshift/batch/cronjobs.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.22206714Z namespaces/openshift/batch/jobs.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.222121761Z namespaces/openshift/build.openshift.io/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.222167042Z namespaces/openshift/build.openshift.io/buildconfigs.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.222270635Z namespaces/openshift/build.openshift.io/builds.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.222310126Z namespaces/openshift/core/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.222373417Z namespaces/openshift/core/configmaps.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.22249773Z namespaces/openshift/core/endpoints.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.222591012Z namespaces/openshift/core/events.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.222710585Z namespaces/openshift/core/persistentvolumeclaims.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.222804967Z namespaces/openshift/core/pods.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.222892949Z namespaces/openshift/core/replicationcontrollers.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.223030483Z namespaces/openshift/core/secrets.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.223207507Z namespaces/openshift/core/services.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.223264828Z namespaces/openshift/discovery.k8s.io/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.22332068Z namespaces/openshift/discovery.k8s.io/endpointslices.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.223377041Z namespaces/openshift/image.openshift.io/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.223418892Z namespaces/openshift/image.openshift.io/imagestreams.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.235892008Z namespaces/openshift/k8s.ovn.org/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.235928749Z namespaces/openshift/k8s.ovn.org/egressfirewalls.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.236029931Z namespaces/openshift/k8s.ovn.org/egressqoses.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.236088133Z namespaces/openshift/monitoring.coreos.com/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.236140674Z namespaces/openshift/monitoring.coreos.com/servicemonitors.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.236196755Z namespaces/openshift/networking.k8s.io/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.236250066Z namespaces/openshift/networking.k8s.io/networkpolicies.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.236303648Z namespaces/openshift/policy/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.236354979Z namespaces/openshift/policy/poddisruptionbudgets.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.23640092Z namespaces/openshift/route.openshift.io/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.236457041Z namespaces/openshift/route.openshift.io/routes.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.236511783Z namespaces/redhat-ods-applications/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.236568914Z namespaces/redhat-ods-applications/redhat-ods-applications.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.236667566Z namespaces/redhat-ods-applications/apps.openshift.io/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.236714817Z namespaces/redhat-ods-applications/apps.openshift.io/deploymentconfigs.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.236778489Z namespaces/redhat-ods-applications/apps/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.23682549Z namespaces/redhat-ods-applications/apps/daemonsets.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.236919722Z namespaces/redhat-ods-applications/apps/deployments.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.237053765Z namespaces/redhat-ods-applications/apps/replicasets.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.237195059Z namespaces/redhat-ods-applications/apps/statefulsets.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.23724981Z namespaces/redhat-ods-applications/autoscaling/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.237304271Z namespaces/redhat-ods-applications/autoscaling/horizontalpodautoscalers.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.237358953Z namespaces/redhat-ods-applications/batch/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.237413534Z namespaces/redhat-ods-applications/batch/cronjobs.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.237505366Z namespaces/redhat-ods-applications/batch/jobs.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.237562848Z namespaces/redhat-ods-applications/build.openshift.io/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.237622459Z namespaces/redhat-ods-applications/build.openshift.io/buildconfigs.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.237731652Z namespaces/redhat-ods-applications/build.openshift.io/builds.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.237775802Z namespaces/redhat-ods-applications/core/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.237832764Z namespaces/redhat-ods-applications/core/configmaps.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.237969447Z namespaces/redhat-ods-applications/core/endpoints.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.23807336Z namespaces/redhat-ods-applications/core/events.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.238221823Z namespaces/redhat-ods-applications/core/persistentvolumeclaims.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.238311175Z namespaces/redhat-ods-applications/core/pods.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.238452629Z namespaces/redhat-ods-applications/core/replicationcontrollers.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.238600792Z namespaces/redhat-ods-applications/core/secrets.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.238807717Z namespaces/redhat-ods-applications/core/services.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.238878699Z namespaces/redhat-ods-applications/discovery.k8s.io/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.23893658Z namespaces/redhat-ods-applications/discovery.k8s.io/endpointslices.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.238996391Z namespaces/redhat-ods-applications/image.openshift.io/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.239051523Z namespaces/redhat-ods-applications/image.openshift.io/imagestreams.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.239178236Z namespaces/redhat-ods-applications/k8s.ovn.org/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.239219787Z namespaces/redhat-ods-applications/k8s.ovn.org/egressfirewalls.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.239321129Z namespaces/redhat-ods-applications/k8s.ovn.org/egressqoses.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.23937154Z namespaces/redhat-ods-applications/monitoring.coreos.com/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.239411311Z namespaces/redhat-ods-applications/monitoring.coreos.com/servicemonitors.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.239477603Z namespaces/redhat-ods-applications/networking.k8s.io/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.239529024Z namespaces/redhat-ods-applications/networking.k8s.io/networkpolicies.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.239618206Z namespaces/redhat-ods-applications/pods/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.239629977Z namespaces/redhat-ods-applications/pods/kubeflow-trainer-controller-manager-79f7d5b57-t96bt/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.239682138Z namespaces/redhat-ods-applications/pods/kubeflow-trainer-controller-manager-79f7d5b57-t96bt/kubeflow-trainer-controller-manager-79f7d5b57-t96bt.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.23979509Z namespaces/redhat-ods-applications/pods/kubeflow-trainer-controller-manager-79f7d5b57-t96bt/manager/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.239806761Z namespaces/redhat-ods-applications/pods/kubeflow-trainer-controller-manager-79f7d5b57-t96bt/manager/manager/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.239812371Z namespaces/redhat-ods-applications/pods/kubeflow-trainer-controller-manager-79f7d5b57-t96bt/manager/manager/logs/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.239855832Z namespaces/redhat-ods-applications/pods/kubeflow-trainer-controller-manager-79f7d5b57-t96bt/manager/manager/logs/current.log [must-gather-2hm78] OUT 2026-10-30T20:52:51.241257475Z namespaces/redhat-ods-applications/pods/kubeflow-trainer-controller-manager-79f7d5b57-t96bt/manager/manager/logs/previous.insecure.log [must-gather-2hm78] OUT 2026-10-30T20:52:51.241345087Z namespaces/redhat-ods-applications/pods/kubeflow-trainer-controller-manager-79f7d5b57-t96bt/manager/manager/logs/previous.log [must-gather-2hm78] OUT 2026-10-30T20:52:51.241379168Z namespaces/redhat-ods-applications/policy/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.241444089Z namespaces/redhat-ods-applications/policy/poddisruptionbudgets.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.24148244Z namespaces/redhat-ods-applications/route.openshift.io/ [must-gather-2hm78] OUT 2026-10-30T20:52:51.241545852Z namespaces/redhat-ods-applications/route.openshift.io/routes.yaml [must-gather-2hm78] OUT 2026-10-30T20:52:51.272214589Z [must-gather-2hm78] OUT 2026-10-30T20:52:51.2722341Z sent 1,376 bytes received 100,605 bytes 67,987.33 bytes/sec [must-gather-2hm78] OUT 2026-10-30T20:52:51.27224419Z total size is 1,267,329 speedup is 12.43 [must-gather ] OUT 2026-10-30T20:52:51.334570608Z namespace/openshift-must-gather-pc87s deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: edcd47ed-099f-43a2-848c-15b4e6653454 ClientVersion: 4.22.15 ClusterVersion: Stable at "4.22.20" ClusterOperators: clusteroperator/cloud-credential is not upgradeable because Upgradeable annotation cloudcredential.openshift.io/upgradeable-to on cloudcredential.operator.openshift.io/cluster object needs updating before upgrade. See Manually Creating IAM documentation for instructions on preparing a cluster for upgrade. clusteroperator/authentication is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing