[must-gather ] OUT 2026-11-02T23:44:56.929188822Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 0e18d20e-7633-4937-905a-1a9375949fdf ClientVersion: 4.22.15 ClusterVersion: Stable at "4.22.20" ClusterOperators: clusteroperator/cloud-credential is not upgradeable because Upgradeable annotation cloudcredential.openshift.io/upgradeable-to on cloudcredential.operator.openshift.io/cluster object needs updating before upgrade. See Manually Creating IAM documentation for instructions on preparing a cluster for upgrade. clusteroperator/authentication is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing [must-gather ] OUT 2026-11-02T23:44:57.047957215Z namespace/openshift-must-gather-xgfsg created [must-gather ] OUT 2026-11-02T23:44:57.066028656Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-lfmtg created [must-gather ] OUT 2026-11-02T23:44:57.104696021Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather-thslq] OUT 2026-11-02T23:48:16.309101721Z waiting for gather to complete [must-gather-thslq] OUT 2026-11-02T23:48:16.322590585Z downloading gather output [must-gather-thslq] OUT 2026-11-02T23:48:17.247907162Z receiving incremental file list [must-gather-thslq] OUT 2026-11-02T23:48:17.382792939Z ./ [must-gather-thslq] OUT 2026-11-02T23:48:17.382896361Z aggregated-discovery-api.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.383942403Z aggregated-discovery-apis.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.384604697Z event-filter.html [must-gather-thslq] OUT 2026-11-02T23:48:17.38473591Z timestamp [must-gather-thslq] OUT 2026-11-02T23:48:17.396459036Z version [must-gather-thslq] OUT 2026-11-02T23:48:17.39663299Z namespaces/ [must-gather-thslq] OUT 2026-11-02T23:48:17.39664845Z namespaces/opendatahub/ [must-gather-thslq] OUT 2026-11-02T23:48:17.396724022Z namespaces/opendatahub/opendatahub.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.396841644Z namespaces/opendatahub/apps.openshift.io/ [must-gather-thslq] OUT 2026-11-02T23:48:17.396903965Z namespaces/opendatahub/apps.openshift.io/deploymentconfigs.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.396964137Z namespaces/opendatahub/apps/ [must-gather-thslq] OUT 2026-11-02T23:48:17.397030898Z namespaces/opendatahub/apps/daemonsets.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.397170921Z namespaces/opendatahub/apps/deployments.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.397318874Z namespaces/opendatahub/apps/replicasets.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.397458067Z namespaces/opendatahub/apps/statefulsets.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.397511608Z namespaces/opendatahub/autoscaling/ [must-gather-thslq] OUT 2026-11-02T23:48:17.39757676Z namespaces/opendatahub/autoscaling/horizontalpodautoscalers.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.397649691Z namespaces/opendatahub/batch/ [must-gather-thslq] OUT 2026-11-02T23:48:17.397715903Z namespaces/opendatahub/batch/cronjobs.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.397821525Z namespaces/opendatahub/batch/jobs.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.397878606Z namespaces/opendatahub/build.openshift.io/ [must-gather-thslq] OUT 2026-11-02T23:48:17.397938707Z namespaces/opendatahub/build.openshift.io/buildconfigs.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.398037179Z namespaces/opendatahub/build.openshift.io/builds.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.398120671Z namespaces/opendatahub/core/ [must-gather-thslq] OUT 2026-11-02T23:48:17.398175812Z namespaces/opendatahub/core/configmaps.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.398333276Z namespaces/opendatahub/core/endpoints.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.398436158Z namespaces/opendatahub/core/events.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.398582121Z namespaces/opendatahub/core/persistentvolumeclaims.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.398685033Z namespaces/opendatahub/core/pods.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.398859887Z namespaces/opendatahub/core/replicationcontrollers.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.39901146Z namespaces/opendatahub/core/secrets.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.399250865Z namespaces/opendatahub/core/services.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.399316366Z namespaces/opendatahub/discovery.k8s.io/ [must-gather-thslq] OUT 2026-11-02T23:48:17.399378958Z namespaces/opendatahub/discovery.k8s.io/endpointslices.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.399451759Z namespaces/opendatahub/image.openshift.io/ [must-gather-thslq] OUT 2026-11-02T23:48:17.39950235Z namespaces/opendatahub/image.openshift.io/imagestreams.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.410391639Z namespaces/opendatahub/k8s.ovn.org/ [must-gather-thslq] OUT 2026-11-02T23:48:17.410483181Z namespaces/opendatahub/k8s.ovn.org/egressfirewalls.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.410639394Z namespaces/opendatahub/k8s.ovn.org/egressqoses.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.410687175Z namespaces/opendatahub/monitoring.coreos.com/ [must-gather-thslq] OUT 2026-11-02T23:48:17.410744297Z namespaces/opendatahub/monitoring.coreos.com/servicemonitors.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.410833098Z namespaces/opendatahub/networking.k8s.io/ [must-gather-thslq] OUT 2026-11-02T23:48:17.41090206Z namespaces/opendatahub/networking.k8s.io/networkpolicies.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.410979662Z namespaces/opendatahub/pods/ [must-gather-thslq] OUT 2026-11-02T23:48:17.410987442Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84b74c44f8-98rwn/ [must-gather-thslq] OUT 2026-11-02T23:48:17.411050523Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84b74c44f8-98rwn/kubeflow-trainer-controller-manager-84b74c44f8-98rwn.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.411186286Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84b74c44f8-98rwn/manager/ [must-gather-thslq] OUT 2026-11-02T23:48:17.411203076Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84b74c44f8-98rwn/manager/manager/ [must-gather-thslq] OUT 2026-11-02T23:48:17.411209716Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84b74c44f8-98rwn/manager/manager/logs/ [must-gather-thslq] OUT 2026-11-02T23:48:17.411261387Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84b74c44f8-98rwn/manager/manager/logs/current.log [must-gather-thslq] OUT 2026-11-02T23:48:17.440563353Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84b74c44f8-98rwn/manager/manager/logs/previous.insecure.log [must-gather-thslq] OUT 2026-11-02T23:48:17.440663966Z namespaces/opendatahub/pods/kubeflow-trainer-controller-manager-84b74c44f8-98rwn/manager/manager/logs/previous.log [must-gather-thslq] OUT 2026-11-02T23:48:17.440703416Z namespaces/opendatahub/policy/ [must-gather-thslq] OUT 2026-11-02T23:48:17.440779078Z namespaces/opendatahub/policy/poddisruptionbudgets.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.440835709Z namespaces/opendatahub/route.openshift.io/ [must-gather-thslq] OUT 2026-11-02T23:48:17.44089015Z namespaces/opendatahub/route.openshift.io/routes.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.440980932Z namespaces/openshift/ [must-gather-thslq] OUT 2026-11-02T23:48:17.441099555Z namespaces/openshift/openshift.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.441201017Z namespaces/openshift/apps.openshift.io/ [must-gather-thslq] OUT 2026-11-02T23:48:17.441280188Z namespaces/openshift/apps.openshift.io/deploymentconfigs.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.44135817Z namespaces/openshift/apps/ [must-gather-thslq] OUT 2026-11-02T23:48:17.441438342Z namespaces/openshift/apps/daemonsets.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.441581015Z namespaces/openshift/apps/deployments.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.441710588Z namespaces/openshift/apps/replicasets.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.441847391Z namespaces/openshift/apps/statefulsets.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.441917432Z namespaces/openshift/autoscaling/ [must-gather-thslq] OUT 2026-11-02T23:48:17.442006894Z namespaces/openshift/autoscaling/horizontalpodautoscalers.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.442122816Z namespaces/openshift/batch/ [must-gather-thslq] OUT 2026-11-02T23:48:17.442199548Z namespaces/openshift/batch/cronjobs.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.442319601Z namespaces/openshift/batch/jobs.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.442376122Z namespaces/openshift/build.openshift.io/ [must-gather-thslq] OUT 2026-11-02T23:48:17.442435163Z namespaces/openshift/build.openshift.io/buildconfigs.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.442536015Z namespaces/openshift/build.openshift.io/builds.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.442590626Z namespaces/openshift/core/ [must-gather-thslq] OUT 2026-11-02T23:48:17.442653888Z namespaces/openshift/core/configmaps.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.442806701Z namespaces/openshift/core/endpoints.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.442949374Z namespaces/openshift/core/events.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.443104057Z namespaces/openshift/core/persistentvolumeclaims.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.44325334Z namespaces/openshift/core/pods.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.443382483Z namespaces/openshift/core/replicationcontrollers.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.443620898Z namespaces/openshift/core/secrets.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.443875493Z namespaces/openshift/core/services.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.443926354Z namespaces/openshift/discovery.k8s.io/ [must-gather-thslq] OUT 2026-11-02T23:48:17.443997796Z namespaces/openshift/discovery.k8s.io/endpointslices.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.444135649Z namespaces/openshift/image.openshift.io/ [must-gather-thslq] OUT 2026-11-02T23:48:17.44420413Z namespaces/openshift/image.openshift.io/imagestreams.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.446332885Z namespaces/openshift/k8s.ovn.org/ [must-gather-thslq] OUT 2026-11-02T23:48:17.446395426Z namespaces/openshift/k8s.ovn.org/egressfirewalls.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.446503628Z namespaces/openshift/k8s.ovn.org/egressqoses.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.44656013Z namespaces/openshift/monitoring.coreos.com/ [must-gather-thslq] OUT 2026-11-02T23:48:17.446619591Z namespaces/openshift/monitoring.coreos.com/servicemonitors.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.446672632Z namespaces/openshift/networking.k8s.io/ [must-gather-thslq] OUT 2026-11-02T23:48:17.446734023Z namespaces/openshift/networking.k8s.io/networkpolicies.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.446826005Z namespaces/openshift/policy/ [must-gather-thslq] OUT 2026-11-02T23:48:17.446879476Z namespaces/openshift/policy/poddisruptionbudgets.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.446940258Z namespaces/openshift/route.openshift.io/ [must-gather-thslq] OUT 2026-11-02T23:48:17.447003379Z namespaces/openshift/route.openshift.io/routes.yaml [must-gather-thslq] OUT 2026-11-02T23:48:17.580186829Z [must-gather-thslq] OUT 2026-11-02T23:48:17.58022054Z sent 1,380 bytes received 154,303 bytes 103,788.67 bytes/sec [must-gather-thslq] OUT 2026-11-02T23:48:17.5802274Z total size is 1,898,983 speedup is 12.20 [must-gather ] OUT 2026-11-02T23:48:17.681826177Z namespace/openshift-must-gather-xgfsg deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 0e18d20e-7633-4937-905a-1a9375949fdf ClientVersion: 4.22.15 ClusterVersion: Stable at "4.22.20" ClusterOperators: clusteroperator/cloud-credential is not upgradeable because Upgradeable annotation cloudcredential.openshift.io/upgradeable-to on cloudcredential.operator.openshift.io/cluster object needs updating before upgrade. See Manually Creating IAM documentation for instructions on preparing a cluster for upgrade. clusteroperator/authentication is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing