[must-gather ] OUT 2026-11-03T07:13:36.526572062Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 93ecd85f-cd03-42b2-96a0-fdc00aad5716 ClientVersion: 4.22.15 ClusterVersion: Stable at "4.22.20" ClusterOperators: clusteroperator/cloud-credential is not upgradeable because Upgradeable annotation cloudcredential.openshift.io/upgradeable-to on cloudcredential.operator.openshift.io/cluster object needs updating before upgrade. See Manually Creating IAM documentation for instructions on preparing a cluster for upgrade. clusteroperator/authentication is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing [must-gather ] OUT 2026-11-03T07:13:36.654663434Z namespace/openshift-must-gather-j9l4d created [must-gather ] OUT 2026-11-03T07:13:36.670698232Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-bl6x6 created [must-gather ] OUT 2026-11-03T07:13:36.714764074Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather-vx7jn] OUT 2026-11-03T07:15:07.657276133Z waiting for gather to complete [must-gather-vx7jn] OUT 2026-11-03T07:15:07.672420959Z downloading gather output [must-gather-vx7jn] OUT 2026-11-03T07:15:08.190446639Z receiving incremental file list [must-gather-vx7jn] OUT 2026-11-03T07:15:08.245952839Z ./ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.246018941Z aggregated-discovery-api.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.246795821Z aggregated-discovery-apis.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.247410088Z event-filter.html [must-gather-vx7jn] OUT 2026-11-03T07:15:08.247529251Z timestamp [must-gather-vx7jn] OUT 2026-11-03T07:15:08.258637041Z version [must-gather-vx7jn] OUT 2026-11-03T07:15:08.258769284Z namespaces/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.258779185Z namespaces/openshift/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.258859257Z namespaces/openshift/openshift.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.258949129Z namespaces/openshift/apps.openshift.io/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.25900334Z namespaces/openshift/apps.openshift.io/deploymentconfigs.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.259087993Z namespaces/openshift/apps/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.259161465Z namespaces/openshift/apps/daemonsets.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.259258767Z namespaces/openshift/apps/deployments.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.25934796Z namespaces/openshift/apps/replicasets.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.259439022Z namespaces/openshift/apps/statefulsets.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.259489083Z namespaces/openshift/autoscaling/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.259542945Z namespaces/openshift/autoscaling/horizontalpodautoscalers.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.259589976Z namespaces/openshift/batch/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.259642947Z namespaces/openshift/batch/cronjobs.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.259727769Z namespaces/openshift/batch/jobs.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.259780541Z namespaces/openshift/build.openshift.io/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.259834372Z namespaces/openshift/build.openshift.io/buildconfigs.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.259930355Z namespaces/openshift/build.openshift.io/builds.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.259979366Z namespaces/openshift/core/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.260032097Z namespaces/openshift/core/configmaps.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.260182851Z namespaces/openshift/core/endpoints.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.260277634Z namespaces/openshift/core/events.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.260365116Z namespaces/openshift/core/persistentvolumeclaims.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.260457498Z namespaces/openshift/core/pods.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.260541991Z namespaces/openshift/core/replicationcontrollers.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.260672824Z namespaces/openshift/core/secrets.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.260851279Z namespaces/openshift/core/services.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.26090755Z namespaces/openshift/discovery.k8s.io/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.260963222Z namespaces/openshift/discovery.k8s.io/endpointslices.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.261014023Z namespaces/openshift/image.openshift.io/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.261087415Z namespaces/openshift/image.openshift.io/imagestreams.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.273036057Z namespaces/openshift/k8s.ovn.org/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.27313081Z namespaces/openshift/k8s.ovn.org/egressfirewalls.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.273233842Z namespaces/openshift/k8s.ovn.org/egressqoses.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.273292844Z namespaces/openshift/monitoring.coreos.com/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.273344585Z namespaces/openshift/monitoring.coreos.com/servicemonitors.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.273392216Z namespaces/openshift/networking.k8s.io/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.273436978Z namespaces/openshift/networking.k8s.io/networkpolicies.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.273486969Z namespaces/openshift/policy/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.273567161Z namespaces/openshift/policy/poddisruptionbudgets.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.273621022Z namespaces/openshift/route.openshift.io/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.273674834Z namespaces/openshift/route.openshift.io/routes.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.273719295Z namespaces/redhat-ods-applications/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.273784857Z namespaces/redhat-ods-applications/redhat-ods-applications.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.273849859Z namespaces/redhat-ods-applications/apps.openshift.io/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.27389883Z namespaces/redhat-ods-applications/apps.openshift.io/deploymentconfigs.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.273958371Z namespaces/redhat-ods-applications/apps/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.273993552Z namespaces/redhat-ods-applications/apps/daemonsets.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.274125356Z namespaces/redhat-ods-applications/apps/deployments.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.27427797Z namespaces/redhat-ods-applications/apps/replicasets.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.274401173Z namespaces/redhat-ods-applications/apps/statefulsets.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.274451214Z namespaces/redhat-ods-applications/autoscaling/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.274515706Z namespaces/redhat-ods-applications/autoscaling/horizontalpodautoscalers.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.274556327Z namespaces/redhat-ods-applications/batch/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.274605348Z namespaces/redhat-ods-applications/batch/cronjobs.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.274696751Z namespaces/redhat-ods-applications/batch/jobs.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.274747492Z namespaces/redhat-ods-applications/build.openshift.io/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.274809084Z namespaces/redhat-ods-applications/build.openshift.io/buildconfigs.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.274885755Z namespaces/redhat-ods-applications/build.openshift.io/builds.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.274942957Z namespaces/redhat-ods-applications/core/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.274987718Z namespaces/redhat-ods-applications/core/configmaps.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.275276996Z namespaces/redhat-ods-applications/core/endpoints.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.275375268Z namespaces/redhat-ods-applications/core/events.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.275516692Z namespaces/redhat-ods-applications/core/persistentvolumeclaims.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.275610565Z namespaces/redhat-ods-applications/core/pods.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.275744838Z namespaces/redhat-ods-applications/core/replicationcontrollers.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.275886942Z namespaces/redhat-ods-applications/core/secrets.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.276097957Z namespaces/redhat-ods-applications/core/services.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.276178129Z namespaces/redhat-ods-applications/discovery.k8s.io/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.276228691Z namespaces/redhat-ods-applications/discovery.k8s.io/endpointslices.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.276298312Z namespaces/redhat-ods-applications/image.openshift.io/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.276344984Z namespaces/redhat-ods-applications/image.openshift.io/imagestreams.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.276470067Z namespaces/redhat-ods-applications/k8s.ovn.org/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.276511798Z namespaces/redhat-ods-applications/k8s.ovn.org/egressfirewalls.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.276604451Z namespaces/redhat-ods-applications/k8s.ovn.org/egressqoses.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.276653322Z namespaces/redhat-ods-applications/monitoring.coreos.com/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.276708923Z namespaces/redhat-ods-applications/monitoring.coreos.com/servicemonitors.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.276767895Z namespaces/redhat-ods-applications/networking.k8s.io/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.276819426Z namespaces/redhat-ods-applications/networking.k8s.io/networkpolicies.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.276880098Z namespaces/redhat-ods-applications/pods/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.276887208Z namespaces/redhat-ods-applications/pods/kubeflow-trainer-controller-manager-6b557f7d55-mfhhb/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.276941719Z namespaces/redhat-ods-applications/pods/kubeflow-trainer-controller-manager-6b557f7d55-mfhhb/kubeflow-trainer-controller-manager-6b557f7d55-mfhhb.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.277033642Z namespaces/redhat-ods-applications/pods/kubeflow-trainer-controller-manager-6b557f7d55-mfhhb/manager/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.277040682Z namespaces/redhat-ods-applications/pods/kubeflow-trainer-controller-manager-6b557f7d55-mfhhb/manager/manager/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.277044892Z namespaces/redhat-ods-applications/pods/kubeflow-trainer-controller-manager-6b557f7d55-mfhhb/manager/manager/logs/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.277131534Z namespaces/redhat-ods-applications/pods/kubeflow-trainer-controller-manager-6b557f7d55-mfhhb/manager/manager/logs/current.log [must-gather-vx7jn] OUT 2026-11-03T07:15:08.286434548Z namespaces/redhat-ods-applications/pods/kubeflow-trainer-controller-manager-6b557f7d55-mfhhb/manager/manager/logs/previous.insecure.log [must-gather-vx7jn] OUT 2026-11-03T07:15:08.28652614Z namespaces/redhat-ods-applications/pods/kubeflow-trainer-controller-manager-6b557f7d55-mfhhb/manager/manager/logs/previous.log [must-gather-vx7jn] OUT 2026-11-03T07:15:08.286575401Z namespaces/redhat-ods-applications/policy/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.286628953Z namespaces/redhat-ods-applications/policy/poddisruptionbudgets.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.286682394Z namespaces/redhat-ods-applications/route.openshift.io/ [must-gather-vx7jn] OUT 2026-11-03T07:15:08.286737505Z namespaces/redhat-ods-applications/route.openshift.io/routes.yaml [must-gather-vx7jn] OUT 2026-11-03T07:15:08.340048019Z [must-gather-vx7jn] OUT 2026-11-03T07:15:08.34009897Z sent 1,376 bytes received 149,120 bytes 100,330.67 bytes/sec [must-gather-vx7jn] OUT 2026-11-03T07:15:08.34010611Z total size is 1,879,758 speedup is 12.49 [must-gather ] OUT 2026-11-03T07:15:08.403785225Z namespace/openshift-must-gather-j9l4d deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 93ecd85f-cd03-42b2-96a0-fdc00aad5716 ClientVersion: 4.22.15 ClusterVersion: Stable at "4.22.20" ClusterOperators: clusteroperator/cloud-credential is not upgradeable because Upgradeable annotation cloudcredential.openshift.io/upgradeable-to on cloudcredential.operator.openshift.io/cluster object needs updating before upgrade. See Manually Creating IAM documentation for instructions on preparing a cluster for upgrade. clusteroperator/authentication is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing