[must-gather ] OUT 2026-09-17T16:23:58.608382472Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 8e49930d-f8de-4713-bbae-7e7c7aec5a46 ClientVersion: 4.21.10 ClusterVersion: Stable at "4.21.34" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing [must-gather ] OUT 2026-09-17T16:23:58.654650709Z namespace/openshift-must-gather-r5xzc created [must-gather ] OUT 2026-09-17T16:23:58.664560484Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-xmkzp created [must-gather ] OUT 2026-09-17T16:23:58.709521787Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather-n8c46] OUT 2026-09-17T16:24:29.557114211Z waiting for gather to complete [must-gather-n8c46] OUT 2026-09-17T16:24:29.560296406Z downloading gather output [must-gather-n8c46] OUT 2026-09-17T16:24:29.858192992Z receiving incremental file list [must-gather-n8c46] OUT 2026-09-17T16:24:29.871520168Z ./ [must-gather-n8c46] OUT 2026-09-17T16:24:29.871614141Z aggregated-discovery-api.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.8719576Z aggregated-discovery-apis.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.872895635Z event-filter.html [must-gather-n8c46] OUT 2026-09-17T16:24:29.87420632Z timestamp [must-gather-n8c46] OUT 2026-09-17T16:24:29.874363964Z version [must-gather-n8c46] OUT 2026-09-17T16:24:29.875384672Z cluster-scoped-resources/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.875400782Z cluster-scoped-resources/datasciencecluster.opendatahub.io/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.875407962Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.875451183Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/test-dsc.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.875545086Z cluster-scoped-resources/dscinitialization.opendatahub.io/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.875557806Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.875614918Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/test-dsci.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.875862134Z namespaces/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.875877055Z namespaces/kserve-ci-e2e-test/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.875887905Z namespaces/kserve-ci-e2e-test/kserve-ci-e2e-test.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.875893735Z namespaces/kserve-ci-e2e-test/apps.openshift.io/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.875924146Z namespaces/kserve-ci-e2e-test/apps.openshift.io/deploymentconfigs.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.875981468Z namespaces/kserve-ci-e2e-test/apps/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.876036779Z namespaces/kserve-ci-e2e-test/apps/daemonsets.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.876142412Z namespaces/kserve-ci-e2e-test/apps/deployments.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.876240805Z namespaces/kserve-ci-e2e-test/apps/replicasets.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.876329557Z namespaces/kserve-ci-e2e-test/apps/statefulsets.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.876384139Z namespaces/kserve-ci-e2e-test/autoscaling/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.87643674Z namespaces/kserve-ci-e2e-test/autoscaling/horizontalpodautoscalers.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.876500942Z namespaces/kserve-ci-e2e-test/batch/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.876546053Z namespaces/kserve-ci-e2e-test/batch/cronjobs.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.876655276Z namespaces/kserve-ci-e2e-test/batch/jobs.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.876703657Z namespaces/kserve-ci-e2e-test/build.openshift.io/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.876739898Z namespaces/kserve-ci-e2e-test/build.openshift.io/buildconfigs.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.876862531Z namespaces/kserve-ci-e2e-test/build.openshift.io/builds.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.876927063Z namespaces/kserve-ci-e2e-test/core/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.876974724Z namespaces/kserve-ci-e2e-test/core/configmaps.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.877128498Z namespaces/kserve-ci-e2e-test/core/endpoints.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.877217381Z namespaces/kserve-ci-e2e-test/core/events.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.880264482Z namespaces/kserve-ci-e2e-test/core/persistentvolumeclaims.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.880351305Z namespaces/kserve-ci-e2e-test/core/pods.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.880596901Z namespaces/kserve-ci-e2e-test/core/replicationcontrollers.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.880742425Z namespaces/kserve-ci-e2e-test/core/secrets.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.880984041Z namespaces/kserve-ci-e2e-test/core/services.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.881036603Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.881097514Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/endpointslices.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.881154756Z namespaces/kserve-ci-e2e-test/image.openshift.io/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.881212248Z namespaces/kserve-ci-e2e-test/image.openshift.io/imagestreams.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.881266869Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.881322591Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressfirewalls.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.881419733Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressqoses.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.881473375Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.881527776Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/servicemonitors.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.881582578Z namespaces/kserve-ci-e2e-test/networking.k8s.io/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.881635119Z namespaces/kserve-ci-e2e-test/networking.k8s.io/networkpolicies.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.88168969Z namespaces/kserve-ci-e2e-test/pods/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.88169749Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.881754752Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.881981818Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/agent/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.881993778Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/agent/agent/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.881997819Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/agent/agent/logs/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.8820434Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/agent/agent/logs/current.log [must-gather-n8c46] OUT 2026-09-17T16:24:29.882154383Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/agent/agent/logs/previous.insecure.log [must-gather-n8c46] OUT 2026-09-17T16:24:29.882238105Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/agent/agent/logs/previous.log [must-gather-n8c46] OUT 2026-09-17T16:24:29.882284006Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/kserve-container/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.882303397Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/kserve-container/kserve-container/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.882309687Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/kserve-container/kserve-container/logs/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.882340178Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/kserve-container/kserve-container/logs/current.log [must-gather-n8c46] OUT 2026-09-17T16:24:29.88243781Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/kserve-container/kserve-container/logs/previous.insecure.log [must-gather-n8c46] OUT 2026-09-17T16:24:29.882521183Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/kserve-container/kserve-container/logs/previous.log [must-gather-n8c46] OUT 2026-09-17T16:24:29.882560384Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/kube-rbac-proxy/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.882566894Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/kube-rbac-proxy/kube-rbac-proxy/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.882570814Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.882623365Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather-n8c46] OUT 2026-09-17T16:24:29.882735728Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather-n8c46] OUT 2026-09-17T16:24:29.882861562Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather-n8c46] OUT 2026-09-17T16:24:29.882917993Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/storage-initializer/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.882926273Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/storage-initializer/storage-initializer/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.882931183Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/storage-initializer/storage-initializer/logs/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.882976265Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/storage-initializer/storage-initializer/logs/current.log [must-gather-n8c46] OUT 2026-09-17T16:24:29.883079127Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/storage-initializer/storage-initializer/logs/previous.insecure.log [must-gather-n8c46] OUT 2026-09-17T16:24:29.88317046Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-5f4c5fb79b-6p8dc/storage-initializer/storage-initializer/logs/previous.log [must-gather-n8c46] OUT 2026-09-17T16:24:29.883220131Z namespaces/kserve-ci-e2e-test/policy/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.883276113Z namespaces/kserve-ci-e2e-test/policy/poddisruptionbudgets.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.883333714Z namespaces/kserve-ci-e2e-test/route.openshift.io/ [must-gather-n8c46] OUT 2026-09-17T16:24:29.883389346Z namespaces/kserve-ci-e2e-test/route.openshift.io/routes.yaml [must-gather-n8c46] OUT 2026-09-17T16:24:29.890311031Z [must-gather-n8c46] OUT 2026-09-17T16:24:29.890329201Z sent 1,072 bytes received 96,678 bytes 195,500.00 bytes/sec [must-gather-n8c46] OUT 2026-09-17T16:24:29.890335701Z total size is 1,441,078 speedup is 14.74 [must-gather ] OUT 2026-09-17T16:24:30.027690225Z namespace/openshift-must-gather-r5xzc deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 8e49930d-f8de-4713-bbae-7e7c7aec5a46 ClientVersion: 4.21.10 ClusterVersion: Stable at "4.21.34" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing