[must-gather ] OUT 2026-09-17T17:08:53.682240956Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 087daeac-b484-4149-a93c-50224406b083 ClientVersion: 4.21.10 ClusterVersion: Stable at "4.21.34" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing [must-gather ] OUT 2026-09-17T17:08:53.727058882Z namespace/openshift-must-gather-x48x6 created [must-gather ] OUT 2026-09-17T17:08:53.73426682Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-wb5r6 created [must-gather ] OUT 2026-09-17T17:08:53.759199094Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather-5khbm] OUT 2026-09-17T17:09:22.722729109Z waiting for gather to complete [must-gather-5khbm] OUT 2026-09-17T17:09:22.725731575Z downloading gather output [must-gather-5khbm] OUT 2026-09-17T17:09:22.982066706Z receiving incremental file list [must-gather-5khbm] OUT 2026-09-17T17:09:22.993322438Z ./ [must-gather-5khbm] OUT 2026-09-17T17:09:22.993421306Z aggregated-discovery-api.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:22.993651796Z aggregated-discovery-apis.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:22.994479027Z event-filter.html [must-gather-5khbm] OUT 2026-09-17T17:09:22.995622374Z timestamp [must-gather-5khbm] OUT 2026-09-17T17:09:22.995752635Z version [must-gather-5khbm] OUT 2026-09-17T17:09:22.997102191Z cluster-scoped-resources/ [must-gather-5khbm] OUT 2026-09-17T17:09:22.997115572Z cluster-scoped-resources/datasciencecluster.opendatahub.io/ [must-gather-5khbm] OUT 2026-09-17T17:09:22.997122063Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/ [must-gather-5khbm] OUT 2026-09-17T17:09:22.997163746Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/test-dsc.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:22.997239573Z cluster-scoped-resources/dscinitialization.opendatahub.io/ [must-gather-5khbm] OUT 2026-09-17T17:09:22.997263295Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/ [must-gather-5khbm] OUT 2026-09-17T17:09:22.997313749Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/test-dsci.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:22.997385685Z namespaces/ [must-gather-5khbm] OUT 2026-09-17T17:09:22.997395416Z namespaces/kserve-ci-e2e-test/ [must-gather-5khbm] OUT 2026-09-17T17:09:22.997429929Z namespaces/kserve-ci-e2e-test/kserve-ci-e2e-test.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:22.997514826Z namespaces/kserve-ci-e2e-test/apps.openshift.io/ [must-gather-5khbm] OUT 2026-09-17T17:09:22.99756673Z namespaces/kserve-ci-e2e-test/apps.openshift.io/deploymentconfigs.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:22.997617385Z namespaces/kserve-ci-e2e-test/apps/ [must-gather-5khbm] OUT 2026-09-17T17:09:22.997665589Z namespaces/kserve-ci-e2e-test/apps/daemonsets.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:22.997764827Z namespaces/kserve-ci-e2e-test/apps/deployments.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:22.997863416Z namespaces/kserve-ci-e2e-test/apps/replicasets.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:22.997977105Z namespaces/kserve-ci-e2e-test/apps/statefulsets.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:22.998036791Z namespaces/kserve-ci-e2e-test/autoscaling/ [must-gather-5khbm] OUT 2026-09-17T17:09:22.998092446Z namespaces/kserve-ci-e2e-test/autoscaling/horizontalpodautoscalers.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:22.9981443Z namespaces/kserve-ci-e2e-test/batch/ [must-gather-5khbm] OUT 2026-09-17T17:09:22.998199175Z namespaces/kserve-ci-e2e-test/batch/cronjobs.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:22.998303333Z namespaces/kserve-ci-e2e-test/batch/jobs.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:22.998363669Z namespaces/kserve-ci-e2e-test/build.openshift.io/ [must-gather-5khbm] OUT 2026-09-17T17:09:22.998404442Z namespaces/kserve-ci-e2e-test/build.openshift.io/buildconfigs.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:22.998528673Z namespaces/kserve-ci-e2e-test/build.openshift.io/builds.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:22.998544784Z namespaces/kserve-ci-e2e-test/core/ [must-gather-5khbm] OUT 2026-09-17T17:09:22.998617Z namespaces/kserve-ci-e2e-test/core/configmaps.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:22.998754782Z namespaces/kserve-ci-e2e-test/core/endpoints.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:22.99885167Z namespaces/kserve-ci-e2e-test/core/events.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:23.001365015Z namespaces/kserve-ci-e2e-test/core/persistentvolumeclaims.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:23.001444552Z namespaces/kserve-ci-e2e-test/core/pods.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:23.001636198Z namespaces/kserve-ci-e2e-test/core/replicationcontrollers.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:23.001757739Z namespaces/kserve-ci-e2e-test/core/secrets.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:23.001950385Z namespaces/kserve-ci-e2e-test/core/services.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:23.002029932Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/ [must-gather-5khbm] OUT 2026-09-17T17:09:23.002080827Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/endpointslices.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:23.00212078Z namespaces/kserve-ci-e2e-test/image.openshift.io/ [must-gather-5khbm] OUT 2026-09-17T17:09:23.002171364Z namespaces/kserve-ci-e2e-test/image.openshift.io/imagestreams.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:23.002215348Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/ [must-gather-5khbm] OUT 2026-09-17T17:09:23.002262802Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressfirewalls.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:23.002351679Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressqoses.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:23.002396523Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/ [must-gather-5khbm] OUT 2026-09-17T17:09:23.002426526Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/servicemonitors.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:23.00247087Z namespaces/kserve-ci-e2e-test/networking.k8s.io/ [must-gather-5khbm] OUT 2026-09-17T17:09:23.002521454Z namespaces/kserve-ci-e2e-test/networking.k8s.io/networkpolicies.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:23.002564688Z namespaces/kserve-ci-e2e-test/pods/ [must-gather-5khbm] OUT 2026-09-17T17:09:23.002572138Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/ [must-gather-5khbm] OUT 2026-09-17T17:09:23.002620722Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:23.002768545Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/agent/ [must-gather-5khbm] OUT 2026-09-17T17:09:23.002779156Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/agent/agent/ [must-gather-5khbm] OUT 2026-09-17T17:09:23.002783347Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/agent/agent/logs/ [must-gather-5khbm] OUT 2026-09-17T17:09:23.002811209Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/agent/agent/logs/current.log [must-gather-5khbm] OUT 2026-09-17T17:09:23.002949361Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/agent/agent/logs/previous.insecure.log [must-gather-5khbm] OUT 2026-09-17T17:09:23.003039828Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/agent/agent/logs/previous.log [must-gather-5khbm] OUT 2026-09-17T17:09:23.00305812Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/kserve-container/ [must-gather-5khbm] OUT 2026-09-17T17:09:23.003070431Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/kserve-container/kserve-container/ [must-gather-5khbm] OUT 2026-09-17T17:09:23.003075352Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/kserve-container/kserve-container/logs/ [must-gather-5khbm] OUT 2026-09-17T17:09:23.003107564Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/kserve-container/kserve-container/logs/current.log [must-gather-5khbm] OUT 2026-09-17T17:09:23.003223274Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/kserve-container/kserve-container/logs/previous.insecure.log [must-gather-5khbm] OUT 2026-09-17T17:09:23.00329554Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/kserve-container/kserve-container/logs/previous.log [must-gather-5khbm] OUT 2026-09-17T17:09:23.003345495Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/kube-rbac-proxy/ [must-gather-5khbm] OUT 2026-09-17T17:09:23.003354405Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/kube-rbac-proxy/kube-rbac-proxy/ [must-gather-5khbm] OUT 2026-09-17T17:09:23.003358666Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather-5khbm] OUT 2026-09-17T17:09:23.00340848Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather-5khbm] OUT 2026-09-17T17:09:23.00352414Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather-5khbm] OUT 2026-09-17T17:09:23.003587625Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather-5khbm] OUT 2026-09-17T17:09:23.003621838Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/storage-initializer/ [must-gather-5khbm] OUT 2026-09-17T17:09:23.003630209Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/storage-initializer/storage-initializer/ [must-gather-5khbm] OUT 2026-09-17T17:09:23.003635269Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/storage-initializer/storage-initializer/logs/ [must-gather-5khbm] OUT 2026-09-17T17:09:23.003658571Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/storage-initializer/storage-initializer/logs/current.log [must-gather-5khbm] OUT 2026-09-17T17:09:23.00376076Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/storage-initializer/storage-initializer/logs/previous.insecure.log [must-gather-5khbm] OUT 2026-09-17T17:09:23.003834146Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-64c6cf64bf-dwf5q/storage-initializer/storage-initializer/logs/previous.log [must-gather-5khbm] OUT 2026-09-17T17:09:23.003866179Z namespaces/kserve-ci-e2e-test/policy/ [must-gather-5khbm] OUT 2026-09-17T17:09:23.003907883Z namespaces/kserve-ci-e2e-test/policy/poddisruptionbudgets.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:23.003973278Z namespaces/kserve-ci-e2e-test/route.openshift.io/ [must-gather-5khbm] OUT 2026-09-17T17:09:23.004012352Z namespaces/kserve-ci-e2e-test/route.openshift.io/routes.yaml [must-gather-5khbm] OUT 2026-09-17T17:09:23.010335912Z [must-gather-5khbm] OUT 2026-09-17T17:09:23.010351083Z sent 1,072 bytes received 98,476 bytes 66,365.33 bytes/sec [must-gather-5khbm] OUT 2026-09-17T17:09:23.010358134Z total size is 1,444,621 speedup is 14.51 [must-gather ] OUT 2026-09-17T17:09:23.126907076Z namespace/openshift-must-gather-x48x6 deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 087daeac-b484-4149-a93c-50224406b083 ClientVersion: 4.21.10 ClusterVersion: Stable at "4.21.34" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing