[must-gather ] OUT 2026-09-09T21:09:00.740309395Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: b8d3a53f-9bec-44f9-8e18-d8874c8295a8 ClientVersion: 4.21.10 ClusterVersion: Stable at "4.21.32" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing [must-gather ] OUT 2026-09-09T21:09:00.782318282Z namespace/openshift-must-gather-ptfbq created [must-gather ] OUT 2026-09-09T21:09:00.787371086Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-529z2 created [must-gather ] OUT 2026-09-09T21:09:00.810729793Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather-gmzcj] OUT 2026-09-09T21:09:39.108181285Z waiting for gather to complete [must-gather-gmzcj] OUT 2026-09-09T21:09:39.115278013Z downloading gather output [must-gather-gmzcj] OUT 2026-09-09T21:09:39.442118657Z receiving incremental file list [must-gather-gmzcj] OUT 2026-09-09T21:09:39.458524044Z ./ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.458609066Z aggregated-discovery-api.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.459149918Z aggregated-discovery-apis.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.459729101Z event-filter.html [must-gather-gmzcj] OUT 2026-09-09T21:09:39.461849918Z timestamp [must-gather-gmzcj] OUT 2026-09-09T21:09:39.461978481Z version [must-gather-gmzcj] OUT 2026-09-09T21:09:39.462656246Z cluster-scoped-resources/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.462670977Z cluster-scoped-resources/datasciencecluster.opendatahub.io/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.462677227Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.462719018Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/test-dsc.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.462788459Z cluster-scoped-resources/dscinitialization.opendatahub.io/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.46280235Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.46283596Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/test-dsci.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.462905522Z namespaces/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.462914752Z namespaces/kserve-ci-e2e-test/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.462943143Z namespaces/kserve-ci-e2e-test/kserve-ci-e2e-test.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.463013575Z namespaces/kserve-ci-e2e-test/apps.openshift.io/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.463058745Z namespaces/kserve-ci-e2e-test/apps.openshift.io/deploymentconfigs.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.463103306Z namespaces/kserve-ci-e2e-test/apps/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.463158448Z namespaces/kserve-ci-e2e-test/apps/daemonsets.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.46326067Z namespaces/kserve-ci-e2e-test/apps/deployments.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.463397233Z namespaces/kserve-ci-e2e-test/apps/replicasets.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.463511866Z namespaces/kserve-ci-e2e-test/apps/statefulsets.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.463552596Z namespaces/kserve-ci-e2e-test/autoscaling/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.463588507Z namespaces/kserve-ci-e2e-test/autoscaling/horizontalpodautoscalers.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.463660939Z namespaces/kserve-ci-e2e-test/batch/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.46370103Z namespaces/kserve-ci-e2e-test/batch/cronjobs.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.463789932Z namespaces/kserve-ci-e2e-test/batch/jobs.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.463835353Z namespaces/kserve-ci-e2e-test/build.openshift.io/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.463866414Z namespaces/kserve-ci-e2e-test/build.openshift.io/buildconfigs.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.463953915Z namespaces/kserve-ci-e2e-test/build.openshift.io/builds.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.464005766Z namespaces/kserve-ci-e2e-test/core/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.464034527Z namespaces/kserve-ci-e2e-test/core/configmaps.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.46416222Z namespaces/kserve-ci-e2e-test/core/endpoints.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.464285893Z namespaces/kserve-ci-e2e-test/core/events.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.469123601Z namespaces/kserve-ci-e2e-test/core/persistentvolumeclaims.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.469197842Z namespaces/kserve-ci-e2e-test/core/pods.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.469393057Z namespaces/kserve-ci-e2e-test/core/replicationcontrollers.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.46952485Z namespaces/kserve-ci-e2e-test/core/secrets.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.469731034Z namespaces/kserve-ci-e2e-test/core/services.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.469787026Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.469844657Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/endpointslices.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.469915398Z namespaces/kserve-ci-e2e-test/image.openshift.io/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.469941389Z namespaces/kserve-ci-e2e-test/image.openshift.io/imagestreams.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.46999247Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.470043201Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressfirewalls.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.470132934Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressqoses.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.470164824Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.470229676Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/servicemonitors.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.470301997Z namespaces/kserve-ci-e2e-test/networking.k8s.io/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.470354338Z namespaces/kserve-ci-e2e-test/networking.k8s.io/networkpolicies.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.470399379Z namespaces/kserve-ci-e2e-test/pods/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.4704094Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.470451461Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.470605644Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/agent/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.470615804Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/agent/agent/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.470621264Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/agent/agent/logs/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.470659775Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/agent/agent/logs/current.log [must-gather-gmzcj] OUT 2026-09-09T21:09:39.470830709Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/agent/agent/logs/previous.insecure.log [must-gather-gmzcj] OUT 2026-09-09T21:09:39.470933821Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/agent/agent/logs/previous.log [must-gather-gmzcj] OUT 2026-09-09T21:09:39.470987963Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/kserve-container/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.471005513Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/kserve-container/kserve-container/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.471013223Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/kserve-container/kserve-container/logs/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.471057864Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/kserve-container/kserve-container/logs/current.log [must-gather-gmzcj] OUT 2026-09-09T21:09:39.471187127Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/kserve-container/kserve-container/logs/previous.insecure.log [must-gather-gmzcj] OUT 2026-09-09T21:09:39.471281459Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/kserve-container/kserve-container/logs/previous.log [must-gather-gmzcj] OUT 2026-09-09T21:09:39.4713215Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/kube-rbac-proxy/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.47132952Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/kube-rbac-proxy/kube-rbac-proxy/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.47133436Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.471377721Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather-gmzcj] OUT 2026-09-09T21:09:39.471494724Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather-gmzcj] OUT 2026-09-09T21:09:39.471567886Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather-gmzcj] OUT 2026-09-09T21:09:39.471614187Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/storage-initializer/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.471626147Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/storage-initializer/storage-initializer/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.471632787Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/storage-initializer/storage-initializer/logs/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.471661478Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/storage-initializer/storage-initializer/logs/current.log [must-gather-gmzcj] OUT 2026-09-09T21:09:39.47176111Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/storage-initializer/storage-initializer/logs/previous.insecure.log [must-gather-gmzcj] OUT 2026-09-09T21:09:39.471825671Z namespaces/kserve-ci-e2e-test/pods/isvc-sklearn-batcher-custom-predictor-6757b57d7d-mj5n5/storage-initializer/storage-initializer/logs/previous.log [must-gather-gmzcj] OUT 2026-09-09T21:09:39.471864422Z namespaces/kserve-ci-e2e-test/policy/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.471907883Z namespaces/kserve-ci-e2e-test/policy/poddisruptionbudgets.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.471961064Z namespaces/kserve-ci-e2e-test/route.openshift.io/ [must-gather-gmzcj] OUT 2026-09-09T21:09:39.472000515Z namespaces/kserve-ci-e2e-test/route.openshift.io/routes.yaml [must-gather-gmzcj] OUT 2026-09-09T21:09:39.479866961Z [must-gather-gmzcj] OUT 2026-09-09T21:09:39.479884971Z sent 1,072 bytes received 193,263 bytes 388,670.00 bytes/sec [must-gather-gmzcj] OUT 2026-09-09T21:09:39.479890601Z total size is 3,128,943 speedup is 16.10 [must-gather ] OUT 2026-09-09T21:09:39.715713202Z namespace/openshift-must-gather-ptfbq deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: b8d3a53f-9bec-44f9-8e18-d8874c8295a8 ClientVersion: 4.21.10 ClusterVersion: Stable at "4.21.32" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing