[must-gather ] OUT 2026-04-22T19:04:34.993024371Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: d18ca5bf-c1dc-4240-8884-75e1e4b86e18 ClientVersion: 4.21.5 ClusterVersion: Stable at "4.20.19" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing [must-gather ] OUT 2026-04-22T19:04:35.047644776Z namespace/openshift-must-gather-82dfm created [must-gather ] OUT 2026-04-22T19:04:35.053313028Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-fcndk created [must-gather ] OUT 2026-04-22T19:04:35.084087004Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather-dm82d] OUT 2026-04-22T19:05:00.244717173Z waiting for gather to complete [must-gather-dm82d] OUT 2026-04-22T19:05:00.248119745Z downloading gather output [must-gather-dm82d] OUT 2026-04-22T19:05:00.547841868Z receiving incremental file list [must-gather-dm82d] OUT 2026-04-22T19:05:00.560929559Z ./ [must-gather-dm82d] OUT 2026-04-22T19:05:00.561020791Z aggregated-discovery-api.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.56133468Z aggregated-discovery-apis.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.561961467Z event-filter.html [must-gather-dm82d] OUT 2026-04-22T19:05:00.562150461Z timestamp [must-gather-dm82d] OUT 2026-04-22T19:05:00.562307796Z version [must-gather-dm82d] OUT 2026-04-22T19:05:00.562391658Z cluster-scoped-resources/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.562403148Z cluster-scoped-resources/datasciencecluster.opendatahub.io/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.562411619Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.562424389Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/test-dsc.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.562502221Z cluster-scoped-resources/dscinitialization.opendatahub.io/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.562512261Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.562525562Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/test-dsci.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.562602614Z namespaces/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.562612204Z namespaces/kserve-ci-e2e-test/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.562648335Z namespaces/kserve-ci-e2e-test/kserve-ci-e2e-test.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.562699256Z namespaces/kserve-ci-e2e-test/apps.openshift.io/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.562743317Z namespaces/kserve-ci-e2e-test/apps.openshift.io/deploymentconfigs.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.562793729Z namespaces/kserve-ci-e2e-test/apps/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.5628372Z namespaces/kserve-ci-e2e-test/apps/daemonsets.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.562910212Z namespaces/kserve-ci-e2e-test/apps/deployments.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.563024735Z namespaces/kserve-ci-e2e-test/apps/replicasets.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.563134488Z namespaces/kserve-ci-e2e-test/apps/statefulsets.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.563183269Z namespaces/kserve-ci-e2e-test/autoscaling/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.563232761Z namespaces/kserve-ci-e2e-test/autoscaling/horizontalpodautoscalers.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.563321853Z namespaces/kserve-ci-e2e-test/batch/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.563367984Z namespaces/kserve-ci-e2e-test/batch/cronjobs.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.563445546Z namespaces/kserve-ci-e2e-test/batch/jobs.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.563483097Z namespaces/kserve-ci-e2e-test/build.openshift.io/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.563531959Z namespaces/kserve-ci-e2e-test/build.openshift.io/buildconfigs.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.563605671Z namespaces/kserve-ci-e2e-test/build.openshift.io/builds.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.563650252Z namespaces/kserve-ci-e2e-test/core/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.563690333Z namespaces/kserve-ci-e2e-test/core/configmaps.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.563836947Z namespaces/kserve-ci-e2e-test/core/endpoints.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.563923139Z namespaces/kserve-ci-e2e-test/core/events.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.564085363Z namespaces/kserve-ci-e2e-test/core/persistentvolumeclaims.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.564162355Z namespaces/kserve-ci-e2e-test/core/pods.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.56431923Z namespaces/kserve-ci-e2e-test/core/replicationcontrollers.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.564444313Z namespaces/kserve-ci-e2e-test/core/secrets.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.564656029Z namespaces/kserve-ci-e2e-test/core/services.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.56471936Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.564762751Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/endpointslices.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.564820563Z namespaces/kserve-ci-e2e-test/image.openshift.io/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.564870705Z namespaces/kserve-ci-e2e-test/image.openshift.io/imagestreams.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.564908126Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.564942886Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressfirewalls.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.565023618Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressqoses.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.56506768Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.565107031Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/servicemonitors.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.565184133Z namespaces/kserve-ci-e2e-test/networking.k8s.io/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.565223684Z namespaces/kserve-ci-e2e-test/networking.k8s.io/networkpolicies.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.565277105Z namespaces/kserve-ci-e2e-test/pods/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.565289566Z namespaces/kserve-ci-e2e-test/pods/isvc-raw-sklearn-batcher-1df94-predictor-685df7dbbc-g2ctd/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.565338327Z namespaces/kserve-ci-e2e-test/pods/isvc-raw-sklearn-batcher-1df94-predictor-685df7dbbc-g2ctd/isvc-raw-sklearn-batcher-1df94-predictor-685df7dbbc-g2ctd.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.565472011Z namespaces/kserve-ci-e2e-test/pods/isvc-raw-sklearn-batcher-1df94-predictor-685df7dbbc-g2ctd/agent/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.565479451Z namespaces/kserve-ci-e2e-test/pods/isvc-raw-sklearn-batcher-1df94-predictor-685df7dbbc-g2ctd/agent/agent/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.565483611Z namespaces/kserve-ci-e2e-test/pods/isvc-raw-sklearn-batcher-1df94-predictor-685df7dbbc-g2ctd/agent/agent/logs/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.565522302Z namespaces/kserve-ci-e2e-test/pods/isvc-raw-sklearn-batcher-1df94-predictor-685df7dbbc-g2ctd/agent/agent/logs/current.log [must-gather-dm82d] OUT 2026-04-22T19:05:00.565619945Z namespaces/kserve-ci-e2e-test/pods/isvc-raw-sklearn-batcher-1df94-predictor-685df7dbbc-g2ctd/agent/agent/logs/previous.insecure.log [must-gather-dm82d] OUT 2026-04-22T19:05:00.565688976Z namespaces/kserve-ci-e2e-test/pods/isvc-raw-sklearn-batcher-1df94-predictor-685df7dbbc-g2ctd/agent/agent/logs/previous.log [must-gather-dm82d] OUT 2026-04-22T19:05:00.565728288Z namespaces/kserve-ci-e2e-test/pods/isvc-raw-sklearn-batcher-1df94-predictor-685df7dbbc-g2ctd/kserve-container/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.565736708Z namespaces/kserve-ci-e2e-test/pods/isvc-raw-sklearn-batcher-1df94-predictor-685df7dbbc-g2ctd/kserve-container/kserve-container/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.565740978Z namespaces/kserve-ci-e2e-test/pods/isvc-raw-sklearn-batcher-1df94-predictor-685df7dbbc-g2ctd/kserve-container/kserve-container/logs/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.565774469Z namespaces/kserve-ci-e2e-test/pods/isvc-raw-sklearn-batcher-1df94-predictor-685df7dbbc-g2ctd/kserve-container/kserve-container/logs/current.log [must-gather-dm82d] OUT 2026-04-22T19:05:00.565874792Z namespaces/kserve-ci-e2e-test/pods/isvc-raw-sklearn-batcher-1df94-predictor-685df7dbbc-g2ctd/kserve-container/kserve-container/logs/previous.insecure.log [must-gather-dm82d] OUT 2026-04-22T19:05:00.565939943Z namespaces/kserve-ci-e2e-test/pods/isvc-raw-sklearn-batcher-1df94-predictor-685df7dbbc-g2ctd/kserve-container/kserve-container/logs/previous.log [must-gather-dm82d] OUT 2026-04-22T19:05:00.565968004Z namespaces/kserve-ci-e2e-test/pods/isvc-raw-sklearn-batcher-1df94-predictor-685df7dbbc-g2ctd/storage-initializer/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.565980194Z namespaces/kserve-ci-e2e-test/pods/isvc-raw-sklearn-batcher-1df94-predictor-685df7dbbc-g2ctd/storage-initializer/storage-initializer/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.565985154Z namespaces/kserve-ci-e2e-test/pods/isvc-raw-sklearn-batcher-1df94-predictor-685df7dbbc-g2ctd/storage-initializer/storage-initializer/logs/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.566019435Z namespaces/kserve-ci-e2e-test/pods/isvc-raw-sklearn-batcher-1df94-predictor-685df7dbbc-g2ctd/storage-initializer/storage-initializer/logs/current.log [must-gather-dm82d] OUT 2026-04-22T19:05:00.566107758Z namespaces/kserve-ci-e2e-test/pods/isvc-raw-sklearn-batcher-1df94-predictor-685df7dbbc-g2ctd/storage-initializer/storage-initializer/logs/previous.insecure.log [must-gather-dm82d] OUT 2026-04-22T19:05:00.56617746Z namespaces/kserve-ci-e2e-test/pods/isvc-raw-sklearn-batcher-1df94-predictor-685df7dbbc-g2ctd/storage-initializer/storage-initializer/logs/previous.log [must-gather-dm82d] OUT 2026-04-22T19:05:00.56621179Z namespaces/kserve-ci-e2e-test/policy/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.566250742Z namespaces/kserve-ci-e2e-test/policy/poddisruptionbudgets.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.566316593Z namespaces/kserve-ci-e2e-test/route.openshift.io/ [must-gather-dm82d] OUT 2026-04-22T19:05:00.566356204Z namespaces/kserve-ci-e2e-test/route.openshift.io/routes.yaml [must-gather-dm82d] OUT 2026-04-22T19:05:00.57514902Z [must-gather-dm82d] OUT 2026-04-22T19:05:00.57516601Z sent 995 bytes received 43,728 bytes 89,446.00 bytes/sec [must-gather-dm82d] OUT 2026-04-22T19:05:00.575170841Z total size is 312,734 speedup is 6.99 [must-gather ] OUT 2026-04-22T19:05:00.607707043Z namespace/openshift-must-gather-82dfm deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: d18ca5bf-c1dc-4240-8884-75e1e4b86e18 ClientVersion: 4.21.5 ClusterVersion: Stable at "4.20.19" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing