[must-gather ] OUT 2026-09-16T19:13:50.912179412Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 5733ba65-b9c2-4dfe-af53-84c6937d094d ClientVersion: 4.21.10 ClusterVersion: Stable at "4.21.33" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing [must-gather ] OUT 2026-09-16T19:13:50.956057581Z namespace/openshift-must-gather-zrdnl created [must-gather ] OUT 2026-09-16T19:13:50.966318769Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-bg6s9 created [must-gather ] OUT 2026-09-16T19:13:50.995714756Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather-shtm4] OUT 2026-09-16T19:14:21.977334204Z waiting for gather to complete [must-gather-shtm4] OUT 2026-09-16T19:14:21.980394644Z downloading gather output [must-gather-shtm4] OUT 2026-09-16T19:14:22.248974645Z receiving incremental file list [must-gather-shtm4] OUT 2026-09-16T19:14:22.262975321Z ./ [must-gather-shtm4] OUT 2026-09-16T19:14:22.263147641Z aggregated-discovery-api.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.263696591Z aggregated-discovery-apis.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.264800392Z event-filter.html [must-gather-shtm4] OUT 2026-09-16T19:14:22.26602546Z timestamp [must-gather-shtm4] OUT 2026-09-16T19:14:22.266152427Z version [must-gather-shtm4] OUT 2026-09-16T19:14:22.267870953Z cluster-scoped-resources/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.267890724Z cluster-scoped-resources/datasciencecluster.opendatahub.io/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.267896624Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.26800133Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/test-dsc.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.268157399Z cluster-scoped-resources/dscinitialization.opendatahub.io/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.268167469Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.268232823Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/test-dsci.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.268329418Z namespaces/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.268347709Z namespaces/kserve-ci-e2e-test/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.268394242Z namespaces/kserve-ci-e2e-test/kserve-ci-e2e-test.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.268493627Z namespaces/kserve-ci-e2e-test/apps.openshift.io/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.268571922Z namespaces/kserve-ci-e2e-test/apps.openshift.io/deploymentconfigs.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.268650306Z namespaces/kserve-ci-e2e-test/apps/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.268672207Z namespaces/kserve-ci-e2e-test/apps/daemonsets.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.268790044Z namespaces/kserve-ci-e2e-test/apps/deployments.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.268890649Z namespaces/kserve-ci-e2e-test/apps/replicasets.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.268970754Z namespaces/kserve-ci-e2e-test/apps/statefulsets.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.269017156Z namespaces/kserve-ci-e2e-test/autoscaling/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.269067729Z namespaces/kserve-ci-e2e-test/autoscaling/horizontalpodautoscalers.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.269113332Z namespaces/kserve-ci-e2e-test/batch/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.269167405Z namespaces/kserve-ci-e2e-test/batch/cronjobs.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.269237729Z namespaces/kserve-ci-e2e-test/batch/jobs.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.269285301Z namespaces/kserve-ci-e2e-test/build.openshift.io/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.269327314Z namespaces/kserve-ci-e2e-test/build.openshift.io/buildconfigs.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.269407098Z namespaces/kserve-ci-e2e-test/build.openshift.io/builds.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.269456921Z namespaces/kserve-ci-e2e-test/core/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.269495443Z namespaces/kserve-ci-e2e-test/core/configmaps.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.26963036Z namespaces/kserve-ci-e2e-test/core/endpoints.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.269709065Z namespaces/kserve-ci-e2e-test/core/events.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.272448397Z namespaces/kserve-ci-e2e-test/core/persistentvolumeclaims.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.272532921Z namespaces/kserve-ci-e2e-test/core/pods.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.272736713Z namespaces/kserve-ci-e2e-test/core/replicationcontrollers.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.272889401Z namespaces/kserve-ci-e2e-test/core/secrets.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.273070461Z namespaces/kserve-ci-e2e-test/core/services.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.273106623Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.273159346Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/endpointslices.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.273195738Z namespaces/kserve-ci-e2e-test/image.openshift.io/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.273246311Z namespaces/kserve-ci-e2e-test/image.openshift.io/imagestreams.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.273280213Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.273331196Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressfirewalls.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.27341172Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressqoses.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.273446442Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.273495185Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/servicemonitors.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.273534017Z namespaces/kserve-ci-e2e-test/networking.k8s.io/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.27359148Z namespaces/kserve-ci-e2e-test/networking.k8s.io/networkpolicies.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.273629732Z namespaces/kserve-ci-e2e-test/pods/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.273638523Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.273684085Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.273873716Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/agent/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.273889027Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/agent/agent/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.273894327Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/agent/agent/logs/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.273925378Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/agent/agent/logs/current.log [must-gather-shtm4] OUT 2026-09-16T19:14:22.274035875Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/agent/agent/logs/previous.insecure.log [must-gather-shtm4] OUT 2026-09-16T19:14:22.274101188Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/agent/agent/logs/previous.log [must-gather-shtm4] OUT 2026-09-16T19:14:22.27413786Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/kserve-container/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.274150841Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/kserve-container/kserve-container/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.274155061Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/kserve-container/kserve-container/logs/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.274185053Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/kserve-container/kserve-container/logs/current.log [must-gather-shtm4] OUT 2026-09-16T19:14:22.274297879Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/kserve-container/kserve-container/logs/previous.insecure.log [must-gather-shtm4] OUT 2026-09-16T19:14:22.274363623Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/kserve-container/kserve-container/logs/previous.log [must-gather-shtm4] OUT 2026-09-16T19:14:22.274400065Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/kube-rbac-proxy/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.274407685Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/kube-rbac-proxy/kube-rbac-proxy/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.274411636Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.274441397Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather-shtm4] OUT 2026-09-16T19:14:22.274574094Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather-shtm4] OUT 2026-09-16T19:14:22.274640408Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather-shtm4] OUT 2026-09-16T19:14:22.274682611Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/storage-initializer/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.274693321Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/storage-initializer/storage-initializer/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.274698992Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/storage-initializer/storage-initializer/logs/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.274724413Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/storage-initializer/storage-initializer/logs/current.log [must-gather-shtm4] OUT 2026-09-16T19:14:22.274818848Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/storage-initializer/storage-initializer/logs/previous.insecure.log [must-gather-shtm4] OUT 2026-09-16T19:14:22.274907993Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-67b78f46c9-dhsmt/storage-initializer/storage-initializer/logs/previous.log [must-gather-shtm4] OUT 2026-09-16T19:14:22.274945765Z namespaces/kserve-ci-e2e-test/policy/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.274986387Z namespaces/kserve-ci-e2e-test/policy/poddisruptionbudgets.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.27503532Z namespaces/kserve-ci-e2e-test/route.openshift.io/ [must-gather-shtm4] OUT 2026-09-16T19:14:22.275077392Z namespaces/kserve-ci-e2e-test/route.openshift.io/routes.yaml [must-gather-shtm4] OUT 2026-09-16T19:14:22.282498044Z [must-gather-shtm4] OUT 2026-09-16T19:14:22.282524765Z sent 1,072 bytes received 99,797 bytes 67,246.00 bytes/sec [must-gather-shtm4] OUT 2026-09-16T19:14:22.282533086Z total size is 1,455,846 speedup is 14.43 [must-gather ] OUT 2026-09-16T19:14:22.395710119Z namespace/openshift-must-gather-zrdnl deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 5733ba65-b9c2-4dfe-af53-84c6937d094d ClientVersion: 4.21.10 ClusterVersion: Stable at "4.21.33" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing