[must-gather ] OUT 2026-09-17T22:24:28.615889602Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 1300dc7a-a92c-4120-a2d6-4857dd1e3702 ClientVersion: 4.21.10 ClusterVersion: Stable at "4.21.34" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing [must-gather ] OUT 2026-09-17T22:24:28.66065529Z namespace/openshift-must-gather-jrksc created [must-gather ] OUT 2026-09-17T22:24:28.670968029Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-wn4vj created [must-gather ] OUT 2026-09-17T22:24:28.69665687Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather-np8bt] OUT 2026-09-17T22:24:59.276439927Z waiting for gather to complete [must-gather-np8bt] OUT 2026-09-17T22:24:59.279731002Z downloading gather output [must-gather-np8bt] OUT 2026-09-17T22:24:59.543446637Z receiving incremental file list [must-gather-np8bt] OUT 2026-09-17T22:24:59.557392811Z ./ [must-gather-np8bt] OUT 2026-09-17T22:24:59.557537714Z aggregated-discovery-api.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.558078645Z aggregated-discovery-apis.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.559003903Z event-filter.html [must-gather-np8bt] OUT 2026-09-17T22:24:59.560175846Z timestamp [must-gather-np8bt] OUT 2026-09-17T22:24:59.560327849Z version [must-gather-np8bt] OUT 2026-09-17T22:24:59.560432531Z cluster-scoped-resources/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.560464712Z cluster-scoped-resources/datasciencecluster.opendatahub.io/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.560481092Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.560522103Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/test-dsc.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.560563863Z cluster-scoped-resources/dscinitialization.opendatahub.io/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.560573824Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.560605714Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/test-dsci.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.560646775Z namespaces/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.560661585Z namespaces/kserve-ci-e2e-test/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.560693406Z namespaces/kserve-ci-e2e-test/kserve-ci-e2e-test.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.560726797Z namespaces/kserve-ci-e2e-test/apps.openshift.io/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.560785168Z namespaces/kserve-ci-e2e-test/apps.openshift.io/deploymentconfigs.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.560828479Z namespaces/kserve-ci-e2e-test/apps/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.560852889Z namespaces/kserve-ci-e2e-test/apps/daemonsets.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.560947201Z namespaces/kserve-ci-e2e-test/apps/deployments.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.561022383Z namespaces/kserve-ci-e2e-test/apps/replicasets.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.561093084Z namespaces/kserve-ci-e2e-test/apps/statefulsets.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.561118694Z namespaces/kserve-ci-e2e-test/autoscaling/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.561145005Z namespaces/kserve-ci-e2e-test/autoscaling/horizontalpodautoscalers.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.561206846Z namespaces/kserve-ci-e2e-test/batch/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.561226047Z namespaces/kserve-ci-e2e-test/batch/cronjobs.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.561311238Z namespaces/kserve-ci-e2e-test/batch/jobs.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.561357249Z namespaces/kserve-ci-e2e-test/build.openshift.io/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.56141629Z namespaces/kserve-ci-e2e-test/build.openshift.io/buildconfigs.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.561503352Z namespaces/kserve-ci-e2e-test/build.openshift.io/builds.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.561548743Z namespaces/kserve-ci-e2e-test/core/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.561575363Z namespaces/kserve-ci-e2e-test/core/configmaps.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.561706646Z namespaces/kserve-ci-e2e-test/core/endpoints.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.561791838Z namespaces/kserve-ci-e2e-test/core/events.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.564234886Z namespaces/kserve-ci-e2e-test/core/persistentvolumeclaims.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.564324348Z namespaces/kserve-ci-e2e-test/core/pods.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.564532772Z namespaces/kserve-ci-e2e-test/core/replicationcontrollers.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.564660634Z namespaces/kserve-ci-e2e-test/core/secrets.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.564851318Z namespaces/kserve-ci-e2e-test/core/services.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.564900319Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.56493572Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/endpointslices.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.56498785Z namespaces/kserve-ci-e2e-test/image.openshift.io/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.565021931Z namespaces/kserve-ci-e2e-test/image.openshift.io/imagestreams.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.565074812Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.565111233Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressfirewalls.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.565191534Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressqoses.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.565240775Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.565270926Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/servicemonitors.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.565320817Z namespaces/kserve-ci-e2e-test/networking.k8s.io/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.565389018Z namespaces/kserve-ci-e2e-test/networking.k8s.io/networkpolicies.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.565434789Z namespaces/kserve-ci-e2e-test/pods/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.56544247Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.565491651Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.565662434Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/agent/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.565672614Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/agent/agent/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.565680124Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/agent/agent/logs/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.565711785Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/agent/agent/logs/current.log [must-gather-np8bt] OUT 2026-09-17T22:24:59.565842967Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/agent/agent/logs/previous.insecure.log [must-gather-np8bt] OUT 2026-09-17T22:24:59.565914939Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/agent/agent/logs/previous.log [must-gather-np8bt] OUT 2026-09-17T22:24:59.565937309Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/kserve-container/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.565957Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/kserve-container/kserve-container/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.5659625Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/kserve-container/kserve-container/logs/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.56599034Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/kserve-container/kserve-container/logs/current.log [must-gather-np8bt] OUT 2026-09-17T22:24:59.566107463Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/kserve-container/kserve-container/logs/previous.insecure.log [must-gather-np8bt] OUT 2026-09-17T22:24:59.566177234Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/kserve-container/kserve-container/logs/previous.log [must-gather-np8bt] OUT 2026-09-17T22:24:59.566222945Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/kube-rbac-proxy/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.566231105Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/kube-rbac-proxy/kube-rbac-proxy/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.566235445Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.566260126Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather-np8bt] OUT 2026-09-17T22:24:59.566402969Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather-np8bt] OUT 2026-09-17T22:24:59.56647552Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather-np8bt] OUT 2026-09-17T22:24:59.566516311Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/storage-initializer/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.566524761Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/storage-initializer/storage-initializer/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.566530681Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/storage-initializer/storage-initializer/logs/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.566559692Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/storage-initializer/storage-initializer/logs/current.log [must-gather-np8bt] OUT 2026-09-17T22:24:59.566665203Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/storage-initializer/storage-initializer/logs/previous.insecure.log [must-gather-np8bt] OUT 2026-09-17T22:24:59.566733565Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c9c6744b4-2tgjl/storage-initializer/storage-initializer/logs/previous.log [must-gather-np8bt] OUT 2026-09-17T22:24:59.566760165Z namespaces/kserve-ci-e2e-test/policy/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.566808196Z namespaces/kserve-ci-e2e-test/policy/poddisruptionbudgets.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.566857927Z namespaces/kserve-ci-e2e-test/route.openshift.io/ [must-gather-np8bt] OUT 2026-09-17T22:24:59.566897508Z namespaces/kserve-ci-e2e-test/route.openshift.io/routes.yaml [must-gather-np8bt] OUT 2026-09-17T22:24:59.574000878Z [must-gather-np8bt] OUT 2026-09-17T22:24:59.574024418Z sent 1,072 bytes received 99,186 bytes 200,516.00 bytes/sec [must-gather-np8bt] OUT 2026-09-17T22:24:59.574032299Z total size is 1,446,732 speedup is 14.43 [must-gather ] OUT 2026-09-17T22:24:59.689605431Z namespace/openshift-must-gather-jrksc deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 1300dc7a-a92c-4120-a2d6-4857dd1e3702 ClientVersion: 4.21.10 ClusterVersion: Stable at "4.21.34" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing