[must-gather ] OUT 2026-09-16T19:07:23.173420156Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 72788819-ab62-4d31-ac19-f3bc63c80a89 ClientVersion: 4.21.10 ClusterVersion: Stable at "4.21.33" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing [must-gather ] OUT 2026-09-16T19:07:23.224013388Z namespace/openshift-must-gather-vbfj9 created [must-gather ] OUT 2026-09-16T19:07:23.23419452Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-dz4fp created [must-gather ] OUT 2026-09-16T19:07:23.285363557Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather-s77bn] OUT 2026-09-16T19:07:54.056007237Z waiting for gather to complete [must-gather-s77bn] OUT 2026-09-16T19:07:54.059832994Z downloading gather output [must-gather-s77bn] OUT 2026-09-16T19:07:54.348025735Z receiving incremental file list [must-gather-s77bn] OUT 2026-09-16T19:07:54.371069886Z ./ [must-gather-s77bn] OUT 2026-09-16T19:07:54.371249791Z aggregated-discovery-api.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.371884548Z aggregated-discovery-apis.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.372899216Z event-filter.html [must-gather-s77bn] OUT 2026-09-16T19:07:54.374091519Z timestamp [must-gather-s77bn] OUT 2026-09-16T19:07:54.374243764Z version [must-gather-s77bn] OUT 2026-09-16T19:07:54.374512541Z cluster-scoped-resources/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.374526982Z cluster-scoped-resources/datasciencecluster.opendatahub.io/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.374532962Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.374596853Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/test-dsc.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.374748778Z cluster-scoped-resources/dscinitialization.opendatahub.io/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.374763858Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.37481099Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/test-dsci.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.374893762Z namespaces/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.374904022Z namespaces/kserve-ci-e2e-test/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.374946243Z namespaces/kserve-ci-e2e-test/kserve-ci-e2e-test.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.375040276Z namespaces/kserve-ci-e2e-test/apps.openshift.io/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.375089577Z namespaces/kserve-ci-e2e-test/apps.openshift.io/deploymentconfigs.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.375146169Z namespaces/kserve-ci-e2e-test/apps/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.37519535Z namespaces/kserve-ci-e2e-test/apps/daemonsets.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.375300673Z namespaces/kserve-ci-e2e-test/apps/deployments.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.375400706Z namespaces/kserve-ci-e2e-test/apps/replicasets.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.375499349Z namespaces/kserve-ci-e2e-test/apps/statefulsets.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.37555542Z namespaces/kserve-ci-e2e-test/autoscaling/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.375632252Z namespaces/kserve-ci-e2e-test/autoscaling/horizontalpodautoscalers.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.375697304Z namespaces/kserve-ci-e2e-test/batch/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.375745316Z namespaces/kserve-ci-e2e-test/batch/cronjobs.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.37589763Z namespaces/kserve-ci-e2e-test/batch/jobs.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.37591712Z namespaces/kserve-ci-e2e-test/build.openshift.io/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.37592215Z namespaces/kserve-ci-e2e-test/build.openshift.io/buildconfigs.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.375990112Z namespaces/kserve-ci-e2e-test/build.openshift.io/builds.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.376032864Z namespaces/kserve-ci-e2e-test/core/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.376073785Z namespaces/kserve-ci-e2e-test/core/configmaps.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.376209558Z namespaces/kserve-ci-e2e-test/core/endpoints.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.376282951Z namespaces/kserve-ci-e2e-test/core/events.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.378971295Z namespaces/kserve-ci-e2e-test/core/persistentvolumeclaims.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.379047657Z namespaces/kserve-ci-e2e-test/core/pods.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.37913741Z namespaces/kserve-ci-e2e-test/core/replicationcontrollers.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.379262753Z namespaces/kserve-ci-e2e-test/core/secrets.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.37948864Z namespaces/kserve-ci-e2e-test/core/services.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.379535611Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.379591992Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/endpointslices.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.379648694Z namespaces/kserve-ci-e2e-test/image.openshift.io/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.379700115Z namespaces/kserve-ci-e2e-test/image.openshift.io/imagestreams.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.379739397Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.379780718Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressfirewalls.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.37985617Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressqoses.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.379904001Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.379952733Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/servicemonitors.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.379993854Z namespaces/kserve-ci-e2e-test/networking.k8s.io/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.380035925Z namespaces/kserve-ci-e2e-test/networking.k8s.io/networkpolicies.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.380075796Z namespaces/kserve-ci-e2e-test/pods/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.380087606Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.380141168Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.380313652Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/agent/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.380323153Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/agent/agent/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.380328623Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/agent/agent/logs/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.380347044Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/agent/agent/logs/current.log [must-gather-s77bn] OUT 2026-09-16T19:07:54.380465127Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/agent/agent/logs/previous.insecure.log [must-gather-s77bn] OUT 2026-09-16T19:07:54.380533569Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/agent/agent/logs/previous.log [must-gather-s77bn] OUT 2026-09-16T19:07:54.38059255Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/kserve-container/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.380621771Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/kserve-container/kserve-container/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.380629521Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/kserve-container/kserve-container/logs/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.380643942Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/kserve-container/kserve-container/logs/current.log [must-gather-s77bn] OUT 2026-09-16T19:07:54.380761765Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/kserve-container/kserve-container/logs/previous.insecure.log [must-gather-s77bn] OUT 2026-09-16T19:07:54.380853618Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/kserve-container/kserve-container/logs/previous.log [must-gather-s77bn] OUT 2026-09-16T19:07:54.380883778Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/kube-rbac-proxy/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.380895829Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/kube-rbac-proxy/kube-rbac-proxy/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.380900949Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.38092156Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather-s77bn] OUT 2026-09-16T19:07:54.381057243Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather-s77bn] OUT 2026-09-16T19:07:54.381122145Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather-s77bn] OUT 2026-09-16T19:07:54.381152696Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/storage-initializer/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.381159236Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/storage-initializer/storage-initializer/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.381163636Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/storage-initializer/storage-initializer/logs/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.381198867Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/storage-initializer/storage-initializer/logs/current.log [must-gather-s77bn] OUT 2026-09-16T19:07:54.38130084Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/storage-initializer/storage-initializer/logs/previous.insecure.log [must-gather-s77bn] OUT 2026-09-16T19:07:54.381374732Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-58d9d7cffd-jqfr7/storage-initializer/storage-initializer/logs/previous.log [must-gather-s77bn] OUT 2026-09-16T19:07:54.381401493Z namespaces/kserve-ci-e2e-test/policy/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.381448584Z namespaces/kserve-ci-e2e-test/policy/poddisruptionbudgets.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.381487495Z namespaces/kserve-ci-e2e-test/route.openshift.io/ [must-gather-s77bn] OUT 2026-09-16T19:07:54.381533756Z namespaces/kserve-ci-e2e-test/route.openshift.io/routes.yaml [must-gather-s77bn] OUT 2026-09-16T19:07:54.3963983Z [must-gather-s77bn] OUT 2026-09-16T19:07:54.39641339Z sent 1,072 bytes received 96,897 bytes 195,938.00 bytes/sec [must-gather-s77bn] OUT 2026-09-16T19:07:54.39642367Z total size is 1,430,125 speedup is 14.60 [must-gather ] OUT 2026-09-16T19:07:54.510290216Z namespace/openshift-must-gather-vbfj9 deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 72788819-ab62-4d31-ac19-f3bc63c80a89 ClientVersion: 4.21.10 ClusterVersion: Stable at "4.21.33" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing