[must-gather ] OUT 2026-09-09T21:19:55.785171462Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: e6b7197f-c210-434c-9234-d3f1d38df91e ClientVersion: 4.21.10 ClusterVersion: Stable at "4.21.32" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing [must-gather ] OUT 2026-09-09T21:19:55.844633467Z namespace/openshift-must-gather-j9fnj created [must-gather ] OUT 2026-09-09T21:19:55.852454398Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-lxld8 created [must-gather ] OUT 2026-09-09T21:19:55.880177322Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather-9g8mn] OUT 2026-09-09T21:20:26.627498569Z waiting for gather to complete [must-gather-9g8mn] OUT 2026-09-09T21:20:26.630526343Z downloading gather output [must-gather-9g8mn] OUT 2026-09-09T21:20:26.888824693Z receiving incremental file list [must-gather-9g8mn] OUT 2026-09-09T21:20:26.900536382Z ./ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.900619497Z aggregated-discovery-api.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.900910206Z aggregated-discovery-apis.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.901837945Z event-filter.html [must-gather-9g8mn] OUT 2026-09-09T21:20:26.903054193Z timestamp [must-gather-9g8mn] OUT 2026-09-09T21:20:26.903191822Z version [must-gather-9g8mn] OUT 2026-09-09T21:20:26.904074098Z cluster-scoped-resources/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.904089899Z cluster-scoped-resources/datasciencecluster.opendatahub.io/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.90409552Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.904145513Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/test-dsc.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.904217747Z cluster-scoped-resources/dscinitialization.opendatahub.io/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.904227288Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.904287692Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/test-dsci.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.904309433Z namespaces/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.904316244Z namespaces/kserve-ci-e2e-test/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.904374267Z namespaces/kserve-ci-e2e-test/kserve-ci-e2e-test.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.904449572Z namespaces/kserve-ci-e2e-test/apps.openshift.io/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.904505966Z namespaces/kserve-ci-e2e-test/apps.openshift.io/deploymentconfigs.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.904564139Z namespaces/kserve-ci-e2e-test/apps/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.904605952Z namespaces/kserve-ci-e2e-test/apps/daemonsets.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.904707869Z namespaces/kserve-ci-e2e-test/apps/deployments.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.904789964Z namespaces/kserve-ci-e2e-test/apps/replicasets.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.904875599Z namespaces/kserve-ci-e2e-test/apps/statefulsets.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.904918082Z namespaces/kserve-ci-e2e-test/autoscaling/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.905019389Z namespaces/kserve-ci-e2e-test/autoscaling/horizontalpodautoscalers.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.905077422Z namespaces/kserve-ci-e2e-test/batch/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.905127806Z namespaces/kserve-ci-e2e-test/batch/cronjobs.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.905225142Z namespaces/kserve-ci-e2e-test/batch/jobs.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.905274325Z namespaces/kserve-ci-e2e-test/build.openshift.io/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.90535551Z namespaces/kserve-ci-e2e-test/build.openshift.io/buildconfigs.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.905452986Z namespaces/kserve-ci-e2e-test/build.openshift.io/builds.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.905498779Z namespaces/kserve-ci-e2e-test/core/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.905557243Z namespaces/kserve-ci-e2e-test/core/configmaps.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.905703952Z namespaces/kserve-ci-e2e-test/core/endpoints.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.905800999Z namespaces/kserve-ci-e2e-test/core/events.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.908580966Z namespaces/kserve-ci-e2e-test/core/persistentvolumeclaims.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.908663672Z namespaces/kserve-ci-e2e-test/core/pods.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.908864734Z namespaces/kserve-ci-e2e-test/core/replicationcontrollers.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.909027675Z namespaces/kserve-ci-e2e-test/core/secrets.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.909231068Z namespaces/kserve-ci-e2e-test/core/services.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.909291302Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.909343185Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/endpointslices.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.909394458Z namespaces/kserve-ci-e2e-test/image.openshift.io/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.909453632Z namespaces/kserve-ci-e2e-test/image.openshift.io/imagestreams.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.909506706Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.90957423Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressfirewalls.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.909669126Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressqoses.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.909715179Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.909769142Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/servicemonitors.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.909814775Z namespaces/kserve-ci-e2e-test/networking.k8s.io/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.909864769Z namespaces/kserve-ci-e2e-test/networking.k8s.io/networkpolicies.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.909910561Z namespaces/kserve-ci-e2e-test/pods/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.909917112Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.910006598Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.910242233Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/agent/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.910249973Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/agent/agent/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.910253713Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/agent/agent/logs/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.910312107Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/agent/agent/logs/current.log [must-gather-9g8mn] OUT 2026-09-09T21:20:26.910430195Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/agent/agent/logs/previous.insecure.log [must-gather-9g8mn] OUT 2026-09-09T21:20:26.91051226Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/agent/agent/logs/previous.log [must-gather-9g8mn] OUT 2026-09-09T21:20:26.910553413Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/kserve-container/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.910569314Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/kserve-container/kserve-container/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.910573674Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/kserve-container/kserve-container/logs/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.910602876Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/kserve-container/kserve-container/logs/current.log [must-gather-9g8mn] OUT 2026-09-09T21:20:26.910725813Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/kserve-container/kserve-container/logs/previous.insecure.log [must-gather-9g8mn] OUT 2026-09-09T21:20:26.910804719Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/kserve-container/kserve-container/logs/previous.log [must-gather-9g8mn] OUT 2026-09-09T21:20:26.910844051Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/kube-rbac-proxy/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.910851572Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/kube-rbac-proxy/kube-rbac-proxy/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.910855542Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.910900095Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather-9g8mn] OUT 2026-09-09T21:20:26.911100017Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather-9g8mn] OUT 2026-09-09T21:20:26.911167622Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather-9g8mn] OUT 2026-09-09T21:20:26.911284029Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/storage-initializer/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.911302541Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/storage-initializer/storage-initializer/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.911309711Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/storage-initializer/storage-initializer/logs/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.911328522Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/storage-initializer/storage-initializer/logs/current.log [must-gather-9g8mn] OUT 2026-09-09T21:20:26.911435299Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/storage-initializer/storage-initializer/logs/previous.insecure.log [must-gather-9g8mn] OUT 2026-09-09T21:20:26.911512694Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-577b6fff5f-7qr8c/storage-initializer/storage-initializer/logs/previous.log [must-gather-9g8mn] OUT 2026-09-09T21:20:26.911553277Z namespaces/kserve-ci-e2e-test/policy/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.91160243Z namespaces/kserve-ci-e2e-test/policy/poddisruptionbudgets.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.911651553Z namespaces/kserve-ci-e2e-test/route.openshift.io/ [must-gather-9g8mn] OUT 2026-09-09T21:20:26.911701856Z namespaces/kserve-ci-e2e-test/route.openshift.io/routes.yaml [must-gather-9g8mn] OUT 2026-09-09T21:20:26.917568171Z [must-gather-9g8mn] OUT 2026-09-09T21:20:26.917593953Z sent 1,072 bytes received 99,894 bytes 201,932.00 bytes/sec [must-gather-9g8mn] OUT 2026-09-09T21:20:26.917601113Z total size is 1,448,709 speedup is 14.35 [must-gather ] OUT 2026-09-09T21:20:27.031310646Z namespace/openshift-must-gather-j9fnj deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: e6b7197f-c210-434c-9234-d3f1d38df91e ClientVersion: 4.21.10 ClusterVersion: Stable at "4.21.32" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing