[must-gather ] OUT 2026-07-31T15:16:34.969405755Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 7cbac07d-3c87-4fe7-9041-a3fb2b17cb2e ClientVersion: 4.21.10 ClusterVersion: Stable at "4.21.27" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing [must-gather ] OUT 2026-07-31T15:16:35.017482858Z namespace/openshift-must-gather-kr6b2 created [must-gather ] OUT 2026-07-31T15:16:35.025523488Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-vjkhx created [must-gather ] OUT 2026-07-31T15:16:35.054867663Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather-mz44f] OUT 2026-07-31T15:17:05.902850352Z waiting for gather to complete [must-gather-mz44f] OUT 2026-07-31T15:17:05.905842799Z downloading gather output [must-gather-mz44f] OUT 2026-07-31T15:17:06.18858131Z receiving incremental file list [must-gather-mz44f] OUT 2026-07-31T15:17:06.200601539Z ./ [must-gather-mz44f] OUT 2026-07-31T15:17:06.200690512Z aggregated-discovery-api.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.200996379Z aggregated-discovery-apis.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.201926453Z event-filter.html [must-gather-mz44f] OUT 2026-07-31T15:17:06.203080413Z timestamp [must-gather-mz44f] OUT 2026-07-31T15:17:06.203218497Z version [must-gather-mz44f] OUT 2026-07-31T15:17:06.204044878Z cluster-scoped-resources/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.204057578Z cluster-scoped-resources/datasciencecluster.opendatahub.io/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.204063868Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.20411006Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/test-dsc.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.204241103Z cluster-scoped-resources/dscinitialization.opendatahub.io/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.204249963Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.204300274Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/test-dsci.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.204404617Z namespaces/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.204416128Z namespaces/kserve-ci-e2e-test/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.204472959Z namespaces/kserve-ci-e2e-test/kserve-ci-e2e-test.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.204558241Z namespaces/kserve-ci-e2e-test/apps.openshift.io/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.204607002Z namespaces/kserve-ci-e2e-test/apps.openshift.io/deploymentconfigs.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.204673834Z namespaces/kserve-ci-e2e-test/apps/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.204717635Z namespaces/kserve-ci-e2e-test/apps/daemonsets.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.204822578Z namespaces/kserve-ci-e2e-test/apps/deployments.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.204923701Z namespaces/kserve-ci-e2e-test/apps/replicasets.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.205004063Z namespaces/kserve-ci-e2e-test/apps/statefulsets.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.205038584Z namespaces/kserve-ci-e2e-test/autoscaling/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.205086425Z namespaces/kserve-ci-e2e-test/autoscaling/horizontalpodautoscalers.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.205131446Z namespaces/kserve-ci-e2e-test/batch/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.205176437Z namespaces/kserve-ci-e2e-test/batch/cronjobs.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.205253569Z namespaces/kserve-ci-e2e-test/batch/jobs.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.20529646Z namespaces/kserve-ci-e2e-test/build.openshift.io/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.205336631Z namespaces/kserve-ci-e2e-test/build.openshift.io/buildconfigs.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.205466884Z namespaces/kserve-ci-e2e-test/build.openshift.io/builds.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.205519106Z namespaces/kserve-ci-e2e-test/core/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.205566857Z namespaces/kserve-ci-e2e-test/core/configmaps.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.20569308Z namespaces/kserve-ci-e2e-test/core/endpoints.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.205775012Z namespaces/kserve-ci-e2e-test/core/events.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.208488052Z namespaces/kserve-ci-e2e-test/core/persistentvolumeclaims.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.208589445Z namespaces/kserve-ci-e2e-test/core/pods.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.208823961Z namespaces/kserve-ci-e2e-test/core/replicationcontrollers.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.208987255Z namespaces/kserve-ci-e2e-test/core/secrets.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.209252642Z namespaces/kserve-ci-e2e-test/core/services.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.209319204Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.209378275Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/endpointslices.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.209466127Z namespaces/kserve-ci-e2e-test/image.openshift.io/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.209513839Z namespaces/kserve-ci-e2e-test/image.openshift.io/imagestreams.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.20957551Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.209629182Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressfirewalls.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.209744835Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressqoses.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.209780266Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.209839827Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/servicemonitors.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.209889498Z namespaces/kserve-ci-e2e-test/networking.k8s.io/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.20994676Z namespaces/kserve-ci-e2e-test/networking.k8s.io/networkpolicies.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.210014922Z namespaces/kserve-ci-e2e-test/pods/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.210023582Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.210077623Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.210620957Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/agent/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.210637148Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/agent/agent/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.210643708Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/agent/agent/logs/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.210676239Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/agent/agent/logs/current.log [must-gather-mz44f] OUT 2026-07-31T15:17:06.210820102Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/agent/agent/logs/previous.insecure.log [must-gather-mz44f] OUT 2026-07-31T15:17:06.210911515Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/agent/agent/logs/previous.log [must-gather-mz44f] OUT 2026-07-31T15:17:06.210952636Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/kserve-container/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.210969156Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/kserve-container/kserve-container/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.210973876Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/kserve-container/kserve-container/logs/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.211019758Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/kserve-container/kserve-container/logs/current.log [must-gather-mz44f] OUT 2026-07-31T15:17:06.211157841Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/kserve-container/kserve-container/logs/previous.insecure.log [must-gather-mz44f] OUT 2026-07-31T15:17:06.211263064Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/kserve-container/kserve-container/logs/previous.log [must-gather-mz44f] OUT 2026-07-31T15:17:06.211304235Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/kube-rbac-proxy/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.211311905Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/kube-rbac-proxy/kube-rbac-proxy/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.211316455Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.211383587Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather-mz44f] OUT 2026-07-31T15:17:06.211553281Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather-mz44f] OUT 2026-07-31T15:17:06.211659034Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather-mz44f] OUT 2026-07-31T15:17:06.211682575Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/storage-initializer/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.211708785Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/storage-initializer/storage-initializer/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.211713495Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/storage-initializer/storage-initializer/logs/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.211762257Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/storage-initializer/storage-initializer/logs/current.log [must-gather-mz44f] OUT 2026-07-31T15:17:06.211914801Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/storage-initializer/storage-initializer/logs/previous.insecure.log [must-gather-mz44f] OUT 2026-07-31T15:17:06.212011673Z namespaces/kserve-ci-e2e-test/pods/isvc-logger-raw-cipn-predictor-7c7ccfc486-m7vwp/storage-initializer/storage-initializer/logs/previous.log [must-gather-mz44f] OUT 2026-07-31T15:17:06.212033573Z namespaces/kserve-ci-e2e-test/policy/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.212102865Z namespaces/kserve-ci-e2e-test/policy/poddisruptionbudgets.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.212154367Z namespaces/kserve-ci-e2e-test/route.openshift.io/ [must-gather-mz44f] OUT 2026-07-31T15:17:06.212205048Z namespaces/kserve-ci-e2e-test/route.openshift.io/routes.yaml [must-gather-mz44f] OUT 2026-07-31T15:17:06.218271574Z [must-gather-mz44f] OUT 2026-07-31T15:17:06.218288745Z sent 1,072 bytes received 94,232 bytes 63,536.00 bytes/sec [must-gather-mz44f] OUT 2026-07-31T15:17:06.218294295Z total size is 1,326,445 speedup is 13.92 [must-gather ] OUT 2026-07-31T15:17:06.332703021Z namespace/openshift-must-gather-kr6b2 deleted Reprinting Cluster State: When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: ClusterID: 7cbac07d-3c87-4fe7-9041-a3fb2b17cb2e ClientVersion: 4.21.10 ClusterVersion: Stable at "4.21.27" ClusterOperators: clusteroperator/authentication is missing clusteroperator/cloud-credential is missing clusteroperator/cluster-autoscaler is missing clusteroperator/config-operator is missing clusteroperator/etcd is missing clusteroperator/machine-api is missing clusteroperator/machine-approver is missing clusteroperator/machine-config is missing clusteroperator/marketplace is missing