
--- BVT pytest attempt 2026-07-09T21:11:33.746946+00:00 (/.local/bin/uv --directory /home/odh/opendatahub-tests...) ---
[2m2026-07-09T21:11:35.751032Z[0m [[32m[1minfo     [0m] [1mWriting tests log to /home/odh/opendatahub-tests/results/pytest-tests.log[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:35.751455Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:35.913408Z[0m [[32m[1minfo     [0m] [1mkind: DataScienceCluster api version: datasciencecluster.opendatahub.io/v2[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:35.991967Z[0m [[32m[1minfo     [0m] [1mkind: DSCInitialization api version: dscinitialization.opendatahub.io/v2[0m [[0m[1m[34mocp_resources.resource[0m][0m
============================= test session starts ==============================
platform linux -- Python 3.14.6, pytest-9.1.1, pluggy-1.6.0 -- /home/odh/opendatahub-tests/.venv/bin/python3
cachedir: .pytest_cache
metadata: {'Python': '3.14.6', 'Platform': 'Linux-5.14.0-570.120.1.el9_6.x86_64-x86_64-with-glibc2.42', 'Packages': {'pytest': '9.1.1', 'pluggy': '1.6.0'}, 'Plugins': {'anyio': '4.14.0', 'langsmith': '0.9.0', 'asyncio': '1.4.0', 'dependency': '0.6.1', 'html': '4.2.0', 'metadata': '3.1.1', 'order': '1.5.0', 'progress': '1.4.0', 'testconfig': '0.2.0', 'xdist': '3.8.0', 'syrupy': '5.3.2'}}
rootdir: /home/odh/opendatahub-tests
configfile: pytest.ini
plugins: anyio-4.14.0, langsmith-0.9.0, asyncio-1.4.0, dependency-0.6.1, html-4.2.0, metadata-3.1.1, order-1.5.0, progress-1.4.0, testconfig-0.2.0, xdist-3.8.0, syrupy-5.3.2
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collecting ... collected 224 items / 219 deselected / 5 selected

tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py::TestKserveTokenAuthenticationRawForRest::test_model_authentication_using_rest_raw[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] [1m
----- test_model_authentication_using_rest_raw[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] -----[0m
[1m-------------------------------------------------------- SETUP --------------------------------------------------------[0m
[2m2026-07-09T21:11:36.462491Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: admin_client[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:36.462688Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:36.463884Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: dsc_resource[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:36.464029Z[0m [[32m[1minfo     [0m] [1mkind: DataScienceCluster api version: datasciencecluster.opendatahub.io/v2[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:36.548832Z[0m [[32m[1minfo     [0m] [1mExecuting package fixture: enabled_kserve_in_dsc[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:36.569255Z[0m [[33m[1mwarning  [0m] [1mComponent kserve was already set to managementState Managed[0m [[0m[1m[34mutilities.data_science_cluster_utils[0m][0m [36mname[0m=[35mutilities.data_science_cluster_utils[0m
[2m2026-07-09T21:11:36.570243Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: event_loop_policy[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:36.570406Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: _syrupy_apply_ide_patches[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:36.570536Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: tmp_path_factory[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:36.570759Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: tests_tmp_dir[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:36.571480Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: os_path_environment[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:36.571604Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: tmpdir_factory[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:36.571798Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: bin_directory[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:36.572631Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: oc_binary_path[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:36.600072Z[0m [[32m[1minfo     [0m] [1mkind: ConsoleCLIDownload api version: console.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:36.641568Z[0m [[32m[1minfo     [0m] [1mAll oc console cli download links: ['https://downloads-openshift-console.apps.rosa.rh-nightly-pm.nuth.p3.openshiftapps.com/amd64/linux/oc.tar'][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:11:36.641762Z[0m [[32m[1minfo     [0m] [1mDownloading archive using: url=https://downloads-openshift-console.apps.rosa.rh-nightly-pm.nuth.p3.openshiftapps.com/amd64/linux/oc.tar[0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:11:37.717434Z[0m [[32m[1minfo     [0m] [1mExtract the downloaded archive.[0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:11:40.281776Z[0m [[32m[1minfo     [0m] [1mDownloaded file: ['oc']       [0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:11:40.488849Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: bin_directory_to_os_path[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:40.483787Z[0m [[32m[1minfo     [0m] [1mOC binary path: /tmp/pytest-2mGtbdLhWwL84YXJsPFXNE/bin0/oc[0m [[0m[1m[34mtests.conftest[0m][0m [36mname[0m=[35mtests.conftest[0m
[2m2026-07-09T21:11:40.483932Z[0m [[32m[1minfo     [0m] [1mAdding /tmp/pytest-2mGtbdLhWwL84YXJsPFXNE/bin0 to $PATH[0m [[0m[1m[34mtests.conftest[0m][0m [36mname[0m=[35mtests.conftest[0m
[2m2026-07-09T21:11:40.489269Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: nodes[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:41.839286Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: dsci_resource[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:41.839438Z[0m [[32m[1minfo     [0m] [1mkind: DSCInitialization api version: dscinitialization.opendatahub.io/v2[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:41.858862Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: record_testsuite_property[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:41.859024Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: junitxml_plugin[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:41.859167Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: cluster_sanity_scope_session[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:41.858244Z[0m [[32m[1minfo     [0m] [1mCheck cluster sanity.         [0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:11:41.859442Z[0m [[32m[1minfo     [0m] [1mVerify all nodes are in a healthy condition.[0m [[0m[1m[34mocp_utilities.infra[0m][0m
[2m2026-07-09T21:11:41.916889Z[0m [[32m[1minfo     [0m] [1mVerify all nodes are schedulable.[0m [[0m[1m[34mocp_utilities.infra[0m][0m
[2m2026-07-09T21:11:41.974657Z[0m [[32m[1minfo     [0m] [1mWait for DSCI default-dsci to be in Ready status.[0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:11:41.975311Z[0m [[32m[1minfo     [0m] [1mGet DSCInitialization default-dsci status[0m [[0m[1m[34mocp_resources DSCInitialization[0m][0m
[2m2026-07-09T21:11:41.993380Z[0m [[32m[1minfo     [0m] [1mWait for DSC default-dsc are Ready.[0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:11:41.993998Z[0m [[32m[1minfo     [0m] [1mGet DataScienceCluster default-dsc status[0m [[0m[1m[34mocp_resources DataScienceCluster[0m][0m
[2m2026-07-09T21:11:42.015809Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: autouse_fixtures[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.015956Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: kserve_health_check[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.058209Z[0m [[32m[1minfo     [0m] [1mkind: Deployment api version: apps/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:42.136255Z[0m [[32m[1minfo     [0m] [1mKServe component health check passed[0m [[0m[1m[34mtests.model_serving.model_server.kserve.conftest[0m][0m
[2m2026-07-09T21:11:42.136404Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: pytestconfig[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.136534Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: aws_access_key_id[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.136660Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: aws_secret_access_key[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.136823Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: ci_s3_bucket_endpoint[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.136965Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: valid_aws_config[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.190005Z[0m [[32m[1minfo     [0m] [1mAWS credentials validated successfully via STS GetCallerIdentity[0m [[0m[1m[34mtests.conftest[0m][0m [36mname[0m=[35mtests.conftest[0m
[2m2026-07-09T21:11:42.192527Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: use_unprivileged_client[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.192648Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: kubconfig_filepath[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.192792Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: cluster_authentication[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.192914Z[0m [[32m[1minfo     [0m] [1mkind: Authentication api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:42.210902Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: is_byoidc[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.232490Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: non_admin_user_password[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.232638Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: unprivileged_client[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.228971Z[0m [[33m[1mwarning  [0m] [1mUnprivileged client is not enabled, using admin client[0m [[0m[1m[34mtests.conftest[0m][0m [36mname[0m=[35mtests.conftest[0m
[2m2026-07-09T21:11:42.232912Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: teardown_resources[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.233044Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: ci_s3_bucket_name[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.233174Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: ci_s3_bucket_region[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.233289Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: models_s3_bucket_name[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.233417Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: models_s3_bucket_region[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.233531Z[0m [[32m[1minfo     [0m] [1mExecuting session fixture: models_s3_bucket_endpoint[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.233644Z[0m [[32m[1minfo     [0m] [1mExecuting class fixture: unprivileged_model_namespace[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.233790Z[0m [[32m[1minfo     [0m] [1mkind: ProjectRequest api version: project.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:42.233910Z[0m [[32m[1minfo     [0m] [1mCreate ProjectRequest test-kserve-raw-token-authentication[0m [[0m[1m[34mocp_resources ProjectRequest[0m][0m
[2m2026-07-09T21:11:42.234024Z[0m [[32m[1minfo     [0m] [1mPosting {'apiVersion': 'project.openshift.io/v1', 'kind': 'ProjectRequest', 'metadata': {'name': 'test-kserve-raw-token-authentication'}}[0m [[0m[1m[34mocp_resources ProjectRequest[0m][0m
[2m2026-07-09T21:11:42.347083Z[0m [[32m[1minfo     [0m] [1mkind: Project api version: project.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:42.347405Z[0m [[32m[1minfo     [0m] [1mWait for Project test-kserve-raw-token-authentication status to be Active[0m [[0m[1m[34mocp_resources Project[0m][0m
[2m2026-07-09T21:11:42.371923Z[0m [[32m[1minfo     [0m] [1mStatus of Project test-kserve-raw-token-authentication is Active[0m [[0m[1m[34mocp_resources Project[0m][0m
[2m2026-07-09T21:11:42.372081Z[0m [[32m[1minfo     [0m] [1mExecuting class fixture: http_s3_ovms_serving_runtime[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.372228Z[0m [[32m[1minfo     [0m] [1mkind: Template api version: template.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:42.493019Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:42.493417Z[0m [[32m[1minfo     [0m] [1mCreate ServingRuntime http-mnist-runtime[0m [[0m[1m[34mocp_resources ServingRuntime[0m][0m
[2m2026-07-09T21:11:42.493584Z[0m [[32m[1minfo     [0m] [1mPosting {'apiVersion': 'serving.kserve.io/v1alpha1', 'kind': 'ServingRuntime', 'metadata': {'annotations': {'opendatahub.io/recommended-accelerators': '["nvidia.com/gpu"]', 'opendatahub.io/runtime-version': 'v2026.1.0', 'openshift.io/display-name': 'OpenVINO Model Server'}, 'labels': {'opendatahub.io/dashboard': 'true'}, 'name': 'http-mnist-runtime', 'namespace': 'test-kserve-raw-token-authentication'}, 'spec': {'annotations': {'opendatahub.io/kserve-runtime': 'ovms', 'prometheus.io/path': '/metrics', 'prometheus.io/port': '8888'}, 'containers': [{'args': ['--model_name={{.Name}}', '--port=8001', '--rest_port=8888', '--model_path=/mnt/models', '--file_system_poll_wait_seconds=0', '--metrics_enable'], 'image': 'registry.redhat.io/rhoai/odh-openvino-model-server-rhel9@sha256:8c1b225509bfdc43756195f34a356d1e2ebd58fcd83fda3d37380dd7f5304df8', 'name': 'kserve-container', 'ports': [{'containerPort': 8888, 'protocol': 'TCP'}]}], 'multiModel': False, 'protocolVersions': ['v2', 'grpc-v2'], 'supportedModelFormats': [{'autoSelect': True, 'name': 'openvino_ir', 'version': 'opset13'}, {'name': 'onnx', 'version': '1'}, {'autoSelect': True, 'name': 'tensorflow', 'version': '1'}, {'autoSelect': True, 'name': 'tensorflow', 'version': '2'}, {'autoSelect': True, 'name': 'paddle', 'version': '2'}, {'autoSelect': True, 'name': 'pytorch', 'version': '2'}]}}[0m [[0m[1m[34mocp_resources ServingRuntime[0m][0m
[2m2026-07-09T21:11:42.553778Z[0m [[32m[1minfo     [0m] [1mExecuting class fixture: ci_endpoint_s3_secret[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.578293Z[0m [[32m[1minfo     [0m] [1mCreate Secret ci-bucket-secret[0m [[0m[1m[34mocp_resources Secret[0m][0m
[2m2026-07-09T21:11:42.578559Z[0m [[32m[1minfo     [0m] [1mPosting {'apiVersion': 'v1', 'kind': 'Secret', 'metadata': {'name': 'ci-bucket-secret', 'labels': {'opendatahub.io/managed': 'true', 'opendatahub.io/dashboard': 'true'}, 'annotations': {'opendatahub.io/connection-type': 's3', 'serving.kserve.io/s3-endpoint': 's3.amazonaws.com/', 'serving.kserve.io/s3-region': 'us-east-1', 'serving.kserve.io/s3-useanoncredential': 'false', 'serving.kserve.io/s3-verifyssl': '0', 'serving.kserve.io/s3-usehttps': '1'}, 'namespace': 'test-kserve-raw-token-authentication'}, 'data': '*******'}[0m [[0m[1m[34mocp_resources Secret[0m][0m
[2m2026-07-09T21:11:42.624118Z[0m [[32m[1minfo     [0m] [1mWait until Secret ci-bucket-secret is created[0m [[0m[1m[34mocp_resources Secret[0m][0m
[2m2026-07-09T21:11:42.642417Z[0m [[32m[1minfo     [0m] [1mExecuting class fixture: models_endpoint_s3_secret[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.663842Z[0m [[32m[1minfo     [0m] [1mCreate Secret models-bucket-secret[0m [[0m[1m[34mocp_resources Secret[0m][0m
[2m2026-07-09T21:11:42.664148Z[0m [[32m[1minfo     [0m] [1mPosting {'apiVersion': 'v1', 'kind': 'Secret', 'metadata': {'name': 'models-bucket-secret', 'labels': {'opendatahub.io/managed': 'true', 'opendatahub.io/dashboard': 'true'}, 'annotations': {'opendatahub.io/connection-type': 's3', 'serving.kserve.io/s3-endpoint': 's3.amazonaws.com/', 'serving.kserve.io/s3-region': 'us-east-1', 'serving.kserve.io/s3-useanoncredential': 'false', 'serving.kserve.io/s3-verifyssl': '0', 'serving.kserve.io/s3-usehttps': '1'}, 'namespace': 'test-kserve-raw-token-authentication'}, 'data': '*******'}[0m [[0m[1m[34mocp_resources Secret[0m][0m
[2m2026-07-09T21:11:42.711978Z[0m [[32m[1minfo     [0m] [1mWait until Secret models-bucket-secret is created[0m [[0m[1m[34mocp_resources Secret[0m][0m
[2m2026-07-09T21:11:42.731640Z[0m [[32m[1minfo     [0m] [1mExecuting class fixture: model_service_account[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.732094Z[0m [[32m[1minfo     [0m] [1mCreate ServiceAccount models-bucket-sa[0m [[0m[1m[34mocp_resources ServiceAccount[0m][0m
[2m2026-07-09T21:11:42.732398Z[0m [[32m[1minfo     [0m] [1mPosting {'apiVersion': 'v1', 'kind': 'ServiceAccount', 'metadata': {'name': 'models-bucket-sa', 'namespace': 'test-kserve-raw-token-authentication'}, 'secrets': [{'name': 'models-bucket-secret'}]}[0m [[0m[1m[34mocp_resources ServiceAccount[0m][0m
[2m2026-07-09T21:11:42.775028Z[0m [[32m[1minfo     [0m] [1mExecuting class fixture: http_s3_ovms_raw_inference_service[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:42.797235Z[0m [[32m[1minfo     [0m] [1mkind: InferenceService api version: serving.kserve.io/v1beta1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:42.797524Z[0m [[32m[1minfo     [0m] [1mCreate InferenceService http-onnx[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:11:42.798290Z[0m [[32m[1minfo     [0m] [1mPosting {'apiVersion': 'serving.kserve.io/v1beta1', 'kind': 'InferenceService', 'metadata': {'name': 'http-onnx', 'labels': {'networking.kserve.io/visibility': 'exposed'}, 'annotations': {'serving.kserve.io/deploymentMode': 'RawDeployment', 'security.opendatahub.io/enable-auth': 'true'}, 'namespace': 'test-kserve-raw-token-authentication'}, 'spec': {'predictor': {'model': {'modelFormat': {'name': 'openvino_ir'}, 'version': None, 'runtime': 'http-mnist-runtime', 'storage': {'key': 'ci-bucket-secret', 'path': '/test-dir/'}}, 'serviceAccountName': 'models-bucket-sa'}}}[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:11:43.091244Z[0m [[32m[1minfo     [0m] [1mWaiting for pods to be created[0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:11:44.129400Z[0m [[32m[1minfo     [0m] [1mWaiting for pods to be created[0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:11:44.203515Z[0m [[32m[1minfo     [0m] [1mVerifying no failed pods      [0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:11:54.424159Z[0m [[32m[1minfo     [0m] [1mkind: Deployment api version: apps/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:54.443089Z[0m [[32m[1minfo     [0m] [1mWaiting for inference deployment replicas to complete[0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:11:54.519543Z[0m [[32m[1minfo     [0m] [1mWait for Deployment http-onnx-predictor to be deployed: True[0m [[0m[1m[34mocp_resources Deployment[0m][0m
[2m2026-07-09T21:11:54.557628Z[0m [[32m[1minfo     [0m] [1mWait for InferenceService/http-onnx's 'Ready' condition to be 'True'[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:11:54.557785Z[0m [[32m[1minfo     [0m] [1mWait until InferenceService http-onnx is created[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:11:54.654356Z[0m [[32m[1minfo     [0m] [1mExecuting class fixture: http_raw_view_role[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:54.654526Z[0m [[32m[1minfo     [0m] [1mkind: Role api version: rbac.authorization.k8s.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:54.654675Z[0m [[32m[1minfo     [0m] [1mCreate Role http-onnx-view    [0m [[0m[1m[34mocp_resources Role[0m][0m
[2m2026-07-09T21:11:54.654900Z[0m [[32m[1minfo     [0m] [1mPosting {'apiVersion': 'rbac.authorization.k8s.io/v1', 'kind': 'Role', 'metadata': {'name': 'http-onnx-view', 'namespace': 'test-kserve-raw-token-authentication'}, 'rules': [{'apiGroups': ['serving.kserve.io'], 'resources': ['inferenceservices'], 'verbs': ['get'], 'resourceNames': ['http-onnx']}]}[0m [[0m[1m[34mocp_resources Role[0m][0m
[2m2026-07-09T21:11:54.693390Z[0m [[32m[1minfo     [0m] [1mExecuting class fixture: http_raw_role_binding[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:54.693544Z[0m [[32m[1minfo     [0m] [1mkind: RoleBinding api version: rbac.authorization.k8s.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:54.693691Z[0m [[32m[1minfo     [0m] [1mCreate RoleBinding http-models-bucket-sa-view[0m [[0m[1m[34mocp_resources RoleBinding[0m][0m
[2m2026-07-09T21:11:54.693868Z[0m [[32m[1minfo     [0m] [1mPosting {'apiVersion': 'rbac.authorization.k8s.io/v1', 'kind': 'RoleBinding', 'metadata': {'name': 'http-models-bucket-sa-view', 'namespace': 'test-kserve-raw-token-authentication'}, 'subjects': [{'kind': 'ServiceAccount', 'name': 'models-bucket-sa'}], 'roleRef': {'kind': 'Role', 'name': 'http-onnx-view', 'apiGroup': 'rbac.authorization.k8s.io'}}[0m [[0m[1m[34mocp_resources RoleBinding[0m][0m
[2m2026-07-09T21:11:54.736285Z[0m [[32m[1minfo     [0m] [1mExecuting class fixture: http_raw_inference_token[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:54.736451Z[0m [[32m[1minfo     [0m] [1mRunning oc create token -n test-kserve-raw-token-authentication models-bucket-sa command[0m [[0m[1m[34mpyhelper_utils.shell[0m][0m
[1m--------------------------------------------------------- CALL ---------------------------------------------------------[0m
[2m2026-07-09T21:11:54.980288Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:55.038532Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:55.045367Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:55.126131Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:11:55.126250Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:11:55.126309Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:11:55.128961Z[0m [[32m[1minfo     [0m] [1mRunning Hide By User command  [0m [[0m[1m[34mpyhelper_utils.shell[0m][0m
[2m2026-07-09T21:11:55.253780Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
content-length: 53
content-type: application/json; charset=utf-8
date: Thu, 09 Jul 2026 21:11:55 GMT
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=c6a20d1ee5048ba00be106b66e64fabd; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
FAILED
_ TestKserveTokenAuthenticationRawForRest.test_model_authentication_using_rest_raw[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] _
Traceback (most recent call last):
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call
    result: TResult | None = func()
                             ~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 184, in pytest_runtest_call
    item.runtest()
    ~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1707, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 167, in pytest_pyfunc_call
    result = testfunction(**testargs)
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py", line 39, in test_model_authentication_using_rest_raw
    verify_inference_response(
    ~~~~~~~~~~~~~~~~~~~~~~~~~^
        inference_service=http_s3_ovms_raw_inference_service,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
        token=http_raw_inference_token,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/utils.py", line 154, in verify_inference_response
    assert re.search(expected_response_text, formatted_res), (  # type: ignore[arg-type]
           ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Expected: {"model_name":"http-onnx","model_version":"1","outputs":\[\{"name":"Plus214_Output_0","shape":\[1,10\],"datatype":"FP32","data":\[.*\]}]} not found in: {"error":"Modelwithrequestedversionisnotfound"}
_ 1 of 5 completed, 0 Pass, 1 Fail, 0 Skip, 0 XPass, 0 XFail, 0 Error, 0 ReRun _

tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py::TestKserveTokenAuthenticationRawForRest::test_disabled_raw_model_authentication[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] [1m
TEST: TestKserveTokenAuthenticationRawForRest.test_model_authentication_using_rest_raw[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] STATUS: [0;31mFAILED[0m[0m
[2m2026-07-09T21:11:55.268852Z[0m [[31m[1merror    [0m] [1mTraceback (most recent call last):
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call
    result: TResult | None = func()
                             ~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 184, in pytest_runtest_call
    item.runtest()
    ~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1707, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 167, in pytest_pyfunc_call
    result = testfunction(**testargs)
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py", line 39, in test_model_authentication_using_rest_raw
    verify_inference_response(
    ~~~~~~~~~~~~~~~~~~~~~~~~~^
        inference_service=http_s3_ovms_raw_inference_service,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
        token=http_raw_inference_token,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/utils.py", line 154, in verify_inference_response
    assert re.search(expected_response_text, formatted_res), (  # type: ignore[arg-type]
           ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Expected: {"model_name":"http-onnx","model_version":"1","outputs":\[\{"name":"Plus214_Output_0","shape":\[1,10\],"datatype":"FP32","data":\[.*\]}]} not found in: {"error":"Modelwithrequestedversionisnotfound"}[0m [[0m[1m[34mconftest[0m][0m
[1m------------------------------------------------------- TEARDOWN -------------------------------------------------------[0m
[1m
------ test_disabled_raw_model_authentication[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] ------[0m
[1m-------------------------------------------------------- SETUP --------------------------------------------------------[0m
[2m2026-07-09T21:11:55.269336Z[0m [[32m[1minfo     [0m] [1mExecuting function fixture: patched_remove_raw_authentication_isvc[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:11:55.269442Z[0m [[32m[1minfo     [0m] [1mResourceEdit: Backing up old data[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:55.287860Z[0m [[32m[1minfo     [0m] [1mResourceEdits: Updating data for resource InferenceService http-onnx[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:55.288015Z[0m [[32m[1minfo     [0m] [1mUpdate InferenceService http-onnx:
{'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'false'}, 'name': 'http-onnx'}}[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:11:55.340351Z[0m [[32m[1minfo     [0m] [1mWait for InferenceService/http-onnx's 'Ready' condition to be 'True'[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:11:55.340513Z[0m [[32m[1minfo     [0m] [1mWait until InferenceService http-onnx is created[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:11:55.419733Z[0m [[32m[1minfo     [0m] [1mWaiting for inference deployment replicas to complete[0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:11:56.594566Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:56.651814Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:56.657895Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:11:56.737802Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:11:56.737999Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:11:56.738048Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:11:56.813619Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 401 Unauthorized
content-type: text/plain; charset=utf-8
x-content-type-options: nosniff
date: Thu, 09 Jul 2026 21:11:56 GMT
content-length: 13
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=c6a20d1ee5048ba00be106b66e64fabd; path=/; HttpOnly; Secure; SameSite=None

Unauthorized
[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:01.853867Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:01.911360Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:01.917418Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:01.995063Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:12:01.995195Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:12:01.995242Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:02.057951Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-methods: *
access-control-allow-headers: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=67086363412e85b4fcf8662b7b6d6877; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:07.098172Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:07.155573Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:07.162040Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:07.305337Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:12:07.305437Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:12:07.305489Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:07.428932Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:12.472120Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:12.529506Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:12.660644Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:12.742346Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:12:12.742490Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:12:12.742538Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:12.822927Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:17.863144Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:17.919652Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:17.926071Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:18.019950Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:12:18.020085Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:12:18.020131Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:18.091220Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:23.132090Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:23.189095Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:23.195594Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:23.289757Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:12:23.289902Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:12:23.289963Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:23.349000Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:28.389220Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:28.446594Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:28.453014Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:28.536183Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:12:28.536318Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:12:28.536380Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:28.600562Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:33.640097Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:33.697897Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:33.704237Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:33.798615Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:12:33.798778Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:12:33.798832Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:33.870758Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:38.913441Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:38.971681Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:38.979850Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:39.126218Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:12:39.126361Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:12:39.126417Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:39.185240Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:44.226261Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:44.286413Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:44.290897Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:44.372293Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:12:44.372470Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:12:44.372547Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:44.448854Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:49.490362Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:49.554390Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:49.560903Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:49.637411Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:12:49.637572Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:12:49.637628Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:49.705379Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:54.745843Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:54.804875Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:54.811478Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:12:54.901617Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:12:54.901840Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:12:54.901890Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:12:54.966690Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:00.009973Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:00.071650Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:00.081751Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:00.159779Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:13:00.159943Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:13:00.159996Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:00.284481Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:05.326113Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:05.388025Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:05.394616Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:05.486761Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:13:05.486925Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:13:05.486979Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:05.555826Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:10.598476Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:10.658779Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:10.665368Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:10.761558Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:13:10.761727Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:13:10.761777Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:10.839492Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:15.881060Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:15.941760Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:15.948385Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:16.084983Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:13:16.085117Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:13:16.085162Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:16.155467Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:21.198171Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:21.259327Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:21.271685Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:21.358668Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:13:21.358865Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:13:21.358911Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:21.425523Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:26.467260Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:26.528845Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:26.537966Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:26.624674Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:13:26.624848Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:13:26.624902Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:26.749460Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:31.793371Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:31.852014Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:31.858319Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:32.008771Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:13:32.008925Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:13:32.008983Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:32.069612Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:37.111417Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:37.171665Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:37.176008Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:37.264175Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:13:37.264309Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:13:37.264365Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:37.389149Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:42.430794Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:42.491451Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:42.495722Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:42.573551Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:13:42.573674Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:13:42.573746Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:42.640037Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:47.680155Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:47.737265Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:47.744189Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:47.841290Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:13:47.841437Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:13:47.841491Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:47.906152Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:52.947007Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:53.004029Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:53.020969Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:53.120517Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:13:53.120799Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:13:53.120893Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:53.185917Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:58.228586Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:58.288692Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:58.295466Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:13:58.398014Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:13:58.398138Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:13:58.398183Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:13:58.463110Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:03.504800Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:03.564570Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:03.570478Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:03.650142Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:14:03.650271Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:14:03.650316Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:03.875482Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:08.915585Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:08.974202Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:08.980862Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:09.063329Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:14:09.063460Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:14:09.063507Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:09.129069Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:14.170165Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:14.228053Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:14.234613Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:14.314554Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:14:14.314734Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:14:14.314787Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:14.379858Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:19.420987Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:19.479587Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:19.489169Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:19.601691Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:14:19.601861Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:14:19.601923Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:19.720653Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:24.765531Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:24.826448Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:24.833409Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:25.066015Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:14:25.066152Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:14:25.066201Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:25.187841Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:30.229222Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:30.289183Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:30.299045Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:30.388018Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:14:30.388157Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:14:30.388201Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:30.457016Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:35.497660Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:35.557560Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:35.561871Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:35.659086Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:14:35.659227Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:14:35.659273Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:35.726721Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:40.766924Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:40.824150Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:40.830646Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:40.910226Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:14:40.910346Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:14:40.910413Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:40.971753Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:46.012439Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:46.070958Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:46.077392Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:46.154374Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:14:46.154536Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:14:46.154590Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:46.216159Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:51.257120Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:51.315312Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:51.322155Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:51.416515Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:14:51.416657Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:14:51.416740Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:51.480689Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:56.521719Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:56.581080Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:56.587248Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:14:56.681377Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:14:56.681514Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:14:56.681567Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:14:56.755302Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:01.795323Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:01.852200Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:01.862079Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:01.949103Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:15:01.949244Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:15:01.949291Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:02.017941Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:07.059833Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:07.121078Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:07.127958Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:07.213770Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:15:07.213910Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:15:07.213972Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:07.280292Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:12.321836Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:12.381322Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:12.387406Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:12.471071Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:15:12.471235Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:15:12.471297Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:12.539762Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:17.582488Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:17.642020Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:17.648448Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:17.739142Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:15:17.739279Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:15:17.739346Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:17.812508Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:22.853490Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:22.913016Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:22.919311Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:23.024010Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:15:23.024107Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:15:23.024149Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:23.099107Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:28.140768Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:28.200412Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:28.206329Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:28.291574Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:15:28.291684Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:15:28.291783Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:28.355258Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:33.396083Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:33.455044Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:33.461685Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:33.564259Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:15:33.564397Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:15:33.564442Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:33.628335Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:38.669404Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:38.728105Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:38.737523Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:38.976398Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:15:38.976534Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:15:38.976580Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:39.052219Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:44.093567Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:44.153437Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:44.159807Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:44.250921Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:15:44.251054Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:15:44.251100Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:44.325646Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:49.367272Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:49.426583Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:49.432860Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:49.514602Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:15:49.514925Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:15:49.514980Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:49.574399Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:54.615031Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:54.672751Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:54.679515Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:54.784737Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:15:54.784906Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:15:54.784959Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:54.851367Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:15:59.891301Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:59.948729Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:15:59.955198Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:00.047289Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:16:00.047447Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:16:00.047502Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:00.112033Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:05.152935Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:05.211871Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:05.220054Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:05.310512Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:16:05.310668Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:16:05.310772Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:05.373231Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:10.417675Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:10.475365Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:10.486412Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:10.572109Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:16:10.572393Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:16:10.572478Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:10.641796Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:15.681365Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:15.738516Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:15.745143Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:15.820683Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:16:15.820832Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:16:15.820877Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:15.888017Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:20.927890Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:20.983992Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:20.990240Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:21.073249Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:16:21.073357Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:16:21.073412Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:21.140378Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:26.180571Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:26.237230Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:26.244036Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:26.334271Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:16:26.334414Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:16:26.334470Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:26.469937Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:31.510061Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:31.571333Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:31.577383Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:31.647646Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:16:31.647767Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:16:31.647818Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:31.707833Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:36.748356Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:36.806007Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:36.812588Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:36.921569Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:16:36.921806Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:16:36.921871Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:36.984081Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:42.025458Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:42.083685Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:42.091960Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:42.187212Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:16:42.187347Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:16:42.187404Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:42.250033Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:47.290149Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:47.354630Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:47.360598Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:47.446633Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:16:47.446831Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:16:47.446875Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:47.518068Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:52.558937Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:52.615793Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:52.622065Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:52.724249Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:16:52.724585Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:16:52.724691Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:52.793802Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=d222c1a133b94b2f9672d221a0bd2770; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:16:57.794415Z[0m [[31m[1merror    [0m] [1mTimeout: InferenceService http-onnx in test-kserve-raw-token-authentication external HTTPS infer not ready within 300s (token=no)[0m [[0m[1m[34mtests.model_serving.model_server.utils[0m][0m [36mname[0m=[35mtests.model_serving.model_server.utils[0m
[2m2026-07-09T21:16:57.796451Z[0m [[32m[1minfo     [0m] [1mResourceEdits: Restoring data for resource InferenceService http-onnx[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:57.796615Z[0m [[32m[1minfo     [0m] [1mUpdate InferenceService http-onnx:
{'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'true'}, 'name': 'http-onnx'}}[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
ERROR
_ ERROR at setup of TestKserveTokenAuthenticationRawForRest.test_disabled_raw_model_authentication[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] _
Traceback (most recent call last):
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call
    result: TResult | None = func()
                             ~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup
    item.session._setupstate.setup(item)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup
    col.setup()
    ~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup
    self._request._fillfixtures()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures
    item.funcargs[argname] = self.getfixturevalue(argname)
                             ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue
    fixturedef = self._get_active_fixturedef(argname)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef
    fixturedef.execute(request=subrequest)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1232, in execute
    result: FixtureValue = ihook.pytest_fixture_setup(
                           ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        fixturedef=self, request=request
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pytest_asyncio/plugin.py", line 924, in pytest_fixture_setup
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/setuponly.py", line 36, in pytest_fixture_setup
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1328, in pytest_fixture_setup
    result = call_fixture_func(fixturefunc, request, kwargs)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 998, in call_fixture_func
    fixture_result = next(generator)
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/kserve/authentication/conftest.py", line 100, in patched_remove_raw_authentication_isvc
    wait_for_raw_isvc_https_infer_ready(isvc=http_s3_ovms_raw_inference_service, token=None)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/utils.py", line 234, in wait_for_raw_isvc_https_infer_ready
    for ok in TimeoutSampler(wait_timeout=timeout, sleep=sleep, func=_https_infer_ok):
              ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/timeout_sampler/__init__.py", line 182, in __iter__
    raise TimeoutExpiredError(self._get_exception_log(exp=last_exp), last_exp=last_exp)
timeout_sampler.TimeoutExpiredError: Timed Out: 300
Function: tests.model_serving.model_server.utils._https_infer_ok
Last exception: N/A.
_ 2 of 5 completed, 0 Pass, 1 Fail, 0 Skip, 0 XPass, 0 XFail, 1 Error, 0 ReRun _

tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py::TestKserveTokenAuthenticationRawForRest::test_re_enabled_raw_model_authentication[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] [1m
TEST: TestKserveTokenAuthenticationRawForRest.test_disabled_raw_model_authentication[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] [setup] STATUS: [0;31mERROR[0m[0m
[2m2026-07-09T21:16:57.920363Z[0m [[31m[1merror    [0m] [1mTraceback (most recent call last):
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call
    result: TResult | None = func()
                             ~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup
    item.session._setupstate.setup(item)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup
    col.setup()
    ~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup
    self._request._fillfixtures()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures
    item.funcargs[argname] = self.getfixturevalue(argname)
                             ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue
    fixturedef = self._get_active_fixturedef(argname)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef
    fixturedef.execute(request=subrequest)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1232, in execute
    result: FixtureValue = ihook.pytest_fixture_setup(
                           ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        fixturedef=self, request=request
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pytest_asyncio/plugin.py", line 924, in pytest_fixture_setup
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/setuponly.py", line 36, in pytest_fixture_setup
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1328, in pytest_fixture_setup
    result = call_fixture_func(fixturefunc, request, kwargs)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 998, in call_fixture_func
    fixture_result = next(generator)
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/kserve/authentication/conftest.py", line 100, in patched_remove_raw_authentication_isvc
    wait_for_raw_isvc_https_infer_ready(isvc=http_s3_ovms_raw_inference_service, token=None)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/utils.py", line 234, in wait_for_raw_isvc_https_infer_ready
    for ok in TimeoutSampler(wait_timeout=timeout, sleep=sleep, func=_https_infer_ok):
              ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/timeout_sampler/__init__.py", line 182, in __iter__
    raise TimeoutExpiredError(self._get_exception_log(exp=last_exp), last_exp=last_exp)
timeout_sampler.TimeoutExpiredError: Timed Out: 300
Function: tests.model_serving.model_server.utils._https_infer_ok
Last exception: N/A.[0m [[0m[1m[34mconftest[0m][0m
[1m------------------------------------------------------- TEARDOWN -------------------------------------------------------[0m
[1m
----- test_re_enabled_raw_model_authentication[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] -----[0m
[1m-------------------------------------------------------- SETUP --------------------------------------------------------[0m
[1m--------------------------------------------------------- CALL ---------------------------------------------------------[0m
[2m2026-07-09T21:16:57.962110Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:58.024440Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:58.030473Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:16:58.116291Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:16:58.116451Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:16:58.116519Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:17:03.215904Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:17:03.220818Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:17:03.296211Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:17:03.296318Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:17:03.296388Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:17:03.401826Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
content-length: 53
content-type: application/json; charset=utf-8
date: Thu, 09 Jul 2026 21:17:03 GMT
set-cookie: 12d2eb568f824d2bcf01971f024ec5f9=19780c580d1d11fef6895f679142eddf; path=/; HttpOnly; Secure; SameSite=None

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
FAILED
_ TestKserveTokenAuthenticationRawForRest.test_re_enabled_raw_model_authentication[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] _
Traceback (most recent call last):
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call
    result: TResult | None = func()
                             ~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 184, in pytest_runtest_call
    item.runtest()
    ~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1707, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 167, in pytest_pyfunc_call
    result = testfunction(**testargs)
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py", line 61, in test_re_enabled_raw_model_authentication
    verify_inference_response(
    ~~~~~~~~~~~~~~~~~~~~~~~~~^
        inference_service=http_s3_ovms_raw_inference_service,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
        token=http_raw_inference_token,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/utils.py", line 154, in verify_inference_response
    assert re.search(expected_response_text, formatted_res), (  # type: ignore[arg-type]
           ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Expected: {"model_name":"http-onnx","model_version":"1","outputs":\[\{"name":"Plus214_Output_0","shape":\[1,10\],"datatype":"FP32","data":\[.*\]}]} not found in: {"error":"Modelwithrequestedversionisnotfound"}
_ 3 of 5 completed, 0 Pass, 2 Fail, 0 Skip, 0 XPass, 0 XFail, 1 Error, 0 ReRun _

tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py::TestKserveTokenAuthenticationRawForRest::test_cross_model_authentication_raw[http_s3_ovms_raw_inference_service_20-unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] [1m
TEST: TestKserveTokenAuthenticationRawForRest.test_re_enabled_raw_model_authentication[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] STATUS: [0;31mFAILED[0m[0m
[2m2026-07-09T21:17:03.410108Z[0m [[31m[1merror    [0m] [1mTraceback (most recent call last):
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call
    result: TResult | None = func()
                             ~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 184, in pytest_runtest_call
    item.runtest()
    ~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1707, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 167, in pytest_pyfunc_call
    result = testfunction(**testargs)
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py", line 61, in test_re_enabled_raw_model_authentication
    verify_inference_response(
    ~~~~~~~~~~~~~~~~~~~~~~~~~^
        inference_service=http_s3_ovms_raw_inference_service,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
        token=http_raw_inference_token,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/utils.py", line 154, in verify_inference_response
    assert re.search(expected_response_text, formatted_res), (  # type: ignore[arg-type]
           ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Expected: {"model_name":"http-onnx","model_version":"1","outputs":\[\{"name":"Plus214_Output_0","shape":\[1,10\],"datatype":"FP32","data":\[.*\]}]} not found in: {"error":"Modelwithrequestedversionisnotfound"}[0m [[0m[1m[34mconftest[0m][0m
[1m------------------------------------------------------- TEARDOWN -------------------------------------------------------[0m
[1m
 test_cross_model_authentication_raw[http_s3_ovms_raw_inference_service_20-unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] [0m
[1m-------------------------------------------------------- SETUP --------------------------------------------------------[0m
[2m2026-07-09T21:17:03.410716Z[0m [[32m[1minfo     [0m] [1mExecuting class fixture: model_service_account_2[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:17:03.410861Z[0m [[32m[1minfo     [0m] [1mCreate ServiceAccount models-bucket-sa-2[0m [[0m[1m[34mocp_resources ServiceAccount[0m][0m
[2m2026-07-09T21:17:03.411011Z[0m [[32m[1minfo     [0m] [1mPosting {'apiVersion': 'v1', 'kind': 'ServiceAccount', 'metadata': {'name': 'models-bucket-sa-2', 'namespace': 'test-kserve-raw-token-authentication'}, 'secrets': [{'name': 'models-bucket-secret'}]}[0m [[0m[1m[34mocp_resources ServiceAccount[0m][0m
[2m2026-07-09T21:17:03.450537Z[0m [[32m[1minfo     [0m] [1mExecuting class fixture: http_s3_ovms_raw_inference_service_2[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:17:03.477466Z[0m [[32m[1minfo     [0m] [1mkind: InferenceService api version: serving.kserve.io/v1beta1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:17:03.477630Z[0m [[32m[1minfo     [0m] [1mCreate InferenceService http-onnx-2[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:17:03.477881Z[0m [[32m[1minfo     [0m] [1mPosting {'apiVersion': 'serving.kserve.io/v1beta1', 'kind': 'InferenceService', 'metadata': {'name': 'http-onnx-2', 'labels': {'networking.kserve.io/visibility': 'exposed'}, 'annotations': {'serving.kserve.io/deploymentMode': 'RawDeployment', 'security.opendatahub.io/enable-auth': 'true'}, 'namespace': 'test-kserve-raw-token-authentication'}, 'spec': {'predictor': {'model': {'modelFormat': {'name': 'openvino_ir'}, 'version': None, 'runtime': 'http-mnist-runtime', 'storage': {'key': 'ci-bucket-secret', 'path': '/test-dir/'}}, 'serviceAccountName': 'models-bucket-sa-2'}}}[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:17:03.573788Z[0m [[32m[1minfo     [0m] [1mWaiting for pods to be created[0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:17:04.612627Z[0m [[32m[1minfo     [0m] [1mWaiting for pods to be created[0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:17:04.687184Z[0m [[32m[1minfo     [0m] [1mVerifying no failed pods      [0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:17:25.020729Z[0m [[32m[1minfo     [0m] [1mWaiting for inference deployment replicas to complete[0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:17:25.097591Z[0m [[32m[1minfo     [0m] [1mWait for Deployment http-onnx-2-predictor to be deployed: True[0m [[0m[1m[34mocp_resources Deployment[0m][0m
[2m2026-07-09T21:17:25.136730Z[0m [[32m[1minfo     [0m] [1mWait for InferenceService/http-onnx-2's 'Ready' condition to be 'True'[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:17:25.136923Z[0m [[32m[1minfo     [0m] [1mWait until InferenceService http-onnx-2 is created[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[1m--------------------------------------------------------- CALL ---------------------------------------------------------[0m
[2m2026-07-09T21:17:25.273790Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:17:25.331341Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:17:25.338124Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:17:25.436654Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:17:25.436794Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:17:25.436847Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
PASSED[2m2026-07-09T21:17:25.680257Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 403 Forbidden
content-type: text/plain; charset=utf-8
x-content-type-options: nosniff
date: Thu, 09 Jul 2026 21:17:25 GMT
content-length: 145
set-cookie: 69f764f782e5acf3035b89de67f2b2b6=f3f64a57cbd3d1bc0c0feb456fc039b5; path=/; HttpOnly; Secure; SameSite=None

Forbidden (user=system:serviceaccount:test-kserve-raw-token-authentication:models-bucket-sa, verb=get, resource=inferenceservices, subresource=)
[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[1m
TEST: TestKserveTokenAuthenticationRawForRest.test_cross_model_authentication_raw[http_s3_ovms_raw_inference_service_20-unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] STATUS: [0;32mPASSED[0m[0m
[1m------------------------------------------------------- TEARDOWN -------------------------------------------------------[0m
[2m2026-07-09T21:17:25.683067Z[0m [[32m[1minfo     [0m] [1mDelete InferenceService http-onnx-2[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:17:25.725157Z[0m [[32m[1minfo     [0m] [1mDeleting {'apiVersion': 'serving.kserve.io/v1beta1', 'kind': 'InferenceService', 'metadata': {'annotations': {'modelFormat': 'openvino_ir', 'security.opendatahub.io/enable-auth': 'true', 'serving.kserve.io/deploymentMode': 'Standard'}, 'creationTimestamp': '2026-07-09T21:17:03Z', 'finalizers': ['inferenceservice.finalizers', 'odh.inferenceservice.finalizers'], 'generation': 1, 'labels': {'networking.kserve.io/visibility': 'exposed'}, 'managedFields': [{'apiVersion': 'serving.kserve.io/v1beta1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'.': {}, 'f:security.opendatahub.io/enable-auth': {}, 'f:serving.kserve.io/deploymentMode': {}}, 'f:labels': {'.': {}, 'f:networking.kserve.io/visibility': {}}}, 'f:spec': {'.': {}, 'f:predictor': {'.': {}, 'f:model': {'.': {}, 'f:modelFormat': {'.': {}, 'f:name': {}}, 'f:runtime': {}, 'f:storage': {'.': {}, 'f:key': {}, 'f:path': {}}}, 'f:serviceAccountName': {}}}}, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': '2026-07-09T21:17:03Z'}, {'apiVersion': 'serving.kserve.io/v1beta1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:finalizers': {'.': {}, 'v:"inferenceservice.finalizers"': {}, 'v:"odh.inferenceservice.finalizers"': {}}}}, 'manager': 'manager', 'operation': 'Update', 'time': '2026-07-09T21:17:03Z'}, {'apiVersion': 'serving.kserve.io/v1beta1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:status': {'.': {}, 'f:address': {'.': {}, 'f:url': {}}, 'f:components': {'.': {}, 'f:predictor': {'.': {}, 'f:url': {}}}, 'f:conditions': {}, 'f:deploymentMode': {}, 'f:modelStatus': {'.': {}, 'f:copies': {'.': {}, 'f:failedCopies': {}, 'f:totalCopies': {}}, 'f:states': {'.': {}, 'f:activeModelState': {}, 'f:targetModelState': {}}, 'f:transitionStatus': {}}, 'f:observedGeneration': {}, 'f:servingRuntimeName': {}, 'f:url': {}}}, 'manager': 'manager', 'operation': 'Update', 'subresource': 'status', 'time': '2026-07-09T21:17:16Z'}], 'name': 'http-onnx-2', 'namespace': 'test-kserve-raw-token-authentication', 'resourceVersion': '208252804', 'uid': '34182fee-010e-4629-a10c-9294f442b6c4'}, 'spec': {'predictor': {'automountServiceAccountToken': False, 'model': {'modelFormat': {'name': 'openvino_ir'}, 'name': '', 'resources': {}, 'runtime': 'http-mnist-runtime', 'storage': {'key': 'ci-bucket-secret', 'path': '/test-dir/'}}, 'serviceAccountName': 'models-bucket-sa-2'}}, 'status': {'address': {'url': 'https://http-onnx-2-predictor.test-kserve-raw-token-authentication.svc.cluster.local:8443'}, 'components': {'predictor': {'url': 'http://http-onnx-2-predictor-test-kserve-raw-token-authentication.example.com'}}, 'conditions': [{'lastTransitionTime': '2026-07-09T21:17:03Z', 'status': 'True', 'type': 'IngressReady'}, {'lastTransitionTime': '2026-07-09T21:17:16Z', 'message': 'ReplicaSet "http-onnx-2-predictor-774cb48ff" has successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'PredictorReady'}, {'lastTransitionTime': '2026-07-09T21:17:16Z', 'status': 'True', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-09T21:17:03Z', 'severity': 'Info', 'status': 'False', 'type': 'Stopped'}], 'deploymentMode': 'Standard', 'modelStatus': {'copies': {'failedCopies': 0, 'totalCopies': 1}, 'states': {'activeModelState': 'Loaded', 'targetModelState': 'Loaded'}, 'transitionStatus': 'UpToDate'}, 'observedGeneration': 1, 'servingRuntimeName': 'http-mnist-runtime', 'url': 'https://http-onnx-2-test-kserve-raw-token-authentication.apps.rosa.rh-nightly-pm.nuth.p3.openshiftapps.com'}}[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:17:25.750268Z[0m [[32m[1minfo     [0m] [1mWait until InferenceService http-onnx-2 is deleted[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:17:26.790199Z[0m [[32m[1minfo     [0m] [1mDelete ServiceAccount models-bucket-sa-2[0m [[0m[1m[34mocp_resources ServiceAccount[0m][0m
[2m2026-07-09T21:17:26.828591Z[0m [[32m[1minfo     [0m] [1mDeleting {'kind': 'ServiceAccount', 'apiVersion': 'v1', 'metadata': {'name': 'models-bucket-sa-2', 'namespace': 'test-kserve-raw-token-authentication', 'uid': 'f96a4293-c3a7-46ba-8f9f-50e8b39dadb8', 'resourceVersion': '208252502', 'creationTimestamp': '2026-07-09T21:17:03Z', 'annotations': {'openshift.io/internal-registry-pull-secret-ref': 'models-bucket-sa-2-dockercfg-b5bkr'}, 'managedFields': [{'manager': 'openshift.io/image-registry-pull-secrets_service-account-controller', 'operation': 'Apply', 'apiVersion': 'v1', 'time': '2026-07-09T21:17:03Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:imagePullSecrets': {}, 'f:metadata': {'f:annotations': {'f:openshift.io/internal-registry-pull-secret-ref': {}}}, 'f:secrets': {'k:{"name":"models-bucket-sa-2-dockercfg-b5bkr"}': {}}}}, {'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2026-07-09T21:17:03Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:secrets': {'.': {}, 'k:{"name":"models-bucket-secret"}': {}}}}]}, 'secrets': [{'name': 'models-bucket-secret'}, {'name': 'models-bucket-sa-2-dockercfg-b5bkr'}], 'imagePullSecrets': [{'name': 'models-bucket-sa-2-dockercfg-b5bkr'}]}[0m [[0m[1m[34mocp_resources ServiceAccount[0m][0m
[2m2026-07-09T21:17:26.864187Z[0m [[32m[1minfo     [0m] [1mWait until ServiceAccount models-bucket-sa-2 is deleted[0m [[0m[1m[34mocp_resources ServiceAccount[0m][0m
[2m2026-07-09T21:17:26.883848Z[0m [[32m[1minfo     [0m] [1mDelete RoleBinding http-models-bucket-sa-view[0m [[0m[1m[34mocp_resources RoleBinding[0m][0m
[2m2026-07-09T21:17:26.921902Z[0m [[32m[1minfo     [0m] [1mDeleting {'kind': 'RoleBinding', 'apiVersion': 'rbac.authorization.k8s.io/v1', 'metadata': {'name': 'http-models-bucket-sa-view', 'namespace': 'test-kserve-raw-token-authentication', 'uid': '73d458f4-0945-4ffd-8cd3-f55c9c27ae51', 'resourceVersion': '208247619', 'creationTimestamp': '2026-07-09T21:11:54Z', 'managedFields': [{'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'rbac.authorization.k8s.io/v1', 'time': '2026-07-09T21:11:54Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:roleRef': {}, 'f:subjects': {}}}]}, 'subjects': [{'kind': 'ServiceAccount', 'name': 'models-bucket-sa'}], 'roleRef': {'apiGroup': 'rbac.authorization.k8s.io', 'kind': 'Role', 'name': 'http-onnx-view'}}[0m [[0m[1m[34mocp_resources RoleBinding[0m][0m
[2m2026-07-09T21:17:26.945497Z[0m [[32m[1minfo     [0m] [1mWait until RoleBinding http-models-bucket-sa-view is deleted[0m [[0m[1m[34mocp_resources RoleBinding[0m][0m
[2m2026-07-09T21:17:26.965077Z[0m [[32m[1minfo     [0m] [1mDelete Role http-onnx-view    [0m [[0m[1m[34mocp_resources Role[0m][0m
[2m2026-07-09T21:17:27.005076Z[0m [[32m[1minfo     [0m] [1mDeleting {'kind': 'Role', 'apiVersion': 'rbac.authorization.k8s.io/v1', 'metadata': {'name': 'http-onnx-view', 'namespace': 'test-kserve-raw-token-authentication', 'uid': '34631a69-8cd1-4063-8a53-e9d30485ef64', 'resourceVersion': '208247618', 'creationTimestamp': '2026-07-09T21:11:54Z', 'managedFields': [{'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'rbac.authorization.k8s.io/v1', 'time': '2026-07-09T21:11:54Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:rules': {}}}]}, 'rules': [{'verbs': ['get'], 'apiGroups': ['serving.kserve.io'], 'resources': ['inferenceservices'], 'resourceNames': ['http-onnx']}]}[0m [[0m[1m[34mocp_resources Role[0m][0m
[2m2026-07-09T21:17:27.026234Z[0m [[32m[1minfo     [0m] [1mWait until Role http-onnx-view is deleted[0m [[0m[1m[34mocp_resources Role[0m][0m
[2m2026-07-09T21:17:27.046080Z[0m [[32m[1minfo     [0m] [1mDelete InferenceService http-onnx[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:17:27.088683Z[0m [[32m[1minfo     [0m] [1mDeleting {'apiVersion': 'serving.kserve.io/v1beta1', 'kind': 'InferenceService', 'metadata': {'annotations': {'modelFormat': 'openvino_ir', 'security.opendatahub.io/enable-auth': 'true', 'serving.kserve.io/deploymentMode': 'Standard'}, 'creationTimestamp': '2026-07-09T21:11:43Z', 'finalizers': ['inferenceservice.finalizers', 'odh.inferenceservice.finalizers'], 'generation': 1, 'labels': {'networking.kserve.io/visibility': 'exposed'}, 'managedFields': [{'apiVersion': 'serving.kserve.io/v1beta1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:finalizers': {'.': {}, 'v:"inferenceservice.finalizers"': {}, 'v:"odh.inferenceservice.finalizers"': {}}}}, 'manager': 'manager', 'operation': 'Update', 'time': '2026-07-09T21:11:43Z'}, {'apiVersion': 'serving.kserve.io/v1beta1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'.': {}, 'f:security.opendatahub.io/enable-auth': {}, 'f:serving.kserve.io/deploymentMode': {}}, 'f:labels': {'.': {}, 'f:networking.kserve.io/visibility': {}}}, 'f:spec': {'.': {}, 'f:predictor': {'.': {}, 'f:model': {'.': {}, 'f:modelFormat': {'.': {}, 'f:name': {}}, 'f:runtime': {}, 'f:storage': {'.': {}, 'f:key': {}, 'f:path': {}}}, 'f:serviceAccountName': {}}}}, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': '2026-07-09T21:16:57Z'}, {'apiVersion': 'serving.kserve.io/v1beta1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:status': {'.': {}, 'f:address': {'.': {}, 'f:url': {}}, 'f:components': {'.': {}, 'f:predictor': {'.': {}, 'f:url': {}}}, 'f:conditions': {}, 'f:deploymentMode': {}, 'f:modelStatus': {'.': {}, 'f:copies': {'.': {}, 'f:failedCopies': {}, 'f:totalCopies': {}}, 'f:states': {'.': {}, 'f:activeModelState': {}, 'f:targetModelState': {}}, 'f:transitionStatus': {}}, 'f:observedGeneration': {}, 'f:servingRuntimeName': {}, 'f:url': {}}}, 'manager': 'manager', 'operation': 'Update', 'subresource': 'status', 'time': '2026-07-09T21:17:09Z'}], 'name': 'http-onnx', 'namespace': 'test-kserve-raw-token-authentication', 'resourceVersion': '208252681', 'uid': 'd86af884-7b9d-4778-84de-f3451209470a'}, 'spec': {'predictor': {'automountServiceAccountToken': False, 'model': {'modelFormat': {'name': 'openvino_ir'}, 'name': '', 'resources': {}, 'runtime': 'http-mnist-runtime', 'storage': {'key': 'ci-bucket-secret', 'path': '/test-dir/'}}, 'serviceAccountName': 'models-bucket-sa'}}, 'status': {'address': {'url': 'https://http-onnx-predictor.test-kserve-raw-token-authentication.svc.cluster.local:8443'}, 'components': {'predictor': {'url': 'http://http-onnx-predictor-test-kserve-raw-token-authentication.example.com'}}, 'conditions': [{'lastTransitionTime': '2026-07-09T21:11:43Z', 'status': 'True', 'type': 'IngressReady'}, {'lastTransitionTime': '2026-07-09T21:17:09Z', 'message': 'ReplicaSet "http-onnx-predictor-74ff48987f" has successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'PredictorReady'}, {'lastTransitionTime': '2026-07-09T21:17:09Z', 'status': 'True', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-09T21:11:43Z', 'severity': 'Info', 'status': 'False', 'type': 'Stopped'}], 'deploymentMode': 'Standard', 'modelStatus': {'copies': {'failedCopies': 0, 'totalCopies': 2}, 'states': {'activeModelState': 'Loaded', 'targetModelState': 'Loaded'}, 'transitionStatus': 'UpToDate'}, 'observedGeneration': 3, 'servingRuntimeName': 'http-mnist-runtime', 'url': 'https://http-onnx-test-kserve-raw-token-authentication.apps.rosa.rh-nightly-pm.nuth.p3.openshiftapps.com'}}[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:17:27.112918Z[0m [[32m[1minfo     [0m] [1mWait until InferenceService http-onnx is deleted[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:17:28.153307Z[0m [[32m[1minfo     [0m] [1mDelete ServiceAccount models-bucket-sa[0m [[0m[1m[34mocp_resources ServiceAccount[0m][0m
[2m2026-07-09T21:17:28.191977Z[0m [[32m[1minfo     [0m] [1mDeleting {'kind': 'ServiceAccount', 'apiVersion': 'v1', 'metadata': {'name': 'models-bucket-sa', 'namespace': 'test-kserve-raw-token-authentication', 'uid': 'fdf7e89f-bbcb-4719-9407-4c9f40a10502', 'resourceVersion': '208247337', 'creationTimestamp': '2026-07-09T21:11:42Z', 'annotations': {'openshift.io/internal-registry-pull-secret-ref': 'models-bucket-sa-dockercfg-njlb7'}, 'managedFields': [{'manager': 'openshift.io/image-registry-pull-secrets_service-account-controller', 'operation': 'Apply', 'apiVersion': 'v1', 'time': '2026-07-09T21:11:42Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:imagePullSecrets': {}, 'f:metadata': {'f:annotations': {'f:openshift.io/internal-registry-pull-secret-ref': {}}}, 'f:secrets': {'k:{"name":"models-bucket-sa-dockercfg-njlb7"}': {}}}}, {'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2026-07-09T21:11:42Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:secrets': {'.': {}, 'k:{"name":"models-bucket-secret"}': {}}}}]}, 'secrets': [{'name': 'models-bucket-secret'}, {'name': 'models-bucket-sa-dockercfg-njlb7'}], 'imagePullSecrets': [{'name': 'models-bucket-sa-dockercfg-njlb7'}]}[0m [[0m[1m[34mocp_resources ServiceAccount[0m][0m
[2m2026-07-09T21:17:28.218146Z[0m [[32m[1minfo     [0m] [1mWait until ServiceAccount models-bucket-sa is deleted[0m [[0m[1m[34mocp_resources ServiceAccount[0m][0m
[2m2026-07-09T21:17:28.238489Z[0m [[32m[1minfo     [0m] [1mDelete Secret models-bucket-secret[0m [[0m[1m[34mocp_resources Secret[0m][0m
[2m2026-07-09T21:17:28.285593Z[0m [[32m[1minfo     [0m] [1mDeleting {'kind': 'Secret', 'apiVersion': 'v1', 'metadata': {'name': 'models-bucket-secret', 'namespace': 'test-kserve-raw-token-authentication', 'uid': 'f6aa14a7-e8c4-4297-b52b-9312d3ce9ef2', 'resourceVersion': '208247330', 'creationTimestamp': '2026-07-09T21:11:42Z', 'labels': {'opendatahub.io/dashboard': 'true', 'opendatahub.io/managed': 'true'}, 'annotations': {'opendatahub.io/connection-type': 's3', 'serving.kserve.io/s3-endpoint': 's3.amazonaws.com/', 'serving.kserve.io/s3-region': 'us-east-1', 'serving.kserve.io/s3-useanoncredential': 'false', 'serving.kserve.io/s3-usehttps': '1', 'serving.kserve.io/s3-verifyssl': '0'}, 'managedFields': [{'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2026-07-09T21:11:42Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:data': {'.': {}, 'f:AWS_ACCESS_KEY_ID': {}, 'f:AWS_DEFAULT_REGION': {}, 'f:AWS_S3_BUCKET': {}, 'f:AWS_S3_ENDPOINT': {}, 'f:AWS_SECRET_ACCESS_KEY': {}}, 'f:metadata': {'f:annotations': {'.': {}, 'f:opendatahub.io/connection-type': {}, 'f:serving.kserve.io/s3-endpoint': {}, 'f:serving.kserve.io/s3-region': {}, 'f:serving.kserve.io/s3-useanoncredential': {}, 'f:serving.kserve.io/s3-usehttps': {}, 'f:serving.kserve.io/s3-verifyssl': {}}, 'f:labels': {'.': {}, 'f:opendatahub.io/dashboard': {}, 'f:opendatahub.io/managed': {}}}, 'f:type': {}}}]}, 'data': '*******', 'type': 'Opaque'}[0m [[0m[1m[34mocp_resources Secret[0m][0m
[2m2026-07-09T21:17:28.311778Z[0m [[32m[1minfo     [0m] [1mWait until Secret models-bucket-secret is deleted[0m [[0m[1m[34mocp_resources Secret[0m][0m
[2m2026-07-09T21:17:28.332850Z[0m [[32m[1minfo     [0m] [1mDelete Secret ci-bucket-secret[0m [[0m[1m[34mocp_resources Secret[0m][0m
[2m2026-07-09T21:17:28.378494Z[0m [[32m[1minfo     [0m] [1mDeleting {'kind': 'Secret', 'apiVersion': 'v1', 'metadata': {'name': 'ci-bucket-secret', 'namespace': 'test-kserve-raw-token-authentication', 'uid': '13bf4b76-2db4-4c42-8528-3c3e27d3aa41', 'resourceVersion': '208247327', 'creationTimestamp': '2026-07-09T21:11:42Z', 'labels': {'opendatahub.io/dashboard': 'true', 'opendatahub.io/managed': 'true'}, 'annotations': {'opendatahub.io/connection-type': 's3', 'serving.kserve.io/s3-endpoint': 's3.amazonaws.com/', 'serving.kserve.io/s3-region': 'us-east-1', 'serving.kserve.io/s3-useanoncredential': 'false', 'serving.kserve.io/s3-usehttps': '1', 'serving.kserve.io/s3-verifyssl': '0'}, 'managedFields': [{'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2026-07-09T21:11:42Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:data': {'.': {}, 'f:AWS_ACCESS_KEY_ID': {}, 'f:AWS_DEFAULT_REGION': {}, 'f:AWS_S3_BUCKET': {}, 'f:AWS_S3_ENDPOINT': {}, 'f:AWS_SECRET_ACCESS_KEY': {}}, 'f:metadata': {'f:annotations': {'.': {}, 'f:opendatahub.io/connection-type': {}, 'f:serving.kserve.io/s3-endpoint': {}, 'f:serving.kserve.io/s3-region': {}, 'f:serving.kserve.io/s3-useanoncredential': {}, 'f:serving.kserve.io/s3-usehttps': {}, 'f:serving.kserve.io/s3-verifyssl': {}}, 'f:labels': {'.': {}, 'f:opendatahub.io/dashboard': {}, 'f:opendatahub.io/managed': {}}}, 'f:type': {}}}]}, 'data': '*******', 'type': 'Opaque'}[0m [[0m[1m[34mocp_resources Secret[0m][0m
[2m2026-07-09T21:17:28.405079Z[0m [[32m[1minfo     [0m] [1mWait until Secret ci-bucket-secret is deleted[0m [[0m[1m[34mocp_resources Secret[0m][0m
[2m2026-07-09T21:17:28.424400Z[0m [[32m[1minfo     [0m] [1mDelete ServingRuntime http-mnist-runtime[0m [[0m[1m[34mocp_resources ServingRuntime[0m][0m
[2m2026-07-09T21:17:28.464265Z[0m [[32m[1minfo     [0m] [1mDeleting {'apiVersion': 'serving.kserve.io/v1alpha1', 'kind': 'ServingRuntime', 'metadata': {'annotations': {'opendatahub.io/recommended-accelerators': '["nvidia.com/gpu"]', 'opendatahub.io/runtime-version': 'v2026.1.0', 'openshift.io/display-name': 'OpenVINO Model Server'}, 'creationTimestamp': '2026-07-09T21:11:42Z', 'generation': 1, 'labels': {'opendatahub.io/dashboard': 'true'}, 'managedFields': [{'apiVersion': 'serving.kserve.io/v1alpha1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'.': {}, 'f:opendatahub.io/recommended-accelerators': {}, 'f:opendatahub.io/runtime-version': {}, 'f:openshift.io/display-name': {}}, 'f:labels': {'.': {}, 'f:opendatahub.io/dashboard': {}}}, 'f:spec': {'.': {}, 'f:annotations': {'.': {}, 'f:opendatahub.io/kserve-runtime': {}, 'f:prometheus.io/path': {}, 'f:prometheus.io/port': {}}, 'f:containers': {}, 'f:multiModel': {}, 'f:protocolVersions': {}, 'f:supportedModelFormats': {}}}, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': '2026-07-09T21:11:42Z'}], 'name': 'http-mnist-runtime', 'namespace': 'test-kserve-raw-token-authentication', 'resourceVersion': '208247323', 'uid': '769b9ba9-e430-4b0a-b277-460c597e454b'}, 'spec': {'annotations': {'opendatahub.io/kserve-runtime': 'ovms', 'prometheus.io/path': '/metrics', 'prometheus.io/port': '8888'}, 'containers': [{'args': ['--model_name={{.Name}}', '--port=8001', '--rest_port=8888', '--model_path=/mnt/models', '--file_system_poll_wait_seconds=0', '--metrics_enable'], 'image': 'registry.redhat.io/rhoai/odh-openvino-model-server-rhel9@sha256:8c1b225509bfdc43756195f34a356d1e2ebd58fcd83fda3d37380dd7f5304df8', 'name': 'kserve-container', 'ports': [{'containerPort': 8888, 'protocol': 'TCP'}]}], 'multiModel': False, 'protocolVersions': ['v2', 'grpc-v2'], 'supportedModelFormats': [{'autoSelect': True, 'name': 'openvino_ir', 'version': 'opset13'}, {'name': 'onnx', 'version': '1'}, {'autoSelect': True, 'name': 'tensorflow', 'version': '1'}, {'autoSelect': True, 'name': 'tensorflow', 'version': '2'}, {'autoSelect': True, 'name': 'paddle', 'version': '2'}, {'autoSelect': True, 'name': 'pytorch', 'version': '2'}]}}[0m [[0m[1m[34mocp_resources ServingRuntime[0m][0m
[2m2026-07-09T21:17:28.486457Z[0m [[32m[1minfo     [0m] [1mWait until ServingRuntime http-mnist-runtime is deleted[0m [[0m[1m[34mocp_resources ServingRuntime[0m][0m
[2m2026-07-09T21:17:28.524978Z[0m [[32m[1minfo     [0m] [1mDelete Project test-kserve-raw-token-authentication[0m [[0m[1m[34mocp_resources Project[0m][0m
[2m2026-07-09T21:17:28.571652Z[0m [[32m[1minfo     [0m] [1mDeleting {'kind': 'Project', 'apiVersion': 'project.openshift.io/v1', 'metadata': {'name': 'test-kserve-raw-token-authentication', 'uid': '529092ed-bc2c-41b5-a98c-89a5ae1512b4', 'resourceVersion': '208247294', 'creationTimestamp': '2026-07-09T21:11:42Z', 'labels': {'kubernetes.io/metadata.name': 'test-kserve-raw-token-authentication', 'pod-security.kubernetes.io/audit': 'restricted', 'pod-security.kubernetes.io/audit-version': 'latest', 'pod-security.kubernetes.io/warn': 'restricted', 'pod-security.kubernetes.io/warn-version': 'latest'}, 'annotations': {'openshift.io/description': '', 'openshift.io/display-name': '', 'openshift.io/requester': 'system:serviceaccount:kube-system:olminstall-cluster-admin', 'openshift.io/sa.scc.mcs': 's0:c31,c5', 'openshift.io/sa.scc.supplemental-groups': '1000940000/10000', 'openshift.io/sa.scc.uid-range': '1000940000/10000', 'security.openshift.io/MinimallySufficientPodSecurityStandard': 'restricted'}, 'managedFields': [{'manager': 'pod-security-admission-label-synchronization-controller', 'operation': 'Apply', 'apiVersion': 'v1', 'time': '2026-07-09T21:11:42Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'f:security.openshift.io/MinimallySufficientPodSecurityStandard': {}}, 'f:labels': {'f:pod-security.kubernetes.io/audit': {}, 'f:pod-security.kubernetes.io/audit-version': {}, 'f:pod-security.kubernetes.io/warn': {}, 'f:pod-security.kubernetes.io/warn-version': {}}}}}, {'manager': 'cluster-policy-controller', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2026-07-09T21:11:42Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'f:openshift.io/sa.scc.mcs': {}, 'f:openshift.io/sa.scc.supplemental-groups': {}, 'f:openshift.io/sa.scc.uid-range': {}}}}}, {'manager': 'openshift-apiserver', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2026-07-09T21:11:42Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'.': {}, 'f:openshift.io/description': {}, 'f:openshift.io/display-name': {}, 'f:openshift.io/requester': {}}, 'f:labels': {'.': {}, 'f:kubernetes.io/metadata.name': {}}}}}, {'manager': 'openshift-controller-manager', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2026-07-09T21:11:42Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:spec': {'f:finalizers': {}}}, 'subresource': 'finalize'}]}, 'spec': {'finalizers': ['kubernetes']}, 'status': {'phase': 'Active'}}[0m [[0m[1m[34mocp_resources Project[0m][0m
[2m2026-07-09T21:17:28.602330Z[0m [[32m[1minfo     [0m] [1mWait until Project test-kserve-raw-token-authentication is deleted[0m [[0m[1m[34mocp_resources Project[0m][0m

_ 4 of 5 completed, 1 Pass, 2 Fail, 0 Skip, 0 XPass, 0 XFail, 1 Error, 0 ReRun _

tests/model_serving/model_server/kserve/authentication/test_non_admin_users.py::TestRawUnprivilegedUser::test_non_admin_deploy_raw_and_query_model[unprivileged_model_namespace0-unprivileged_s3_ovms_raw_inference_service0] [1m
- test_non_admin_deploy_raw_and_query_model[unprivileged_model_namespace0-unprivileged_s3_ovms_raw_inference_service0] -[0m
[1m-------------------------------------------------------- SETUP --------------------------------------------------------[0m
[2m2026-07-09T21:17:38.847889Z[0m [[32m[1minfo     [0m] [1mExecuting class fixture: unprivileged_model_namespace[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:17:38.848040Z[0m [[32m[1minfo     [0m] [1mkind: ProjectRequest api version: project.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:17:38.848169Z[0m [[32m[1minfo     [0m] [1mCreate ProjectRequest test-non-admin-raw[0m [[0m[1m[34mocp_resources ProjectRequest[0m][0m
[2m2026-07-09T21:17:38.848301Z[0m [[32m[1minfo     [0m] [1mPosting {'apiVersion': 'project.openshift.io/v1', 'kind': 'ProjectRequest', 'metadata': {'name': 'test-non-admin-raw'}}[0m [[0m[1m[34mocp_resources ProjectRequest[0m][0m
[2m2026-07-09T21:17:38.953760Z[0m [[32m[1minfo     [0m] [1mkind: Project api version: project.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:17:38.953906Z[0m [[32m[1minfo     [0m] [1mWait for Project test-non-admin-raw status to be Active[0m [[0m[1m[34mocp_resources Project[0m][0m
[2m2026-07-09T21:17:38.973999Z[0m [[32m[1minfo     [0m] [1mStatus of Project test-non-admin-raw is Active[0m [[0m[1m[34mocp_resources Project[0m][0m
[2m2026-07-09T21:17:38.974159Z[0m [[32m[1minfo     [0m] [1mExecuting class fixture: http_s3_ovms_serving_runtime[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:17:38.974296Z[0m [[32m[1minfo     [0m] [1mkind: Template api version: template.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:17:39.020430Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:17:39.020593Z[0m [[32m[1minfo     [0m] [1mCreate ServingRuntime http-mnist-runtime[0m [[0m[1m[34mocp_resources ServingRuntime[0m][0m
[2m2026-07-09T21:17:39.020813Z[0m [[32m[1minfo     [0m] [1mPosting {'apiVersion': 'serving.kserve.io/v1alpha1', 'kind': 'ServingRuntime', 'metadata': {'annotations': {'opendatahub.io/recommended-accelerators': '["nvidia.com/gpu"]', 'opendatahub.io/runtime-version': 'v2026.1.0', 'openshift.io/display-name': 'OpenVINO Model Server'}, 'labels': {'opendatahub.io/dashboard': 'true'}, 'name': 'http-mnist-runtime', 'namespace': 'test-non-admin-raw'}, 'spec': {'annotations': {'opendatahub.io/kserve-runtime': 'ovms', 'prometheus.io/path': '/metrics', 'prometheus.io/port': '8888'}, 'containers': [{'args': ['--model_name={{.Name}}', '--port=8001', '--rest_port=8888', '--model_path=/mnt/models', '--file_system_poll_wait_seconds=0', '--metrics_enable'], 'image': 'registry.redhat.io/rhoai/odh-openvino-model-server-rhel9@sha256:8c1b225509bfdc43756195f34a356d1e2ebd58fcd83fda3d37380dd7f5304df8', 'name': 'kserve-container', 'ports': [{'containerPort': 8888, 'protocol': 'TCP'}]}], 'multiModel': False, 'protocolVersions': ['v2', 'grpc-v2'], 'supportedModelFormats': [{'autoSelect': True, 'name': 'openvino_ir', 'version': 'opset13'}, {'name': 'onnx', 'version': '1'}, {'autoSelect': True, 'name': 'tensorflow', 'version': '1'}, {'autoSelect': True, 'name': 'tensorflow', 'version': '2'}, {'autoSelect': True, 'name': 'paddle', 'version': '2'}, {'autoSelect': True, 'name': 'pytorch', 'version': '2'}]}}[0m [[0m[1m[34mocp_resources ServingRuntime[0m][0m
[2m2026-07-09T21:17:39.081928Z[0m [[32m[1minfo     [0m] [1mExecuting class fixture: unprivileged_ci_endpoint_s3_secret[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:17:39.103462Z[0m [[32m[1minfo     [0m] [1mCreate Secret ci-bucket-unprivileged[0m [[0m[1m[34mocp_resources Secret[0m][0m
[2m2026-07-09T21:17:39.103613Z[0m [[32m[1minfo     [0m] [1mPosting {'apiVersion': 'v1', 'kind': 'Secret', 'metadata': {'name': 'ci-bucket-unprivileged', 'labels': {'opendatahub.io/managed': 'true', 'opendatahub.io/dashboard': 'true'}, 'annotations': {'opendatahub.io/connection-type': 's3', 'serving.kserve.io/s3-endpoint': 's3.amazonaws.com/', 'serving.kserve.io/s3-region': 'us-east-1', 'serving.kserve.io/s3-useanoncredential': 'false', 'serving.kserve.io/s3-verifyssl': '0', 'serving.kserve.io/s3-usehttps': '1'}, 'namespace': 'test-non-admin-raw'}, 'data': '*******'}[0m [[0m[1m[34mocp_resources Secret[0m][0m
[2m2026-07-09T21:17:39.149950Z[0m [[32m[1minfo     [0m] [1mWait until Secret ci-bucket-unprivileged is created[0m [[0m[1m[34mocp_resources Secret[0m][0m
[2m2026-07-09T21:17:39.170688Z[0m [[32m[1minfo     [0m] [1mExecuting class fixture: unprivileged_s3_ovms_raw_inference_service[0m [[0m[1m[34mconftest[0m][0m
[2m2026-07-09T21:17:39.194674Z[0m [[32m[1minfo     [0m] [1mkind: InferenceService api version: serving.kserve.io/v1beta1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:17:39.194891Z[0m [[32m[1minfo     [0m] [1mCreate InferenceService http-onnx-raw[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:17:39.195060Z[0m [[32m[1minfo     [0m] [1mPosting {'apiVersion': 'serving.kserve.io/v1beta1', 'kind': 'InferenceService', 'metadata': {'name': 'http-onnx-raw', 'annotations': {'serving.kserve.io/deploymentMode': 'RawDeployment'}, 'namespace': 'test-non-admin-raw'}, 'spec': {'predictor': {'model': {'modelFormat': {'name': 'openvino_ir'}, 'version': None, 'runtime': 'http-mnist-runtime', 'storage': {'key': 'ci-bucket-unprivileged', 'path': '/test-dir/'}}}}}[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:17:39.263200Z[0m [[32m[1minfo     [0m] [1mWaiting for pods to be created[0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:17:40.304778Z[0m [[32m[1minfo     [0m] [1mWaiting for pods to be created[0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:17:40.388384Z[0m [[32m[1minfo     [0m] [1mVerifying no failed pods      [0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:18:00.722813Z[0m [[32m[1minfo     [0m] [1mWaiting for inference deployment replicas to complete[0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:18:00.803893Z[0m [[32m[1minfo     [0m] [1mWait for Deployment http-onnx-raw-predictor to be deployed: True[0m [[0m[1m[34mocp_resources Deployment[0m][0m
[2m2026-07-09T21:18:00.842595Z[0m [[32m[1minfo     [0m] [1mWait for InferenceService/http-onnx-raw's 'Ready' condition to be 'True'[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:18:00.842817Z[0m [[32m[1minfo     [0m] [1mWait until InferenceService http-onnx-raw is created[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[1m--------------------------------------------------------- CALL ---------------------------------------------------------[0m
[2m2026-07-09T21:18:00.979787Z[0m [[32m[1minfo     [0m] [1mkind: ServingRuntime api version: serving.kserve.io/v1alpha1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:18:01.019923Z[0m [[32m[1minfo     [0m] [1mTrying to get client via new_client_from_config[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:18:01.025935Z[0m [[32m[1minfo     [0m] [1mkind: Infrastructure api version: config.openshift.io/v1[0m [[0m[1m[34mocp_resources.resource[0m][0m
[2m2026-07-09T21:18:01.125756Z[0m [[32m[1minfo     [0m] [1mInfrastructure cluster resource tags: [{'key': 'api.openshift.com/environment', 'value': 'production'}, {'key': 'api.openshift.com/id', 'value': '2o8clk8b6cj832lmkd2oj78crnf7qn6d'}, {'key': 'api.openshift.com/legal-entity-id',
 'value': '1pwwsfazToamNegaehP6eaDg80K'}, {'key': 'api.openshift.com/name', 'value': 'rh-nightly-pm'}, {'key': 'dns', 'value': 'external'}, {'key': 'red-hat-clustertype', 'value': 'rosa'}, {'key': 'red-hat-managed', 'value': 'true'}][0m [[0m[1m[34mutilities.infra[0m][0m [36mname[0m=[35mutilities.infra[0m
[2m2026-07-09T21:18:01.125859Z[0m [[32m[1minfo     [0m] [1mRunning on managed cluster, not using ca bundle[0m [[0m[1m[34mutilities.certificates_utils[0m][0m [36mname[0m=[35mutilities.certificates_utils[0m
[2m2026-07-09T21:18:01.125909Z[0m [[33m[1mwarning  [0m] [1mNo CA bundle found, using insecure access[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
[2m2026-07-09T21:18:01.497361Z[0m [[32m[1minfo     [0m] [1mInference output:
HTTP/1.1 404 Not Found
content-length: 53
content-type: application/json; charset=utf-8
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *

{"error":"Model with requested version is not found"}[0m [[0m[1m[34mutilities.inference_utils[0m][0m [36mname[0m=[35mutilities.inference_utils[0m
FAILED[1m
TEST: TestRawUnprivilegedUser.test_non_admin_deploy_raw_and_query_model[unprivileged_model_namespace0-unprivileged_s3_ovms_raw_inference_service0] STATUS: [0;31mFAILED[0m[0m
[2m2026-07-09T21:18:01.502212Z[0m [[31m[1merror    [0m] [1mTraceback (most recent call last):
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call
    result: TResult | None = func()
                             ~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 184, in pytest_runtest_call
    item.runtest()
    ~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1707, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 167, in pytest_pyfunc_call
    result = testfunction(**testargs)
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/kserve/authentication/test_non_admin_users.py", line 37, in test_non_admin_deploy_raw_and_query_model
    verify_inference_response(
    ~~~~~~~~~~~~~~~~~~~~~~~~~^
        inference_service=unprivileged_s3_ovms_raw_inference_service,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<3 lines>...
        use_default_query=True,
        ^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/utils.py", line 154, in verify_inference_response
    assert re.search(expected_response_text, formatted_res), (  # type: ignore[arg-type]
           ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Expected: {"model_name":"http-onnx-raw","model_version":"1","outputs":\[\{"name":"Plus214_Output_0","shape":\[1,10\],"datatype":"FP32","data":\[.*\]}]} not found in: {"error":"Modelwithrequestedversionisnotfound"}[0m [[0m[1m[34mconftest[0m][0m
[1m------------------------------------------------------- TEARDOWN -------------------------------------------------------[0m
[2m2026-07-09T21:18:01.502457Z[0m [[32m[1minfo     [0m] [1mDelete InferenceService http-onnx-raw[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:18:01.547833Z[0m [[32m[1minfo     [0m] [1mDeleting {'apiVersion': 'serving.kserve.io/v1beta1', 'kind': 'InferenceService', 'metadata': {'annotations': {'modelFormat': 'openvino_ir', 'serving.kserve.io/deploymentMode': 'Standard'}, 'creationTimestamp': '2026-07-09T21:17:39Z', 'finalizers': ['inferenceservice.finalizers', 'odh.inferenceservice.finalizers'], 'generation': 1, 'managedFields': [{'apiVersion': 'serving.kserve.io/v1beta1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'.': {}, 'f:serving.kserve.io/deploymentMode': {}}}, 'f:spec': {'.': {}, 'f:predictor': {'.': {}, 'f:model': {'.': {}, 'f:modelFormat': {'.': {}, 'f:name': {}}, 'f:runtime': {}, 'f:storage': {'.': {}, 'f:key': {}, 'f:path': {}}}}}}, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': '2026-07-09T21:17:39Z'}, {'apiVersion': 'serving.kserve.io/v1beta1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:finalizers': {'.': {}, 'v:"inferenceservice.finalizers"': {}, 'v:"odh.inferenceservice.finalizers"': {}}}}, 'manager': 'manager', 'operation': 'Update', 'time': '2026-07-09T21:17:39Z'}, {'apiVersion': 'serving.kserve.io/v1beta1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:status': {'.': {}, 'f:address': {'.': {}, 'f:url': {}}, 'f:components': {'.': {}, 'f:predictor': {'.': {}, 'f:url': {}}}, 'f:conditions': {}, 'f:deploymentMode': {}, 'f:modelStatus': {'.': {}, 'f:copies': {'.': {}, 'f:failedCopies': {}, 'f:totalCopies': {}}, 'f:states': {'.': {}, 'f:activeModelState': {}, 'f:targetModelState': {}}, 'f:transitionStatus': {}}, 'f:observedGeneration': {}, 'f:servingRuntimeName': {}, 'f:url': {}}}, 'manager': 'manager', 'operation': 'Update', 'subresource': 'status', 'time': '2026-07-09T21:17:52Z'}], 'name': 'http-onnx-raw', 'namespace': 'test-non-admin-raw', 'resourceVersion': '208253719', 'uid': '14e21102-34e9-4c05-9848-3d769eaf913a'}, 'spec': {'predictor': {'automountServiceAccountToken': False, 'model': {'modelFormat': {'name': 'openvino_ir'}, 'name': '', 'resources': {}, 'runtime': 'http-mnist-runtime', 'storage': {'key': 'ci-bucket-unprivileged', 'path': '/test-dir/'}}}}, 'status': {'address': {'url': 'http://http-onnx-raw-predictor.test-non-admin-raw.svc.cluster.local:8080'}, 'components': {'predictor': {'url': 'http://http-onnx-raw-predictor-test-non-admin-raw.example.com'}}, 'conditions': [{'lastTransitionTime': '2026-07-09T21:17:39Z', 'status': 'True', 'type': 'IngressReady'}, {'lastTransitionTime': '2026-07-09T21:17:52Z', 'message': 'ReplicaSet "http-onnx-raw-predictor-59f5884774" has successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'PredictorReady'}, {'lastTransitionTime': '2026-07-09T21:17:52Z', 'status': 'True', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-09T21:17:39Z', 'severity': 'Info', 'status': 'False', 'type': 'Stopped'}], 'deploymentMode': 'Standard', 'modelStatus': {'copies': {'failedCopies': 0, 'totalCopies': 1}, 'states': {'activeModelState': 'Loaded', 'targetModelState': 'Loaded'}, 'transitionStatus': 'UpToDate'}, 'observedGeneration': 1, 'servingRuntimeName': 'http-mnist-runtime', 'url': 'http://http-onnx-raw-predictor.test-non-admin-raw.svc.cluster.local'}}[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:18:01.572285Z[0m [[32m[1minfo     [0m] [1mWait until InferenceService http-onnx-raw is deleted[0m [[0m[1m[34mocp_resources InferenceService[0m][0m
[2m2026-07-09T21:18:02.613874Z[0m [[32m[1minfo     [0m] [1mDelete Secret ci-bucket-unprivileged[0m [[0m[1m[34mocp_resources Secret[0m][0m
[2m2026-07-09T21:18:02.660783Z[0m [[32m[1minfo     [0m] [1mDeleting {'kind': 'Secret', 'apiVersion': 'v1', 'metadata': {'name': 'ci-bucket-unprivileged', 'namespace': 'test-non-admin-raw', 'uid': '7b7a1e6e-a0c7-4a42-aecd-714cbd74b3de', 'resourceVersion': '208253397', 'creationTimestamp': '2026-07-09T21:17:39Z', 'labels': {'opendatahub.io/dashboard': 'true', 'opendatahub.io/managed': 'true'}, 'annotations': {'opendatahub.io/connection-type': 's3', 'serving.kserve.io/s3-endpoint': 's3.amazonaws.com/', 'serving.kserve.io/s3-region': 'us-east-1', 'serving.kserve.io/s3-useanoncredential': 'false', 'serving.kserve.io/s3-usehttps': '1', 'serving.kserve.io/s3-verifyssl': '0'}, 'managedFields': [{'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2026-07-09T21:17:39Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:data': {'.': {}, 'f:AWS_ACCESS_KEY_ID': {}, 'f:AWS_DEFAULT_REGION': {}, 'f:AWS_S3_BUCKET': {}, 'f:AWS_S3_ENDPOINT': {}, 'f:AWS_SECRET_ACCESS_KEY': {}}, 'f:metadata': {'f:annotations': {'.': {}, 'f:opendatahub.io/connection-type': {}, 'f:serving.kserve.io/s3-endpoint': {}, 'f:serving.kserve.io/s3-region': {}, 'f:serving.kserve.io/s3-useanoncredential': {}, 'f:serving.kserve.io/s3-usehttps': {}, 'f:serving.kserve.io/s3-verifyssl': {}}, 'f:labels': {'.': {}, 'f:opendatahub.io/dashboard': {}, 'f:opendatahub.io/managed': {}}}, 'f:type': {}}}]}, 'data': '*******', 'type': 'Opaque'}[0m [[0m[1m[34mocp_resources Secret[0m][0m
[2m2026-07-09T21:18:02.688855Z[0m [[32m[1minfo     [0m] [1mWait until Secret ci-bucket-unprivileged is deleted[0m [[0m[1m[34mocp_resources Secret[0m][0m
[2m2026-07-09T21:18:02.709536Z[0m [[32m[1minfo     [0m] [1mDelete ServingRuntime http-mnist-runtime[0m [[0m[1m[34mocp_resources ServingRuntime[0m][0m
[2m2026-07-09T21:18:02.751020Z[0m [[32m[1minfo     [0m] [1mDeleting {'apiVersion': 'serving.kserve.io/v1alpha1', 'kind': 'ServingRuntime', 'metadata': {'annotations': {'opendatahub.io/recommended-accelerators': '["nvidia.com/gpu"]', 'opendatahub.io/runtime-version': 'v2026.1.0', 'openshift.io/display-name': 'OpenVINO Model Server'}, 'creationTimestamp': '2026-07-09T21:17:39Z', 'generation': 1, 'labels': {'opendatahub.io/dashboard': 'true'}, 'managedFields': [{'apiVersion': 'serving.kserve.io/v1alpha1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'.': {}, 'f:opendatahub.io/recommended-accelerators': {}, 'f:opendatahub.io/runtime-version': {}, 'f:openshift.io/display-name': {}}, 'f:labels': {'.': {}, 'f:opendatahub.io/dashboard': {}}}, 'f:spec': {'.': {}, 'f:annotations': {'.': {}, 'f:opendatahub.io/kserve-runtime': {}, 'f:prometheus.io/path': {}, 'f:prometheus.io/port': {}}, 'f:containers': {}, 'f:multiModel': {}, 'f:protocolVersions': {}, 'f:supportedModelFormats': {}}}, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': '2026-07-09T21:17:39Z'}], 'name': 'http-mnist-runtime', 'namespace': 'test-non-admin-raw', 'resourceVersion': '208253394', 'uid': '0be10db8-da36-4d98-83d9-1c3bc153d7d2'}, 'spec': {'annotations': {'opendatahub.io/kserve-runtime': 'ovms', 'prometheus.io/path': '/metrics', 'prometheus.io/port': '8888'}, 'containers': [{'args': ['--model_name={{.Name}}', '--port=8001', '--rest_port=8888', '--model_path=/mnt/models', '--file_system_poll_wait_seconds=0', '--metrics_enable'], 'image': 'registry.redhat.io/rhoai/odh-openvino-model-server-rhel9@sha256:8c1b225509bfdc43756195f34a356d1e2ebd58fcd83fda3d37380dd7f5304df8', 'name': 'kserve-container', 'ports': [{'containerPort': 8888, 'protocol': 'TCP'}]}], 'multiModel': False, 'protocolVersions': ['v2', 'grpc-v2'], 'supportedModelFormats': [{'autoSelect': True, 'name': 'openvino_ir', 'version': 'opset13'}, {'name': 'onnx', 'version': '1'}, {'autoSelect': True, 'name': 'tensorflow', 'version': '1'}, {'autoSelect': True, 'name': 'tensorflow', 'version': '2'}, {'autoSelect': True, 'name': 'paddle', 'version': '2'}, {'autoSelect': True, 'name': 'pytorch', 'version': '2'}]}}[0m [[0m[1m[34mocp_resources ServingRuntime[0m][0m
[2m2026-07-09T21:18:02.774017Z[0m [[32m[1minfo     [0m] [1mWait until ServingRuntime http-mnist-runtime is deleted[0m [[0m[1m[34mocp_resources ServingRuntime[0m][0m
[2m2026-07-09T21:18:02.814112Z[0m [[32m[1minfo     [0m] [1mDelete Project test-non-admin-raw[0m [[0m[1m[34mocp_resources Project[0m][0m
[2m2026-07-09T21:18:02.864626Z[0m [[32m[1minfo     [0m] [1mDeleting {'kind': 'Project', 'apiVersion': 'project.openshift.io/v1', 'metadata': {'name': 'test-non-admin-raw', 'uid': 'c232012c-c80d-4359-9db3-85054e60c537', 'resourceVersion': '208253376', 'creationTimestamp': '2026-07-09T21:17:38Z', 'labels': {'kubernetes.io/metadata.name': 'test-non-admin-raw', 'pod-security.kubernetes.io/audit': 'restricted', 'pod-security.kubernetes.io/audit-version': 'latest', 'pod-security.kubernetes.io/warn': 'restricted', 'pod-security.kubernetes.io/warn-version': 'latest'}, 'annotations': {'openshift.io/description': '', 'openshift.io/display-name': '', 'openshift.io/requester': 'system:serviceaccount:kube-system:olminstall-cluster-admin', 'openshift.io/sa.scc.mcs': 's0:c31,c10', 'openshift.io/sa.scc.supplemental-groups': '1000950000/10000', 'openshift.io/sa.scc.uid-range': '1000950000/10000', 'security.openshift.io/MinimallySufficientPodSecurityStandard': 'restricted'}, 'managedFields': [{'manager': 'pod-security-admission-label-synchronization-controller', 'operation': 'Apply', 'apiVersion': 'v1', 'time': '2026-07-09T21:17:38Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'f:security.openshift.io/MinimallySufficientPodSecurityStandard': {}}, 'f:labels': {'f:pod-security.kubernetes.io/audit': {}, 'f:pod-security.kubernetes.io/audit-version': {}, 'f:pod-security.kubernetes.io/warn': {}, 'f:pod-security.kubernetes.io/warn-version': {}}}}}, {'manager': 'cluster-policy-controller', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2026-07-09T21:17:38Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'f:openshift.io/sa.scc.mcs': {}, 'f:openshift.io/sa.scc.supplemental-groups': {}, 'f:openshift.io/sa.scc.uid-range': {}}}}}, {'manager': 'openshift-apiserver', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2026-07-09T21:17:38Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'.': {}, 'f:openshift.io/description': {}, 'f:openshift.io/display-name': {}, 'f:openshift.io/requester': {}}, 'f:labels': {'.': {}, 'f:kubernetes.io/metadata.name': {}}}}}, {'manager': 'openshift-controller-manager', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2026-07-09T21:17:38Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:spec': {'f:finalizers': {}}}, 'subresource': 'finalize'}]}, 'spec': {'finalizers': ['kubernetes']}, 'status': {'phase': 'Active'}}[0m [[0m[1m[34mocp_resources Project[0m][0m
[2m2026-07-09T21:18:02.902020Z[0m [[32m[1minfo     [0m] [1mWait until Project test-non-admin-raw is deleted[0m [[0m[1m[34mocp_resources Project[0m][0m

_ TestRawUnprivilegedUser.test_non_admin_deploy_raw_and_query_model[unprivileged_model_namespace0-unprivileged_s3_ovms_raw_inference_service0] _
Traceback (most recent call last):
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call
    result: TResult | None = func()
                             ~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 184, in pytest_runtest_call
    item.runtest()
    ~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1707, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 167, in pytest_pyfunc_call
    result = testfunction(**testargs)
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/kserve/authentication/test_non_admin_users.py", line 37, in test_non_admin_deploy_raw_and_query_model
    verify_inference_response(
    ~~~~~~~~~~~~~~~~~~~~~~~~~^
        inference_service=unprivileged_s3_ovms_raw_inference_service,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<3 lines>...
        use_default_query=True,
        ^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/utils.py", line 154, in verify_inference_response
    assert re.search(expected_response_text, formatted_res), (  # type: ignore[arg-type]
           ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Expected: {"model_name":"http-onnx-raw","model_version":"1","outputs":\[\{"name":"Plus214_Output_0","shape":\[1,10\],"datatype":"FP32","data":\[.*\]}]} not found in: {"error":"Modelwithrequestedversionisnotfound"}
_ 5 of 5 completed, 1 Pass, 3 Fail, 0 Skip, 0 XPass, 0 XFail, 1 Error, 0 ReRun _
======= 3 failed, 1 passed, 219 deselected, 1 error in 396.12s (0:06:36) =======

==================================== ERRORS ====================================
_ ERROR at setup of TestKserveTokenAuthenticationRawForRest.test_disabled_raw_model_authentication[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] _
Traceback (most recent call last):
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call
    result: TResult | None = func()
                             ~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup
    item.session._setupstate.setup(item)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup
    col.setup()
    ~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup
    self._request._fillfixtures()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures
    item.funcargs[argname] = self.getfixturevalue(argname)
                             ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue
    fixturedef = self._get_active_fixturedef(argname)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef
    fixturedef.execute(request=subrequest)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1232, in execute
    result: FixtureValue = ihook.pytest_fixture_setup(
                           ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        fixturedef=self, request=request
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pytest_asyncio/plugin.py", line 924, in pytest_fixture_setup
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/setuponly.py", line 36, in pytest_fixture_setup
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1328, in pytest_fixture_setup
    result = call_fixture_func(fixturefunc, request, kwargs)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 998, in call_fixture_func
    fixture_result = next(generator)
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/kserve/authentication/conftest.py", line 100, in patched_remove_raw_authentication_isvc
    wait_for_raw_isvc_https_infer_ready(isvc=http_s3_ovms_raw_inference_service, token=None)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/utils.py", line 234, in wait_for_raw_isvc_https_infer_ready
    for ok in TimeoutSampler(wait_timeout=timeout, sleep=sleep, func=_https_infer_ok):
              ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/timeout_sampler/__init__.py", line 182, in __iter__
    raise TimeoutExpiredError(self._get_exception_log(exp=last_exp), last_exp=last_exp)
timeout_sampler.TimeoutExpiredError: Timed Out: 300
Function: tests.model_serving.model_server.utils._https_infer_ok
Last exception: N/A.
=================================== FAILURES ===================================
_ TestKserveTokenAuthenticationRawForRest.test_model_authentication_using_rest_raw[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] _
Traceback (most recent call last):
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call
    result: TResult | None = func()
                             ~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 184, in pytest_runtest_call
    item.runtest()
    ~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1707, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 167, in pytest_pyfunc_call
    result = testfunction(**testargs)
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py", line 39, in test_model_authentication_using_rest_raw
    verify_inference_response(
    ~~~~~~~~~~~~~~~~~~~~~~~~~^
        inference_service=http_s3_ovms_raw_inference_service,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
        token=http_raw_inference_token,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/utils.py", line 154, in verify_inference_response
    assert re.search(expected_response_text, formatted_res), (  # type: ignore[arg-type]
           ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Expected: {"model_name":"http-onnx","model_version":"1","outputs":\[\{"name":"Plus214_Output_0","shape":\[1,10\],"datatype":"FP32","data":\[.*\]}]} not found in: {"error":"Modelwithrequestedversionisnotfound"}
_ TestKserveTokenAuthenticationRawForRest.test_re_enabled_raw_model_authentication[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] _
Traceback (most recent call last):
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call
    result: TResult | None = func()
                             ~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 184, in pytest_runtest_call
    item.runtest()
    ~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1707, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 167, in pytest_pyfunc_call
    result = testfunction(**testargs)
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py", line 61, in test_re_enabled_raw_model_authentication
    verify_inference_response(
    ~~~~~~~~~~~~~~~~~~~~~~~~~^
        inference_service=http_s3_ovms_raw_inference_service,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
        token=http_raw_inference_token,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/utils.py", line 154, in verify_inference_response
    assert re.search(expected_response_text, formatted_res), (  # type: ignore[arg-type]
           ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Expected: {"model_name":"http-onnx","model_version":"1","outputs":\[\{"name":"Plus214_Output_0","shape":\[1,10\],"datatype":"FP32","data":\[.*\]}]} not found in: {"error":"Modelwithrequestedversionisnotfound"}
_ TestRawUnprivilegedUser.test_non_admin_deploy_raw_and_query_model[unprivileged_model_namespace0-unprivileged_s3_ovms_raw_inference_service0] _
Traceback (most recent call last):
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call
    result: TResult | None = func()
                             ~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 184, in pytest_runtest_call
    item.runtest()
    ~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1707, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 167, in pytest_pyfunc_call
    result = testfunction(**testargs)
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/kserve/authentication/test_non_admin_users.py", line 37, in test_non_admin_deploy_raw_and_query_model
    verify_inference_response(
    ~~~~~~~~~~~~~~~~~~~~~~~~~^
        inference_service=unprivileged_s3_ovms_raw_inference_service,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<3 lines>...
        use_default_query=True,
        ^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/utils.py", line 154, in verify_inference_response
    assert re.search(expected_response_text, formatted_res), (  # type: ignore[arg-type]
           ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Expected: {"model_name":"http-onnx-raw","model_version":"1","outputs":\[\{"name":"Plus214_Output_0","shape":\[1,10\],"datatype":"FP32","data":\[.*\]}]} not found in: {"error":"Modelwithrequestedversionisnotfound"}
- generated xml file: /workspace/tests-shared/tests-payload/results/model_server-smoke.xml -
=========================== short test summary info ============================
ERROR tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py::TestKserveTokenAuthenticationRawForRest::test_disabled_raw_model_authentication[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] - timeout_sampler.TimeoutExpiredError: Timed Out: 300
Function: tests.model_serving.model_server.utils._https_infer_ok
Last exception: N/A.
FAILED tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py::TestKserveTokenAuthenticationRawForRest::test_model_authentication_using_rest_raw[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] - AssertionError: Expected: {"model_name":"http-onnx","model_version":"1","outputs":\[\{"name":"Plus214_Output_0","shape":\[1,10\],"datatype":"FP32","data":\[.*\]}]} not found in: {"error":"Modelwithrequestedversionisnotfound"}
FAILED tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py::TestKserveTokenAuthenticationRawForRest::test_re_enabled_raw_model_authentication[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] - AssertionError: Expected: {"model_name":"http-onnx","model_version":"1","outputs":\[\{"name":"Plus214_Output_0","shape":\[1,10\],"datatype":"FP32","data":\[.*\]}]} not found in: {"error":"Modelwithrequestedversionisnotfound"}
FAILED tests/model_serving/model_server/kserve/authentication/test_non_admin_users.py::TestRawUnprivilegedUser::test_non_admin_deploy_raw_and_query_model[unprivileged_model_namespace0-unprivileged_s3_ovms_raw_inference_service0] - AssertionError: Expected: {"model_name":"http-onnx-raw","model_version":"1","outputs":\[\{"name":"Plus214_Output_0","shape":\[1,10\],"datatype":"FP32","data":\[.*\]}]} not found in: {"error":"Modelwithrequestedversionisnotfound"}
======= 3 failed, 1 passed, 219 deselected, 1 error in 396.13s (0:06:36) =======