--- pytest attempt 2026-09-14T09:05:58.218968+00:00 (/.local/bin/uv --directory /home/odh/opendatahub-tests...) ---
2026-09-14T09:06:00.703916Z [info     ] Writing tests log to /home/odh/opendatahub-tests/results/pytest-tests.log [conftest]
2026-09-14T09:06:00.704259Z [info     ] Trying to get client via new_client_from_config [ocp_resources.resource]
2026-09-14T09:06:00.829865Z [info     ] kind: DataScienceCluster api version: datasciencecluster.opendatahub.io/v2 [ocp_resources.resource]
2026-09-14T09:06:00.907272Z [info     ] kind: DSCInitialization api version: dscinitialization.opendatahub.io/v2 [ocp_resources.resource]
============================= test session starts ==============================
platform linux -- Python 3.14.7, pytest-9.1.1, pluggy-1.6.0 -- /home/odh/opendatahub-tests/.venv/bin/python3
cachedir: .pytest_cache
metadata: {'Python': '3.14.7', 'Platform': 'Linux-5.14.0-570.132.1.el9_6.x86_64-x86_64-with-glibc2.42', 'Packages': {'pytest': '9.1.1', 'pluggy': '1.6.0'}, 'Plugins': {'anyio': '4.14.2', 'langsmith': '0.10.10', 'asyncio': '1.4.0', 'dependency': '0.6.1', 'html': '4.2.0', 'metadata': '3.1.1', 'order': '1.5.0', 'progress': '1.4.0', 'testconfig': '0.2.0', 'xdist': '3.8.0', 'syrupy': '5.5.3'}}
rootdir: /home/odh/opendatahub-tests
configfile: pytest.ini
plugins: anyio-4.14.2, langsmith-0.10.10, asyncio-1.4.0, dependency-0.6.1, html-4.2.0, metadata-3.1.1, order-1.5.0, progress-1.4.0, testconfig-0.2.0, xdist-3.8.0, syrupy-5.5.3
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collecting ... collected 307 items / 298 deselected / 9 selected

tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py::TestKserveTokenAuthenticationRawForRest::test_model_authentication_using_rest_raw[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] 
----- test_model_authentication_using_rest_raw[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] -----
-------------------------------------------------------- SETUP --------------------------------------------------------
2026-09-14T09:06:01.774732Z [info     ] Executing session fixture: admin_client [conftest]
2026-09-14T09:06:01.774915Z [info     ] Trying to get client via new_client_from_config [ocp_resources.resource]
2026-09-14T09:06:01.776150Z [info     ] Executing session fixture: dsc_resource [conftest]
2026-09-14T09:06:01.776288Z [info     ] kind: DataScienceCluster api version: datasciencecluster.opendatahub.io/v2 [ocp_resources.resource]
2026-09-14T09:06:01.853316Z [info     ] Executing package fixture: enabled_kserve_in_dsc [conftest]
2026-09-14T09:06:01.874684Z [warning  ] Component kserve was already set to managementState Managed [utilities.data_science_cluster_utils] name=utilities.data_science_cluster_utils
2026-09-14T09:06:01.876096Z [info     ] Executing session fixture: event_loop_policy [conftest]
2026-09-14T09:06:01.876247Z [info     ] Executing session fixture: _syrupy_apply_ide_patches [conftest]
2026-09-14T09:06:01.876363Z [info     ] Executing session fixture: tmp_path_factory [conftest]
2026-09-14T09:06:01.876489Z [info     ] Executing session fixture: tests_tmp_dir [conftest]
2026-09-14T09:06:01.877707Z [info     ] Executing session fixture: os_path_environment [conftest]
2026-09-14T09:06:01.877921Z [info     ] Executing session fixture: tmpdir_factory [conftest]
2026-09-14T09:06:01.878174Z [info     ] Executing session fixture: bin_directory [conftest]
2026-09-14T09:06:01.879335Z [info     ] Executing session fixture: oc_binary_path [conftest]
2026-09-14T09:06:01.878482Z [warning  ] Using previously installed: /workspace/tests-shared/tests-payload/.tools/bin/oc [tests.conftest] name=tests.conftest
2026-09-14T09:06:01.879629Z [info     ] Executing session fixture: bin_directory_to_os_path [conftest]
2026-09-14T09:06:01.878600Z [info     ] OC binary path: /workspace/tests-shared/tests-payload/.tools/bin/oc [tests.conftest] name=tests.conftest
2026-09-14T09:06:01.878664Z [info     ] Adding /tmp/pytest-8p79K7KEWNKApYJ3grmQ2n/bin0 to $PATH [tests.conftest] name=tests.conftest
2026-09-14T09:06:01.880035Z [info     ] Executing session fixture: nodes [conftest]
2026-09-14T09:06:03.105720Z [info     ] Executing session fixture: dsci_resource [conftest]
2026-09-14T09:06:03.105914Z [info     ] kind: DSCInitialization api version: dscinitialization.opendatahub.io/v2 [ocp_resources.resource]
2026-09-14T09:06:03.124405Z [info     ] Executing session fixture: record_testsuite_property [conftest]
2026-09-14T09:06:03.124538Z [info     ] Executing session fixture: junitxml_plugin [conftest]
2026-09-14T09:06:03.124723Z [info     ] Executing session fixture: cluster_sanity_scope_session [conftest]
2026-09-14T09:06:03.123874Z [info     ] Check cluster sanity.          [utilities.infra] name=utilities.infra
2026-09-14T09:06:03.125002Z [info     ] Verify all nodes are in a healthy condition. [ocp_utilities.infra]
2026-09-14T09:06:03.179987Z [info     ] Verify all nodes are schedulable. [ocp_utilities.infra]
2026-09-14T09:06:03.234428Z [info     ] Wait for DSCI default-dsci to be in Ready status. [utilities.infra] name=utilities.infra
2026-09-14T09:06:03.235053Z [info     ] Get DSCInitialization default-dsci status [ocp_resources DSCInitialization]
2026-09-14T09:06:03.253911Z [success  ] Function: utilities.infra.wait_for_dsci_status_ready succeeded after 4.2438507080078125e-05 [0:00:00.000042]. [timeout_sampler]
2026-09-14T09:06:03.253009Z [info     ] Wait for DSC default-dsc are Ready. [utilities.infra] name=utilities.infra
2026-09-14T09:06:03.254183Z [info     ] Get DataScienceCluster default-dsc status [ocp_resources DataScienceCluster]
2026-09-14T09:06:03.275862Z [success  ] Function: utilities.infra.wait_for_dsc_status_ready succeeded after 4.5299530029296875e-05 [0:00:00.000045]. [timeout_sampler]
2026-09-14T09:06:03.276018Z [info     ] Executing session fixture: autouse_fixtures [conftest]
2026-09-14T09:06:03.276159Z [info     ] Executing session fixture: kserve_health_check [conftest]
2026-09-14T09:06:03.318276Z [info     ] kind: Deployment api version: apps/v1 [ocp_resources.resource]
2026-09-14T09:06:03.389034Z [info     ] KServe component health check passed [tests.model_serving.model_server.kserve.conftest]
2026-09-14T09:06:03.389214Z [info     ] Executing session fixture: pytestconfig [conftest]
2026-09-14T09:06:03.389378Z [info     ] Executing session fixture: aws_access_key_id [conftest]
2026-09-14T09:06:03.389533Z [info     ] Executing session fixture: aws_secret_access_key [conftest]
2026-09-14T09:06:03.389722Z [info     ] Executing session fixture: ci_s3_bucket_endpoint [conftest]
2026-09-14T09:06:03.389879Z [info     ] Executing session fixture: valid_aws_config [conftest]
2026-09-14T09:06:03.435224Z [info     ] AWS credentials validated successfully via STS GetCallerIdentity [tests.conftest] name=tests.conftest
2026-09-14T09:06:03.437988Z [info     ] Executing session fixture: use_unprivileged_client [conftest]
2026-09-14T09:06:03.438261Z [info     ] Executing session fixture: kubconfig_filepath [conftest]
2026-09-14T09:06:03.438581Z [info     ] Executing session fixture: cluster_authentication [conftest]
2026-09-14T09:06:03.438962Z [info     ] kind: Authentication api version: config.openshift.io/v1 [ocp_resources.resource]
2026-09-14T09:06:03.455532Z [info     ] Executing session fixture: is_byoidc [conftest]
2026-09-14T09:06:03.474551Z [info     ] Executing session fixture: non_admin_user_password [conftest]
2026-09-14T09:06:03.474778Z [info     ] Executing session fixture: unprivileged_client [conftest]
2026-09-14T09:06:03.472473Z [warning  ] Unprivileged client is not enabled, using admin client [tests.conftest] name=tests.conftest
2026-09-14T09:06:03.475088Z [info     ] Executing session fixture: teardown_resources [conftest]
2026-09-14T09:06:03.475234Z [info     ] Executing session fixture: ci_s3_bucket_name [conftest]
2026-09-14T09:06:03.475359Z [info     ] Executing session fixture: ci_s3_bucket_region [conftest]
2026-09-14T09:06:03.475466Z [info     ] Executing session fixture: models_s3_bucket_name [conftest]
2026-09-14T09:06:03.475576Z [info     ] Executing session fixture: models_s3_bucket_region [conftest]
2026-09-14T09:06:03.475719Z [info     ] Executing session fixture: models_s3_bucket_endpoint [conftest]
2026-09-14T09:06:03.475836Z [info     ] Executing class fixture: unprivileged_model_namespace [conftest]
2026-09-14T09:06:03.475948Z [info     ] kind: ProjectRequest api version: project.openshift.io/v1 [ocp_resources.resource]
2026-09-14T09:06:03.476062Z [info     ] Create ProjectRequest test-kserve-raw-token-authentication [ocp_resources ProjectRequest]
2026-09-14T09:06:03.476176Z [info     ] Posting {'apiVersion': 'project.openshift.io/v1', 'kind': 'ProjectRequest', 'metadata': {'name': 'test-kserve-raw-token-authentication'}} [ocp_resources ProjectRequest]
2026-09-14T09:06:03.653183Z [info     ] kind: Project api version: project.openshift.io/v1 [ocp_resources.resource]
2026-09-14T09:06:03.653471Z [info     ] Wait for Project test-kserve-raw-token-authentication status to be Active [ocp_resources Project]
2026-09-14T09:06:03.686102Z [info     ] Status of Project test-kserve-raw-token-authentication is Active [ocp_resources Project]
2026-09-14T09:06:03.686268Z [success  ] Function: lambda: self.exists succeeded after 2.86102294921875e-05 [0:00:00.000029]. [timeout_sampler]
2026-09-14T09:06:03.686389Z [info     ] Executing class fixture: http_s3_ovms_serving_runtime [conftest]
2026-09-14T09:06:03.686495Z [info     ] kind: Template api version: template.openshift.io/v1 [ocp_resources.resource]
2026-09-14T09:06:03.816688Z [info     ] kind: ServingRuntime api version: serving.kserve.io/v1alpha1 [ocp_resources.resource]
2026-09-14T09:06:03.816923Z [info     ] Create ServingRuntime http-mnist-runtime [ocp_resources ServingRuntime]
2026-09-14T09:06:03.817124Z [info     ] Posting {'apiVersion': 'serving.kserve.io/v1alpha1', 'kind': 'ServingRuntime', 'metadata': {'annotations': {'opendatahub.io/recommended-accelerators': '["nvidia.com/gpu"]', 'opendatahub.io/runtime-version': 'v2026.1.0', 'openshift.io/display-name': 'OpenVINO Model Server'}, 'labels': {'opendatahub.io/dashboard': 'true'}, 'name': 'http-mnist-runtime', 'namespace': 'test-kserve-raw-token-authentication'}, 'spec': {'annotations': {'opendatahub.io/kserve-runtime': 'ovms', 'prometheus.io/path': '/metrics', 'prometheus.io/port': '8888'}, 'containers': [{'args': ['--model_name={{.Name}}', '--port=8001', '--rest_port=8888', '--model_path=/mnt/models', '--file_system_poll_wait_seconds=0', '--metrics_enable'], 'image': 'registry.redhat.io/rhoai/odh-openvino-model-server-rhel9@sha256:3c6a0438295d80ae45e0a83c0b1e5d2d6bb955b286f34d042312b9e1763a3cab', 'name': 'kserve-container', 'ports': [{'containerPort': 8888, 'protocol': 'TCP'}]}], 'multiModel': False, 'protocolVersions': ['v2', 'grpc-v2'], 'supportedModelFormats': [{'autoSelect': True, 'name': 'openvino_ir', 'version': 'opset13'}, {'name': 'onnx', 'version': '1'}, {'autoSelect': True, 'name': 'tensorflow', 'version': '1'}, {'autoSelect': True, 'name': 'tensorflow', 'version': '2'}, {'autoSelect': True, 'name': 'paddle', 'version': '2'}, {'autoSelect': True, 'name': 'pytorch', 'version': '2'}]}} [ocp_resources ServingRuntime]
2026-09-14T09:06:09.920265Z [info     ] Executing class fixture: ci_endpoint_s3_secret [conftest]
2026-09-14T09:06:09.943721Z [info     ] Create Secret ci-bucket-secret [ocp_resources Secret]
2026-09-14T09:06:09.943914Z [info     ] Posting {'apiVersion': 'v1', 'kind': 'Secret', 'metadata': {'name': 'ci-bucket-secret', 'labels': {'opendatahub.io/managed': 'true', 'opendatahub.io/dashboard': 'true'}, 'annotations': {'opendatahub.io/connection-type': 's3', 'serving.kserve.io/s3-endpoint': 's3.us-east-1.amazonaws.com/', 'serving.kserve.io/s3-region': 'us-east-1', 'serving.kserve.io/s3-useanoncredential': 'false', 'serving.kserve.io/s3-verifyssl': '0', 'serving.kserve.io/s3-usehttps': '1'}, 'namespace': 'test-kserve-raw-token-authentication'}, 'data': '*******'} [ocp_resources Secret]
2026-09-14T09:06:09.987483Z [info     ] Wait until Secret ci-bucket-secret is created [ocp_resources Secret]
2026-09-14T09:06:10.006292Z [success  ] Function: lambda: self.exists succeeded after 2.9802322387695312e-05 [0:00:00.000030]. [timeout_sampler]
2026-09-14T09:06:10.006462Z [info     ] Executing class fixture: models_endpoint_s3_secret [conftest]
2026-09-14T09:06:10.025939Z [info     ] Create Secret models-bucket-secret [ocp_resources Secret]
2026-09-14T09:06:10.026167Z [info     ] Posting {'apiVersion': 'v1', 'kind': 'Secret', 'metadata': {'name': 'models-bucket-secret', 'labels': {'opendatahub.io/managed': 'true', 'opendatahub.io/dashboard': 'true'}, 'annotations': {'opendatahub.io/connection-type': 's3', 'serving.kserve.io/s3-endpoint': 's3.us-east-2.amazonaws.com/', 'serving.kserve.io/s3-region': 'us-east-2', 'serving.kserve.io/s3-useanoncredential': 'false', 'serving.kserve.io/s3-verifyssl': '0', 'serving.kserve.io/s3-usehttps': '1'}, 'namespace': 'test-kserve-raw-token-authentication'}, 'data': '*******'} [ocp_resources Secret]
2026-09-14T09:06:10.074217Z [info     ] Wait until Secret models-bucket-secret is created [ocp_resources Secret]
2026-09-14T09:06:10.091869Z [success  ] Function: lambda: self.exists succeeded after 3.266334533691406e-05 [0:00:00.000033]. [timeout_sampler]
2026-09-14T09:06:10.092047Z [info     ] Executing class fixture: model_service_account [conftest]
2026-09-14T09:06:10.092196Z [info     ] Create ServiceAccount models-bucket-sa [ocp_resources ServiceAccount]
2026-09-14T09:06:10.092816Z [info     ] Posting {'apiVersion': 'v1', 'kind': 'ServiceAccount', 'metadata': {'name': 'models-bucket-sa', 'namespace': 'test-kserve-raw-token-authentication'}, 'secrets': [{'name': 'models-bucket-secret'}]} [ocp_resources ServiceAccount]
2026-09-14T09:06:10.129566Z [info     ] Executing class fixture: http_s3_ovms_raw_inference_service [conftest]
2026-09-14T09:06:10.149565Z [info     ] kind: InferenceService api version: serving.kserve.io/v1beta1 [ocp_resources.resource]
2026-09-14T09:06:10.149765Z [info     ] Create InferenceService http-onnx [ocp_resources InferenceService]
2026-09-14T09:06:10.149900Z [info     ] Posting {'apiVersion': 'serving.kserve.io/v1beta1', 'kind': 'InferenceService', 'metadata': {'name': 'http-onnx', 'labels': {'networking.kserve.io/visibility': 'exposed'}, 'annotations': {'serving.kserve.io/deploymentMode': 'RawDeployment', 'security.opendatahub.io/enable-auth': 'true'}, 'namespace': 'test-kserve-raw-token-authentication'}, 'spec': {'predictor': {'model': {'modelFormat': {'name': 'openvino_ir'}, 'version': None, 'runtime': 'http-mnist-runtime', 'storage': {'key': 'ci-bucket-secret', 'path': '/test-dir/'}}, 'serviceAccountName': 'models-bucket-sa'}}} [ocp_resources InferenceService]
2026-09-14T09:06:16.314718Z [info     ] Waiting for pods to be created [utilities.infra] name=utilities.infra
2026-09-14T09:06:17.350287Z [info     ] Waiting for pods to be created [utilities.infra] name=utilities.infra
2026-09-14T09:06:18.388512Z [info     ] Waiting for pods to be created [utilities.infra] name=utilities.infra
2026-09-14T09:06:19.426374Z [info     ] Waiting for pods to be created [utilities.infra] name=utilities.infra
2026-09-14T09:06:20.464106Z [info     ] Waiting for pods to be created [utilities.infra] name=utilities.infra
2026-09-14T09:06:21.501288Z [info     ] Waiting for pods to be created [utilities.infra] name=utilities.infra
2026-09-14T09:06:22.538965Z [info     ] Waiting for pods to be created [utilities.infra] name=utilities.infra
2026-09-14T09:06:23.577235Z [info     ] Waiting for pods to be created [utilities.infra] name=utilities.infra
2026-09-14T09:06:24.614265Z [info     ] Waiting for pods to be created [utilities.infra] name=utilities.infra
2026-09-14T09:06:25.652399Z [info     ] Waiting for pods to be created [utilities.infra] name=utilities.infra
2026-09-14T09:06:26.687275Z [info     ] Waiting for pods to be created [utilities.infra] name=utilities.infra
2026-09-14T09:06:27.722881Z [info     ] Waiting for pods to be created [utilities.infra] name=utilities.infra
2026-09-14T09:06:28.760897Z [info     ] Waiting for pods to be created [utilities.infra] name=utilities.infra
2026-09-14T09:06:29.796976Z [info     ] Waiting for pods to be created [utilities.infra] name=utilities.infra
2026-09-14T09:06:30.834071Z [info     ] Waiting for pods to be created [utilities.infra] name=utilities.infra
2026-09-14T09:06:31.871100Z [info     ] Waiting for pods to be created [utilities.infra] name=utilities.infra
2026-09-14T09:06:31.920229Z [success  ] Function: utilities.infra.wait_for_isvc_pods succeeded after 15.556416273117065 [0:00:15.556416]. [timeout_sampler]
2026-09-14T09:06:31.936550Z [info     ] Verifying no failed pods       [utilities.infra] name=utilities.infra
2026-09-14T09:15:58.422875Z [info     ] Delete InferenceService http-onnx [ocp_resources InferenceService]
2026-09-14T09:15:58.460251Z [info     ] Deleting {'apiVersion': 'serving.kserve.io/v1beta1', 'kind': 'InferenceService', 'metadata': {'annotations': {'modelFormat': 'openvino_ir', 'security.opendatahub.io/enable-auth': 'true', 'serving.kserve.io/deploymentMode': 'Standard'}, 'creationTimestamp': '2026-09-14T09:06:16Z', 'finalizers': ['inferenceservice.finalizers', 'odh.inferenceservice.finalizers'], 'generation': 1, 'labels': {'networking.kserve.io/visibility': 'exposed'}, 'managedFields': [{'apiVersion': 'serving.kserve.io/v1beta1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'.': {}, 'f:security.opendatahub.io/enable-auth': {}, 'f:serving.kserve.io/deploymentMode': {}}, 'f:labels': {'.': {}, 'f:networking.kserve.io/visibility': {}}}, 'f:spec': {'.': {}, 'f:predictor': {'.': {}, 'f:model': {'.': {}, 'f:modelFormat': {'.': {}, 'f:name': {}}, 'f:runtime': {}, 'f:storage': {'.': {}, 'f:key': {}, 'f:path': {}}}, 'f:serviceAccountName': {}}}}, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': '2026-09-14T09:06:16Z'}, {'apiVersion': 'serving.kserve.io/v1beta1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:finalizers': {'.': {}, 'v:"inferenceservice.finalizers"': {}, 'v:"odh.inferenceservice.finalizers"': {}}}}, 'manager': 'manager', 'operation': 'Update', 'time': '2026-09-14T09:06:16Z'}, {'apiVersion': 'serving.kserve.io/v1beta1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:status': {'.': {}, 'f:address': {'.': {}, 'f:url': {}}, 'f:components': {'.': {}, 'f:predictor': {}}, 'f:conditions': {}, 'f:deploymentMode': {}, 'f:modelStatus': {'.': {}, 'f:copies': {'.': {}, 'f:failedCopies': {}}, 'f:states': {'.': {}, 'f:activeModelState': {}, 'f:targetModelState': {}}, 'f:transitionStatus': {}}, 'f:observedGeneration': {}, 'f:servingRuntimeName': {}, 'f:url': {}}}, 'manager': 'manager', 'operation': 'Update', 'subresource': 'status', 'time': '2026-09-14T09:06:31Z'}], 'name': 'http-onnx', 'namespace': 'test-kserve-raw-token-authentication', 'resourceVersion': '306755095', 'uid': 'ffe4543f-d699-44fb-a484-034ddbf17df5'}, 'spec': {'predictor': {'automountServiceAccountToken': False, 'model': {'modelFormat': {'name': 'openvino_ir'}, 'name': '', 'resources': {}, 'runtime': 'http-mnist-runtime', 'storage': {'key': 'ci-bucket-secret', 'path': '/test-dir/'}}, 'serviceAccountName': 'models-bucket-sa'}}, 'status': {'address': {'url': 'https://http-onnx-predictor.test-kserve-raw-token-authentication.svc.cluster.local:8443'}, 'components': {'predictor': {}}, 'conditions': [{'lastTransitionTime': '2026-09-14T09:06:16Z', 'status': 'True', 'type': 'IngressReady'}, {'lastTransitionTime': '2026-09-14T09:06:31Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'PredictorReady'}, {'lastTransitionTime': '2026-09-14T09:06:31Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-09-14T09:06:16Z', 'severity': 'Info', 'status': 'False', 'type': 'Stopped'}], 'deploymentMode': 'Standard', 'modelStatus': {'copies': {'failedCopies': 0}, 'states': {'activeModelState': '', 'targetModelState': 'Pending'}, 'transitionStatus': 'InProgress'}, 'observedGeneration': 1, 'servingRuntimeName': 'http-mnist-runtime', 'url': 'https://http-onnx-test-kserve-raw-token-authentication.apps.rosa.rh-nightly-pm.nuth.p3.openshiftapps.com'}} [ocp_resources InferenceService]
2026-09-14T09:15:58.485996Z [info     ] Wait until InferenceService http-onnx is deleted [ocp_resources InferenceService]
2026-09-14T09:15:59.522519Z [success  ] Function: lambda: self.exists succeeded after 1.01735258102417 [0:00:01.017353]. [timeout_sampler]
2026-09-14T09:15:59.522695Z [error    ] Function: utilities.infra.get_pods_by_isvc_label failed after 564.8436465263367 [0:09:24.843647]. [timeout_sampler]
2026-09-14T09:15:59.522814Z [info     ] Delete InferenceService http-onnx [ocp_resources InferenceService]
2026-09-14T09:15:59.540222Z [warning  ] Resource InferenceService http-onnx was not found, and wasn't deleted [ocp_resources InferenceService]
ERROR
_ ERROR at setup of TestKserveTokenAuthenticationRawForRest.test_model_authentication_using_rest_raw[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] _
Traceback (most recent call last):
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call
    result: TResult | None = func()
                             ~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup
    item.session._setupstate.setup(item)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup
    col.setup()
    ~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup
    self._request._fillfixtures()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures
    item.funcargs[argname] = self.getfixturevalue(argname)
                             ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue
    fixturedef = self._get_active_fixturedef(argname)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef
    fixturedef.execute(request=subrequest)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1232, in execute
    result: FixtureValue = ihook.pytest_fixture_setup(
                           ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        fixturedef=self, request=request
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pytest_asyncio/plugin.py", line 924, in pytest_fixture_setup
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/setuponly.py", line 36, in pytest_fixture_setup
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1328, in pytest_fixture_setup
    result = call_fixture_func(fixturefunc, request, kwargs)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 998, in call_fixture_func
    fixture_result = next(generator)
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/kserve/authentication/conftest.py", line 143, in http_s3_ovms_raw_inference_service
    with create_isvc(
         ~~~~~~~~~~~^
        client=unprivileged_client,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
        external_route=True,
        ^^^^^^^^^^^^^^^^^^^^
    ) as isvc:
    ^
  File "/usr/lib64/python3.14/contextlib.py", line 141, in __enter__
    return next(self.gen)
  File "/home/odh/opendatahub-tests/utilities/inference_utils.py", line 805, in create_isvc
    verify_no_failed_pods(
    ~~~~~~~~~~~~~~~~~~~~~^
        client=client,
        ^^^^^^^^^^^^^^
    ...<2 lines>...
        timeout=timeout_watch.remaining_time(),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/odh/opendatahub-tests/utilities/infra.py", line 766, in verify_no_failed_pods
    for pods in TimeoutSampler(
                ~~~~~~~~~~~~~~^
        wait_timeout=timeout,
        ^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
        runtime_name=runtime_name,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ):
    ^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/timeout_sampler/__init__.py", line 319, in __iter__
    time.sleep(self.sleep)
    ~~~~~~~~~~^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 799, in _sigint_handler
    sys.exit(signal_received)
    ~~~~~~~~^^^^^^^^^^^^^^^^^
SystemExit: 2
_ 1 of 9 completed, 0 Pass, 0 Fail, 0 Skip, 0 XPass, 0 XFail, 1 Error, 0 ReRun _

tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py::TestKserveTokenAuthenticationRawForRest::test_disabled_raw_model_authentication[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] 
TEST: TestKserveTokenAuthenticationRawForRest.test_model_authentication_using_rest_raw[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] [setup] STATUS: ERROR
2026-09-14T09:15:59.551404Z [error    ] Traceback (most recent call last):
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call
    result: TResult | None = func()
                             ~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup
    item.session._setupstate.setup(item)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup
    col.setup()
    ~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup
    self._request._fillfixtures()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures
    item.funcargs[argname] = self.getfixturevalue(argname)
                             ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue
    fixturedef = self._get_active_fixturedef(argname)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef
    fixturedef.execute(request=subrequest)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1232, in execute
    result: FixtureValue = ihook.pytest_fixture_setup(
                           ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        fixturedef=self, request=request
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pytest_asyncio/plugin.py", line 924, in pytest_fixture_setup
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper
    return result.get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result
    raise exc.with_traceback(tb)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper
    res = yield
          ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/setuponly.py", line 36, in pytest_fixture_setup
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1328, in pytest_fixture_setup
    result = call_fixture_func(fixturefunc, request, kwargs)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 998, in call_fixture_func
    fixture_result = next(generator)
  File "/home/odh/opendatahub-tests/tests/model_serving/model_server/kserve/authentication/conftest.py", line 143, in http_s3_ovms_raw_inference_service
    with create_isvc(
         ~~~~~~~~~~~^
        client=unprivileged_client,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
        external_route=True,
        ^^^^^^^^^^^^^^^^^^^^
    ) as isvc:
    ^
  File "/usr/lib64/python3.14/contextlib.py", line 141, in __enter__
    return next(self.gen)
  File "/home/odh/opendatahub-tests/utilities/inference_utils.py", line 805, in create_isvc
    verify_no_failed_pods(
    ~~~~~~~~~~~~~~~~~~~~~^
        client=client,
        ^^^^^^^^^^^^^^
    ...<2 lines>...
        timeout=timeout_watch.remaining_time(),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/odh/opendatahub-tests/utilities/infra.py", line 766, in verify_no_failed_pods
    for pods in TimeoutSampler(
                ~~~~~~~~~~~~~~^
        wait_timeout=timeout,
        ^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
        runtime_name=runtime_name,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ):
    ^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/timeout_sampler/__init__.py", line 319, in __iter__
    time.sleep(self.sleep)
    ~~~~~~~~~~^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 799, in _sigint_handler
    sys.exit(signal_received)
    ~~~~~~~~^^^^^^^^^^^^^^^^^
SystemExit: 2 [conftest]
------------------------------------------------------- TEARDOWN -------------------------------------------------------

------ test_disabled_raw_model_authentication[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] ------
-------------------------------------------------------- SETUP --------------------------------------------------------
ERROR
_ ERROR at setup of TestKserveTokenAuthenticationRawForRest.test_disabled_raw_model_authentication[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] _
Traceback (most recent call last):
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call
    result: TResult | None = func()
                             ~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup
    item.session._setupstate.setup(item)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup
    col.setup()
    ~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup
    self._request._fillfixtures()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures
    item.funcargs[argname] = self.getfixturevalue(argname)
                             ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue
    fixturedef = self._get_active_fixturedef(argname)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef
    fixturedef.execute(request=subrequest)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1221, in execute
    assert not self._finalizers
           ^^^^^^^^^^^^^^^^^^^^
AssertionError
_ 2 of 9 completed, 0 Pass, 0 Fail, 0 Skip, 0 XPass, 0 XFail, 2 Error, 0 ReRun _

tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py::TestKserveTokenAuthenticationRawForRest::test_re_enabled_raw_model_authentication[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] 
TEST: TestKserveTokenAuthenticationRawForRest.test_disabled_raw_model_authentication[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] [setup] STATUS: ERROR
2026-09-14T09:15:59.554996Z [error    ] Traceback (most recent call last):
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call
    result: TResult | None = func()
                             ~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup
    item.session._setupstate.setup(item)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup
    col.setup()
    ~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup
    self._request._fillfixtures()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures
    item.funcargs[argname] = self.getfixturevalue(argname)
                             ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue
    fixturedef = self._get_active_fixturedef(argname)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef
    fixturedef.execute(request=subrequest)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1221, in execute
    assert not self._finalizers
           ^^^^^^^^^^^^^^^^^^^^
AssertionError [conftest]
------------------------------------------------------- TEARDOWN -------------------------------------------------------
ERROR
_ ERROR at setup of TestKserveTokenAuthenticationRawForRest.test_re_enabled_raw_model_authentication[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] _
Traceback (most recent call last):
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call
    result: TResult | None = func()
                             ~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup
    item.session._setupstate.setup(item)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup
    col.setup()
    ~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup
    self._request._fillfixtures()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures
    item.funcargs[argname] = self.getfixturevalue(argname)
                             ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue
    fixturedef = self._get_active_fixturedef(argname)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef
    fixturedef.execute(request=subrequest)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1221, in execute
    assert not self._finalizers
           ^^^^^^^^^^^^^^^^^^^^
AssertionError
_ 3 of 9 completed, 0 Pass, 0 Fail, 0 Skip, 0 XPass, 0 XFail, 3 Error, 0 ReRun _

tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py::TestKserveTokenAuthenticationRawForRest::test_cross_model_authentication_raw[http_s3_ovms_raw_inference_service_20-unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] 
----- test_re_enabled_raw_model_authentication[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] -----
-------------------------------------------------------- SETUP --------------------------------------------------------

TEST: TestKserveTokenAuthenticationRawForRest.test_re_enabled_raw_model_authentication[unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] [setup] STATUS: ERROR
2026-09-14T09:15:59.559284Z [error    ] Traceback (most recent call last):
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call
    result: TResult | None = func()
                             ~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup
    return (yield)
            ^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup
    item.session._setupstate.setup(item)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup
    col.setup()
    ~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup
    self._request._fillfixtures()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures
    item.funcargs[argname] = self.getfixturevalue(argname)
                             ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue
    fixturedef = self._get_active_fixturedef(argname)
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef
    fixturedef.execute(request=subrequest)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1221, in execute
    assert not self._finalizers
           ^^^^^^^^^^^^^^^^^^^^
AssertionError [conftest]
------------------------------------------------------- TEARDOWN -------------------------------------------------------

 test_cross_model_authentication_raw[http_s3_ovms_raw_inference_service_20-unprivileged_model_namespace0-http_s3_ovms_raw_inference_service0] 
-------------------------------------------------------- SETUP --------------------------------------------------------
2026-09-14T09:15:59.560237Z [info     ] Executing class fixture: model_service_account_2 [conftest]
2026-09-14T09:15:59.560341Z [info     ] Create ServiceAccount models-bucket-sa-2 [ocp_resources ServiceAccount]
2026-09-14T09:15:59.560444Z [info     ] Posting {'apiVersion': 'v1', 'kind': 'ServiceAccount', 'metadata': {'name': 'models-bucket-sa-2', 'namespace': 'test-kserve-raw-token-authentication'}, 'secrets': [{'name': 'models-bucket-secret'}]} [ocp_resources ServiceAccount]
2026-09-14T09:15:59.595151Z [info     ] Executing class fixture: http_s3_ovms_raw_inference_service_2 [conftest]
2026-09-14T09:15:59.613936Z [info     ] Create InferenceService http-onnx-2 [ocp_resources InferenceService]
2026-09-14T09:15:59.614099Z [info     ] Posting {'apiVersion': 'serving.kserve.io/v1beta1', 'kind': 'InferenceService', 'metadata': {'name': 'http-onnx-2', 'labels': {'networking.kserve.io/visibility': 'exposed'}, 'annotations': {'serving.kserve.io/deploymentMode': 'RawDeployment', 'security.opendatahub.io/enable-auth': 'true'}, 'namespace': 'test-kserve-raw-token-authentication'}, 'spec': {'predictor': {'model': {'modelFormat': {'name': 'openvino_ir'}, 'version': None, 'runtime': 'http-mnist-runtime', 'storage': {'key': 'ci-bucket-secret', 'path': '/test-dir/'}}, 'serviceAccountName': 'models-bucket-sa-2'}}} [ocp_resources InferenceService]
2026-09-14T09:15:59.678555Z [info     ] Waiting for pods to be created [utilities.infra] name=utilities.infra
2026-09-14T09:16:00.717330Z [info     ] Waiting for pods to be created [utilities.infra] name=utilities.infra
2026-09-14T09:16:00.767805Z [success  ] Function: utilities.infra.wait_for_isvc_pods succeeded after 1.0387747287750244 [0:00:01.038775]. [timeout_sampler]
2026-09-14T09:16:00.785356Z [info     ] Verifying no failed pods       [utilities.infra] name=utilities.infra