--- BVT pytest attempt 2026-07-04T18:59:32.129898+00:00 (/.local/bin/uv --directory /home/odh/opendatahub-tests...) --- 2026-07-04T18:59:34.101199Z [info ] Writing tests log to /home/odh/opendatahub-tests/results/pytest-tests.log [conftest] 2026-07-04T18:59:34.103666Z [info ] Trying to get client via new_client_from_config [ocp_resources.resource] /home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/urllib3/connectionpool.py:1110: InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.ods-qe-psi-07.osp.rh-ods.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( /home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/urllib3/connectionpool.py:1110: InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.ods-qe-psi-07.osp.rh-ods.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( /home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/urllib3/connectionpool.py:1110: InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.ods-qe-psi-07.osp.rh-ods.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( /home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/urllib3/connectionpool.py:1110: InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.ods-qe-psi-07.osp.rh-ods.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( /home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/urllib3/connectionpool.py:1110: InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.ods-qe-psi-07.osp.rh-ods.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( 2026-07-04T18:59:34.208396Z [info ] kind: DataScienceCluster api version: datasciencecluster.opendatahub.io/v2 [ocp_resources.resource] /home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/urllib3/connectionpool.py:1110: InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.ods-qe-psi-07.osp.rh-ods.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( /home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/urllib3/connectionpool.py:1110: InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.ods-qe-psi-07.osp.rh-ods.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( /home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/urllib3/connectionpool.py:1110: InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.ods-qe-psi-07.osp.rh-ods.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( /home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/urllib3/connectionpool.py:1110: InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.ods-qe-psi-07.osp.rh-ods.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( 2026-07-04T18:59:34.265858Z [info ] kind: DSCInitialization api version: dscinitialization.opendatahub.io/v2 [ocp_resources.resource] /home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/urllib3/connectionpool.py:1110: InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.ods-qe-psi-07.osp.rh-ods.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( ============================= test session starts ============================== platform linux -- Python 3.14.6, pytest-9.1.1, pluggy-1.6.0 -- /home/odh/opendatahub-tests/.venv/bin/python3 cachedir: .pytest_cache metadata: {'Python': '3.14.6', 'Platform': 'Linux-5.14.0-570.120.1.el9_6.x86_64-x86_64-with-glibc2.42', 'Packages': {'pytest': '9.1.1', 'pluggy': '1.6.0'}, 'Plugins': {'anyio': '4.14.0', 'langsmith': '0.9.0', 'asyncio': '1.4.0', 'dependency': '0.6.1', 'html': '4.2.0', 'metadata': '3.1.1', 'order': '1.5.0', 'progress': '1.4.0', 'testconfig': '0.2.0', 'xdist': '3.8.0', 'syrupy': '5.3.2'}} rootdir: /home/odh/opendatahub-tests configfile: pytest.ini plugins: anyio-4.14.0, langsmith-0.9.0, asyncio-1.4.0, dependency-0.6.1, html-4.2.0, metadata-3.1.1, order-1.5.0, progress-1.4.0, testconfig-0.2.0, xdist-3.8.0, syrupy-5.3.2 asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function collecting ... collected 231 items / 221 deselected / 10 selected tests/ai_safety/evalhub/mcp/test_evalhub_mcp_deployment.py::TestEvalHubMcpDeployment::test_evalhub_mcp_pod_has_expected_containers[model_namespace0]  ---------------------------- test_evalhub_mcp_pod_has_expected_containers[model_namespace0] ---------------------------- -------------------------------------------------------- SETUP -------------------------------------------------------- 2026-07-04T18:59:34.822560Z [info ] Executing session fixture: event_loop_policy [conftest] 2026-07-04T18:59:34.824829Z [info ] Executing session fixture: _syrupy_apply_ide_patches [conftest] 2026-07-04T18:59:34.826155Z [info ] Executing session fixture: admin_client [conftest] 2026-07-04T18:59:34.826357Z [info ] Trying to get client via new_client_from_config [ocp_resources.resource] 2026-07-04T18:59:34.826507Z [info ] Executing session fixture: dsc_resource [conftest] 2026-07-04T18:59:34.827177Z [info ] kind: DataScienceCluster api version: datasciencecluster.opendatahub.io/v2 [ocp_resources.resource] 2026-07-04T18:59:34.933123Z [info ] Executing session fixture: tmp_path_factory [conftest] 2026-07-04T18:59:34.933404Z [info ] Executing session fixture: tests_tmp_dir [conftest] 2026-07-04T18:59:34.934501Z [info ] Executing session fixture: os_path_environment [conftest] 2026-07-04T18:59:34.934676Z [info ] Executing session fixture: tmpdir_factory [conftest] 2026-07-04T18:59:34.934835Z [info ] Executing session fixture: bin_directory [conftest] 2026-07-04T18:59:34.936050Z [info ] Executing session fixture: oc_binary_path [conftest] 2026-07-04T18:59:34.960273Z [info ] kind: ConsoleCLIDownload api version: console.openshift.io/v1 [ocp_resources.resource] 2026-07-04T18:59:34.988482Z [info ] All oc console cli download links: ['https://downloads-openshift-console.apps.ods-qe-psi-07.osp.rh-ods.com/amd64/linux/oc.tar'] [utilities.infra] name=utilities.infra 2026-07-04T18:59:34.988570Z [info ] Downloading archive using: url=https://downloads-openshift-console.apps.ods-qe-psi-07.osp.rh-ods.com/amd64/linux/oc.tar [utilities.infra] name=utilities.infra 2026-07-04T18:59:34.989593Z [info ] Waiting for 120 seconds [0:02:00], retry every 10 seconds. (Function: utilities.infra._download_with_retry Kwargs: {'url': 'https://downloads-openshift-console.apps.ods-qe-psi-07.osp.rh-ods.com/amd64/linux/oc.tar'}) [timeout_sampler] 2026-07-04T18:59:35.041228Z [info ] Elapsed time: 5.364418029785156e-05 [0:00:00.000054] [timeout_sampler] 2026-07-04T18:59:36.158657Z [info ] Extract the downloaded archive. [utilities.infra] name=utilities.infra 2026-07-04T18:59:36.348257Z [info ] Downloaded file: ['oc']  [utilities.infra] name=utilities.infra 2026-07-04T18:59:36.377911Z [info ] Executing session fixture: bin_directory_to_os_path [conftest] 2026-07-04T18:59:36.373598Z [info ] OC binary path: /tmp/pytest-DYLPTF8dLhUKEGPMoeFhxy/bin0/oc [tests.conftest] name=tests.conftest 2026-07-04T18:59:36.373685Z [info ] Adding /tmp/pytest-DYLPTF8dLhUKEGPMoeFhxy/bin0 to $PATH [tests.conftest] name=tests.conftest 2026-07-04T18:59:36.378417Z [info ] Executing session fixture: nodes [conftest] 2026-07-04T18:59:37.375646Z [info ] Executing session fixture: dsci_resource [conftest] 2026-07-04T18:59:37.375853Z [info ] kind: DSCInitialization api version: dscinitialization.opendatahub.io/v2 [ocp_resources.resource] 2026-07-04T18:59:37.396620Z [info ] Executing session fixture: record_testsuite_property [conftest] 2026-07-04T18:59:37.396789Z [info ] Executing session fixture: junitxml_plugin [conftest] 2026-07-04T18:59:37.396928Z [info ] Executing session fixture: cluster_sanity_scope_session [conftest] 2026-07-04T18:59:37.395870Z [info ] Check cluster sanity.  [utilities.infra] name=utilities.infra 2026-07-04T18:59:37.397203Z [info ] Verify all nodes are in a healthy condition. [ocp_utilities.infra] 2026-07-04T18:59:37.546643Z [info ] Verify all nodes are schedulable. [ocp_utilities.infra] 2026-07-04T18:59:37.652658Z [warning ] Skipping RHOAI resource checks, got --cluster-sanity-skip-rhoai-check [utilities.infra] name=utilities.infra 2026-07-04T18:59:37.655661Z [info ] Executing session fixture: autouse_fixtures [conftest] 2026-07-04T18:59:37.655793Z [info ] Executing session fixture: pytestconfig [conftest] 2026-07-04T18:59:37.655919Z [info ] Executing session fixture: teardown_resources [conftest] 2026-07-04T18:59:37.656046Z [info ] Executing class fixture: model_namespace [conftest] 2026-07-04T18:59:37.656163Z [info ] Create Namespace test-evalhub-mcp-deployment [ocp_resources Namespace] 2026-07-04T18:59:37.656323Z [info ] Posting {'apiVersion': 'v1', 'kind': 'Namespace', 'metadata': {'name': 'test-evalhub-mcp-deployment'}, 'spec': {}} [ocp_resources Namespace] 2026-07-04T18:59:37.688297Z [info ] Wait for Namespace test-evalhub-mcp-deployment status to be Active [ocp_resources Namespace] 2026-07-04T18:59:37.688459Z [info ] Waiting for 120 seconds [0:02:00], retry every 1 seconds. (Function: ocp_resources.resource.wait_for_status.lambda: self.exists) [timeout_sampler] 2026-07-04T18:59:37.710853Z [info ] Status of Namespace test-evalhub-mcp-deployment is Active [ocp_resources Namespace] 2026-07-04T18:59:37.710996Z [info ] Elapsed time: 9.989738464355469e-05 [0:00:00.000100] [timeout_sampler] 2026-07-04T18:59:37.711118Z [info ] Executing class fixture: tenant_a_namespace [conftest] 2026-07-04T18:59:37.711309Z [info ] Create Namespace test-evalhub-tenant-a-testevalhubmcpdeployment-e3072f [ocp_resources Namespace] 2026-07-04T18:59:37.711446Z [info ] Posting {'apiVersion': 'v1', 'kind': 'Namespace', 'metadata': {'name': 'test-evalhub-tenant-a-testevalhubmcpdeployment-e3072f', 'labels': {'evalhub.trustyai.opendatahub.io/tenant': 'true'}}, 'spec': {}} [ocp_resources Namespace] 2026-07-04T18:59:37.748059Z [info ] Wait for Namespace test-evalhub-tenant-a-testevalhubmcpdeployment-e3072f status to be Active [ocp_resources Namespace] 2026-07-04T18:59:37.748256Z [info ] Waiting for 120 seconds [0:02:00], retry every 1 seconds. (Function: ocp_resources.resource.wait_for_status.lambda: self.exists) [timeout_sampler] 2026-07-04T18:59:37.768713Z [info ] Status of Namespace test-evalhub-tenant-a-testevalhubmcpdeployment-e3072f is Active [ocp_resources Namespace] 2026-07-04T18:59:37.768889Z [info ] Elapsed time: 4.291534423828125e-05 [0:00:00.000043] [timeout_sampler] 2026-07-04T18:59:37.769044Z [info ] Executing class fixture: evalhub_mcp_mt_cr [conftest] 2026-07-04T18:59:37.769177Z [info ] kind: CustomResourceDefinition api version: apiextensions.k8s.io/v1 [ocp_resources.resource] 2026-07-04T18:59:37.810377Z [info ] kind: EvalHub api version: trustyai.opendatahub.io/v1alpha1 [ocp_resources.resource] 2026-07-04T18:59:37.810558Z [info ] Create EvalHub evalhub-mcp-mt  [ocp_resources EvalHub] 2026-07-04T18:59:37.810695Z [info ] Posting {'apiVersion': 'trustyai.opendatahub.io/v1alpha1', 'kind': 'EvalHub', 'metadata': {'name': 'evalhub-mcp-mt', 'namespace': 'test-evalhub-mcp-deployment'}, 'spec': {'collections': ['leaderboard-v2'], 'database': {'type': 'sqlite'}}} [ocp_resources EvalHub] 2026-07-04T18:59:37.841666Z [info ] Wait until EvalHub evalhub-mcp-mt is created [ocp_resources EvalHub] 2026-07-04T18:59:37.841839Z [info ] Waiting for 300 seconds [0:05:00], retry every 1 seconds. (Function: ocp_resources.resource.wait.lambda: self.exists) [timeout_sampler] 2026-07-04T18:59:37.854527Z [info ] Elapsed time: 5.14984130859375e-05 [0:00:00.000051] [timeout_sampler] 2026-07-04T18:59:37.854665Z [info ] Executing class fixture: evalhub_mcp_service_account [conftest] 2026-07-04T18:59:37.853611Z [info ] Waiting for ServiceAccount 'evalhub-mcp-mt-service' in namespace 'test-evalhub-mcp-deployment' [tests.ai_safety.evalhub.utils] name=tests.ai_safety.evalhub.utils 2026-07-04T18:59:37.854905Z [info ] Waiting for 120 seconds [0:02:00], retry every 10 seconds. (Function: tests.ai_safety.evalhub.utils._sa_exists) [timeout_sampler] 2026-07-04T18:59:47.885271Z [info ] ServiceAccount 'evalhub-mcp-mt-service' found in namespace 'test-evalhub-mcp-deployment' [tests.ai_safety.evalhub.utils] name=tests.ai_safety.evalhub.utils 2026-07-04T18:59:47.886791Z [info ] Elapsed time: 10.017558097839355 [0:00:10.017558] [timeout_sampler] 2026-07-04T18:59:47.886978Z [info ] Executing class fixture: evalhub_mcp_mt_cr_with_auth [conftest] 2026-07-04T18:59:47.887144Z [info ] Running oc create token -n test-evalhub-mcp-deployment evalhub-mcp-mt-service command [pyhelper_utils.shell] 2026-07-04T18:59:48.030377Z [info ] Create Secret evalhub-mcp-mt-mcp-token [ocp_resources Secret] 2026-07-04T18:59:48.030626Z [info ] Posting {'apiVersion': 'v1', 'kind': 'Secret', 'metadata': {'name': 'evalhub-mcp-mt-mcp-token', 'namespace': 'test-evalhub-mcp-deployment'}, 'stringData': '*******'} [ocp_resources Secret] 2026-07-04T18:59:48.061058Z [info ] Update EvalHub evalhub-mcp-mt: {'metadata': {'name': 'evalhub-mcp-mt', 'namespace': 'test-evalhub-mcp-deployment'}, 'spec': {'mcp': {'enabled': True, 'replicas': 1, 'authSecret': 'evalhub-mcp-mt-mcp-token', 'env': [{'name': 'EVALHUB_TENANT', 'value': 'test-evalhub-tenant-a-testevalhubmcpdeployment-e3072f'}]}}} [ocp_resources EvalHub] 2026-07-04T18:59:48.078504Z [info ] Wait until EvalHub evalhub-mcp-mt is created [ocp_resources EvalHub] 2026-07-04T18:59:48.078675Z [info ] Waiting for 300 seconds [0:05:00], retry every 1 seconds. (Function: ocp_resources.resource.wait.lambda: self.exists) [timeout_sampler] 2026-07-04T18:59:48.092326Z [info ] Elapsed time: 6.4849853515625e-05 [0:00:00.000065] [timeout_sampler] 2026-07-04T18:59:48.092765Z [info ] Executing class fixture: evalhub_mcp_mt_deployment [conftest] 2026-07-04T18:59:48.092976Z [info ] kind: Deployment api version: apps/v1 [ocp_resources.resource] 2026-07-04T18:59:48.093106Z [info ] Wait for Deployment evalhub-mcp-mt-mcp to be deployed: True [ocp_resources Deployment] 2026-07-04T18:59:48.093250Z [info ] Waiting for 300 seconds [0:05:00], retry every 1 seconds. (Function: ocp_resources.deployment.wait_for_replicas.lambda: self.exists) [timeout_sampler] 2026-07-04T18:59:49.120837Z [info ] Elapsed time: 1.013948917388916 [0:00:01.013949] [timeout_sampler] 2026-07-04T18:59:49.121142Z [info ] Waiting for 298.971563577652 seconds [0:04:58.971564], retry every 1 seconds. (Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance) [timeout_sampler] ERROR _ ERROR at setup of TestEvalHubMcpDeployment.test_evalhub_mcp_pod_has_expected_containers[model_namespace0] _ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1232, in execute result: FixtureValue = ihook.pytest_fixture_setup( ~~~~~~~~~~~~~~~~~~~~~~~~~~^ fixturedef=self, request=request ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pytest_asyncio/plugin.py", line 924, in pytest_fixture_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/setuponly.py", line 36, in pytest_fixture_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1328, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1005, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/mcp/conftest.py", line 222, in evalhub_mcp_mt_deployment deployment.wait_for_replicas(timeout=Timeout.TIMEOUT_5MIN) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/deployment.py", line 371, in wait_for_replicas for sample in samples: ^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/timeout_sampler/__init__.py", line 182, in __iter__ raise TimeoutExpiredError(self._get_exception_log(exp=last_exp), last_exp=last_exp) timeout_sampler.TimeoutExpiredError: Timed Out: 298.971563577652 Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance Last exception: N/A. _ 1 of 10 completed, 0 Pass, 0 Fail, 0 Skip, 0 XPass, 0 XFail, 1 Error, 0 ReRun _ tests/ai_safety/evalhub/mcp/test_evalhub_mcp_deployment.py::TestEvalHubMcpDeployment::test_evalhub_mcp_pod_labels_match_operator[model_namespace0]  TEST: TestEvalHubMcpDeployment.test_evalhub_mcp_pod_has_expected_containers[model_namespace0] [setup] STATUS: ERROR 2026-07-04T19:04:48.860002Z [error ] Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1232, in execute result: FixtureValue = ihook.pytest_fixture_setup( ~~~~~~~~~~~~~~~~~~~~~~~~~~^ fixturedef=self, request=request ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pytest_asyncio/plugin.py", line 924, in pytest_fixture_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/setuponly.py", line 36, in pytest_fixture_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1328, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1005, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/mcp/conftest.py", line 222, in evalhub_mcp_mt_deployment deployment.wait_for_replicas(timeout=Timeout.TIMEOUT_5MIN) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/deployment.py", line 371, in wait_for_replicas for sample in samples: ^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/timeout_sampler/__init__.py", line 182, in __iter__ raise TimeoutExpiredError(self._get_exception_log(exp=last_exp), last_exp=last_exp) timeout_sampler.TimeoutExpiredError: Timed Out: 298.971563577652 Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance Last exception: N/A. [conftest] ------------------------------------------------------- TEARDOWN -------------------------------------------------------  ----------------------------- test_evalhub_mcp_pod_labels_match_operator[model_namespace0] ----------------------------- -------------------------------------------------------- SETUP -------------------------------------------------------- ERROR _ ERROR at setup of TestEvalHubMcpDeployment.test_evalhub_mcp_pod_labels_match_operator[model_namespace0] _ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1204, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1328, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1005, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/mcp/conftest.py", line 222, in evalhub_mcp_mt_deployment deployment.wait_for_replicas(timeout=Timeout.TIMEOUT_5MIN) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/deployment.py", line 371, in wait_for_replicas for sample in samples: ^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/timeout_sampler/__init__.py", line 182, in __iter__ raise TimeoutExpiredError(self._get_exception_log(exp=last_exp), last_exp=last_exp) timeout_sampler.TimeoutExpiredError: Timed Out: 298.971563577652 Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance Last exception: N/A. _ 2 of 10 completed, 0 Pass, 0 Fail, 0 Skip, 0 XPass, 0 XFail, 2 Error, 0 ReRun _ tests/ai_safety/evalhub/mcp/test_evalhub_mcp_deployment.py::TestEvalHubMcpDeployment::test_evalhub_mcp_configmap_mounted_in_pod[model_namespace0]  TEST: TestEvalHubMcpDeployment.test_evalhub_mcp_pod_labels_match_operator[model_namespace0] [setup] STATUS: ERROR 2026-07-04T19:04:48.864102Z [error ] Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1204, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1328, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1005, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/mcp/conftest.py", line 222, in evalhub_mcp_mt_deployment deployment.wait_for_replicas(timeout=Timeout.TIMEOUT_5MIN) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/deployment.py", line 371, in wait_for_replicas for sample in samples: ^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/timeout_sampler/__init__.py", line 182, in __iter__ raise TimeoutExpiredError(self._get_exception_log(exp=last_exp), last_exp=last_exp) timeout_sampler.TimeoutExpiredError: Timed Out: 298.971563577652 Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance Last exception: N/A. [conftest] ------------------------------------------------------- TEARDOWN -------------------------------------------------------  ----------------------------- test_evalhub_mcp_configmap_mounted_in_pod[model_namespace0] ----------------------------- -------------------------------------------------------- SETUP -------------------------------------------------------- ERROR TEST: TestEvalHubMcpDeployment.test_evalhub_mcp_configmap_mounted_in_pod[model_namespace0] [setup] STATUS: ERROR 2026-07-04T19:04:48.867323Z [error ] Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1204, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1328, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1005, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/mcp/conftest.py", line 222, in evalhub_mcp_mt_deployment deployment.wait_for_replicas(timeout=Timeout.TIMEOUT_5MIN) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/deployment.py", line 371, in wait_for_replicas for sample in samples: ^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/timeout_sampler/__init__.py", line 182, in __iter__ raise TimeoutExpiredError(self._get_exception_log(exp=last_exp), last_exp=last_exp) timeout_sampler.TimeoutExpiredError: Timed Out: 298.971563577652 Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance Last exception: N/A. [conftest] ------------------------------------------------------- TEARDOWN ------------------------------------------------------- 2026-07-04T19:04:48.867584Z [info ] Delete Secret evalhub-mcp-mt-mcp-token [ocp_resources Secret] 2026-07-04T19:04:48.896512Z [info ] Deleting {'kind': 'Secret', 'apiVersion': 'v1', 'metadata': {'name': 'evalhub-mcp-mt-mcp-token', 'namespace': 'test-evalhub-mcp-deployment', 'uid': 'c542f3af-76eb-4f01-a7a2-560893a8310f', 'resourceVersion': '378380541', 'creationTimestamp': '2026-07-04T18:59:48Z', 'managedFields': [{'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2026-07-04T18:59:48Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:data': {'.': {}, 'f:token': {}}, 'f:type': {}}}]}, 'data': '*******', 'type': 'Opaque'} [ocp_resources Secret] 2026-07-04T19:04:48.915763Z [info ] Wait until Secret evalhub-mcp-mt-mcp-token is deleted [ocp_resources Secret] 2026-07-04T19:04:48.916152Z [info ] Waiting for 240 seconds [0:04:00], retry every 1 seconds. (Function: ocp_resources.resource.wait_deleted.lambda: self.exists) [timeout_sampler] 2026-07-04T19:04:48.929830Z [info ] Elapsed time: 0.00010657310485839844 [0:00:00.000107] [timeout_sampler] 2026-07-04T19:04:48.931001Z [info ] Delete EvalHub evalhub-mcp-mt  [ocp_resources EvalHub] 2026-07-04T19:04:48.962145Z [info ] Deleting {'apiVersion': 'trustyai.opendatahub.io/v1alpha1', 'kind': 'EvalHub', 'metadata': {'creationTimestamp': '2026-07-04T18:59:37Z', 'finalizers': ['trustyai.opendatahub.io/evalhub-finalizer'], 'generation': 2, 'managedFields': [{'apiVersion': 'trustyai.opendatahub.io/v1alpha1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:finalizers': {'.': {}, 'v:"trustyai.opendatahub.io/evalhub-finalizer"': {}}}}, 'manager': 'manager', 'operation': 'Update', 'time': '2026-07-04T18:59:37Z'}, {'apiVersion': 'trustyai.opendatahub.io/v1alpha1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:spec': {'.': {}, 'f:collections': {}, 'f:database': {'.': {}, 'f:maxIdleConns': {}, 'f:maxOpenConns': {}, 'f:type': {}}, 'f:mcp': {'.': {}, 'f:authSecret': {}, 'f:enabled': {}, 'f:env': {}, 'f:replicas': {}, 'f:transport': {}}, 'f:providers': {}, 'f:replicas': {}}}, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': '2026-07-04T18:59:48Z'}, {'apiVersion': 'trustyai.opendatahub.io/v1alpha1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:status': {'.': {}, 'f:activeCollections': {}, 'f:activeProviders': {}, 'f:conditions': {}, 'f:lastUpdateTime': {}, 'f:mcp': {'.': {}, 'f:conditions': {}, 'f:phase': {}, 'f:ready': {}}, 'f:phase': {}, 'f:ready': {}, 'f:readyReplicas': {}, 'f:replicas': {}, 'f:url': {}}}, 'manager': 'manager', 'operation': 'Update', 'subresource': 'status', 'time': '2026-07-04T19:04:48Z'}], 'name': 'evalhub-mcp-mt', 'namespace': 'test-evalhub-mcp-deployment', 'resourceVersion': '378385781', 'uid': '1868dd33-3383-4b92-be51-8207d8d26392'}, 'spec': {'collections': ['leaderboard-v2'], 'database': {'maxIdleConns': 5, 'maxOpenConns': 25, 'type': 'sqlite'}, 'mcp': {'authSecret': 'evalhub-mcp-mt-mcp-token', 'enabled': True, 'env': [{'name': 'EVALHUB_TENANT', 'value': 'test-evalhub-tenant-a-testevalhubmcpdeployment-e3072f'}], 'replicas': 1, 'transport': 'http'}, 'providers': ['garak', 'garak-kfp', 'lm-evaluation-harness'], 'replicas': 1}, 'status': {'activeCollections': ['leaderboard-v2'], 'activeProviders': ['garak', 'garak-kfp', 'lm-evaluation-harness'], 'conditions': [{'lastTransitionTime': '2026-07-04T19:04:48Z', 'message': 'All replicas are ready', 'reason': 'DeploymentReady', 'status': 'True', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-04T19:04:48Z', 'reason': 'MonitoringReady', 'status': 'False', 'type': 'MonitoringDegraded'}], 'lastUpdateTime': '2026-07-04T19:04:48Z', 'mcp': {'conditions': [{'lastTransitionTime': '2026-07-04T18:59:49Z', 'message': 'Route reconciliation succeeded', 'observedGeneration': 2, 'reason': 'MCPRouteReady', 'status': 'True', 'type': 'RouteReady'}, {'lastTransitionTime': '2026-07-04T18:59:49Z', 'message': '', 'observedGeneration': 2, 'reason': 'ReconcileComplete', 'status': 'True', 'type': 'Reconciled'}, {'lastTransitionTime': '2026-07-04T18:59:49Z', 'message': 'Waiting for MCP deployment (0/1 replicas ready)', 'observedGeneration': 2, 'reason': 'DeploymentNotReady', 'status': 'False', 'type': 'Ready'}], 'phase': 'Pending', 'ready': False}, 'phase': 'Ready', 'ready': 'True', 'readyReplicas': 1, 'replicas': 1, 'url': 'https://evalhub-mcp-mt.test-evalhub-mcp-deployment.svc.cluster.local:8443'}} [ocp_resources EvalHub] 2026-07-04T19:04:48.981332Z [info ] Wait until EvalHub evalhub-mcp-mt is deleted [ocp_resources EvalHub] 2026-07-04T19:04:48.981474Z [info ] Waiting for 240 seconds [0:04:00], retry every 1 seconds. (Function: ocp_resources.resource.wait_deleted.lambda: self.exists) [timeout_sampler] 2026-07-04T19:04:50.010025Z [info ] Elapsed time: 1.013063669204712 [0:00:01.013064] [timeout_sampler] 2026-07-04T19:04:50.023168Z [info ] Delete Namespace test-evalhub-tenant-a-testevalhubmcpdeployment-e3072f [ocp_resources Namespace] 2026-07-04T19:04:50.053052Z [info ] Deleting {'kind': 'Namespace', 'apiVersion': 'v1', 'metadata': {'name': 'test-evalhub-tenant-a-testevalhubmcpdeployment-e3072f', 'uid': '585ab969-5b0c-46ae-87ad-47b5c866a881', 'resourceVersion': '378380243', 'creationTimestamp': '2026-07-04T18:59:37Z', 'labels': {'evalhub.trustyai.opendatahub.io/tenant': 'true', 'kubernetes.io/metadata.name': 'test-evalhub-tenant-a-testevalhubmcpdeployment-e3072f', 'pod-security.kubernetes.io/audit': 'restricted', 'pod-security.kubernetes.io/audit-version': 'latest', 'pod-security.kubernetes.io/warn': 'restricted', 'pod-security.kubernetes.io/warn-version': 'latest'}, 'annotations': {'openshift.io/sa.scc.mcs': 's0:c34,c24', 'openshift.io/sa.scc.supplemental-groups': '1001170000/10000', 'openshift.io/sa.scc.uid-range': '1001170000/10000', 'security.openshift.io/MinimallySufficientPodSecurityStandard': 'restricted'}, 'managedFields': [{'manager': 'pod-security-admission-label-synchronization-controller', 'operation': 'Apply', 'apiVersion': 'v1', 'time': '2026-07-04T18:59:37Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'f:security.openshift.io/MinimallySufficientPodSecurityStandard': {}}, 'f:labels': {'f:pod-security.kubernetes.io/audit': {}, 'f:pod-security.kubernetes.io/audit-version': {}, 'f:pod-security.kubernetes.io/warn': {}, 'f:pod-security.kubernetes.io/warn-version': {}}}}}, {'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2026-07-04T18:59:37Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:labels': {'.': {}, 'f:evalhub.trustyai.opendatahub.io/tenant': {}, 'f:kubernetes.io/metadata.name': {}}}}}, {'manager': 'cluster-policy-controller', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2026-07-04T18:59:37Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'.': {}, 'f:openshift.io/sa.scc.mcs': {}, 'f:openshift.io/sa.scc.supplemental-groups': {}, 'f:openshift.io/sa.scc.uid-range': {}}}}}]}, 'spec': {'finalizers': ['kubernetes']}, 'status': {'phase': 'Active'}} [ocp_resources Namespace] 2026-07-04T19:04:50.071284Z [info ] Wait until Namespace test-evalhub-tenant-a-testevalhubmcpdeployment-e3072f is deleted [ocp_resources Namespace] 2026-07-04T19:04:50.071485Z [info ] Waiting for 240 seconds [0:04:00], retry every 1 seconds. (Function: ocp_resources.resource.wait_deleted.lambda: self.exists) [timeout_sampler] 2026-07-04T19:04:58.211792Z [info ] Elapsed time: 8.125520467758179 [0:00:08.125520] [timeout_sampler] 2026-07-04T19:04:58.289511Z [info ] Delete Namespace test-evalhub-mcp-deployment [ocp_resources Namespace] 2026-07-04T19:04:58.318720Z [info ] Deleting {'kind': 'Namespace', 'apiVersion': 'v1', 'metadata': {'name': 'test-evalhub-mcp-deployment', 'uid': '9d5b5723-5a23-4656-bdeb-fab99b72df9c', 'resourceVersion': '378380216', 'creationTimestamp': '2026-07-04T18:59:37Z', 'labels': {'kubernetes.io/metadata.name': 'test-evalhub-mcp-deployment', 'pod-security.kubernetes.io/audit': 'restricted', 'pod-security.kubernetes.io/audit-version': 'latest', 'pod-security.kubernetes.io/warn': 'restricted', 'pod-security.kubernetes.io/warn-version': 'latest'}, 'annotations': {'openshift.io/sa.scc.mcs': 's0:c34,c19', 'openshift.io/sa.scc.supplemental-groups': '1001160000/10000', 'openshift.io/sa.scc.uid-range': '1001160000/10000', 'security.openshift.io/MinimallySufficientPodSecurityStandard': 'restricted'}, 'managedFields': [{'manager': 'pod-security-admission-label-synchronization-controller', 'operation': 'Apply', 'apiVersion': 'v1', 'time': '2026-07-04T18:59:37Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'f:security.openshift.io/MinimallySufficientPodSecurityStandard': {}}, 'f:labels': {'f:pod-security.kubernetes.io/audit': {}, 'f:pod-security.kubernetes.io/audit-version': {}, 'f:pod-security.kubernetes.io/warn': {}, 'f:pod-security.kubernetes.io/warn-version': {}}}}}, {'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2026-07-04T18:59:37Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:labels': {'.': {}, 'f:kubernetes.io/metadata.name': {}}}}}, {'manager': 'cluster-policy-controller', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2026-07-04T18:59:37Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'.': {}, 'f:openshift.io/sa.scc.mcs': {}, 'f:openshift.io/sa.scc.supplemental-groups': {}, 'f:openshift.io/sa.scc.uid-range': {}}}}}]}, 'spec': {'finalizers': ['kubernetes']}, 'status': {'phase': 'Active'}} [ocp_resources Namespace] 2026-07-04T19:04:58.335395Z [info ] Wait until Namespace test-evalhub-mcp-deployment is deleted [ocp_resources Namespace] 2026-07-04T19:04:58.335573Z [info ] Waiting for 240 seconds [0:04:00], retry every 1 seconds. (Function: ocp_resources.resource.wait_deleted.lambda: self.exists) [timeout_sampler] _ ERROR at setup of TestEvalHubMcpDeployment.test_evalhub_mcp_configmap_mounted_in_pod[model_namespace0] _ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1204, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1328, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1005, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/mcp/conftest.py", line 222, in evalhub_mcp_mt_deployment deployment.wait_for_replicas(timeout=Timeout.TIMEOUT_5MIN) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/deployment.py", line 371, in wait_for_replicas for sample in samples: ^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/timeout_sampler/__init__.py", line 182, in __iter__ raise TimeoutExpiredError(self._get_exception_log(exp=last_exp), last_exp=last_exp) timeout_sampler.TimeoutExpiredError: Timed Out: 298.971563577652 Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance Last exception: N/A. _ 3 of 10 completed, 0 Pass, 0 Fail, 0 Skip, 0 XPass, 0 XFail, 3 Error, 0 ReRun _ tests/ai_safety/evalhub/mcp/test_evalhub_mcp_route.py::TestEvalHubMcpRoute::test_evalhub_mcp_service_exposes_https_port[model_namespace0] 2026-07-04T19:05:30.895989Z [info ] Elapsed time: 32.52609658241272 [0:00:32.526097] [timeout_sampler]  ---------------------------- test_evalhub_mcp_service_exposes_https_port[model_namespace0] ---------------------------- -------------------------------------------------------- SETUP -------------------------------------------------------- 2026-07-04T19:05:30.896522Z [info ] Executing class fixture: model_namespace [conftest] 2026-07-04T19:05:30.896630Z [info ] Create Namespace test-evalhub-mcp-route [ocp_resources Namespace] 2026-07-04T19:05:30.896734Z [info ] Posting {'apiVersion': 'v1', 'kind': 'Namespace', 'metadata': {'name': 'test-evalhub-mcp-route'}, 'spec': {}} [ocp_resources Namespace] 2026-07-04T19:05:30.963049Z [info ] Wait for Namespace test-evalhub-mcp-route status to be Active [ocp_resources Namespace] 2026-07-04T19:05:30.963295Z [info ] Waiting for 120 seconds [0:02:00], retry every 1 seconds. (Function: ocp_resources.resource.wait_for_status.lambda: self.exists) [timeout_sampler] 2026-07-04T19:05:30.983452Z [info ] Status of Namespace test-evalhub-mcp-route is Active [ocp_resources Namespace] 2026-07-04T19:05:30.983628Z [info ] Elapsed time: 3.5762786865234375e-05 [0:00:00.000036] [timeout_sampler] 2026-07-04T19:05:30.983768Z [info ] Executing class fixture: tenant_a_namespace [conftest] 2026-07-04T19:05:30.983906Z [info ] Create Namespace test-evalhub-tenant-a-testevalhubmcproute-777de6 [ocp_resources Namespace] 2026-07-04T19:05:30.984072Z [info ] Posting {'apiVersion': 'v1', 'kind': 'Namespace', 'metadata': {'name': 'test-evalhub-tenant-a-testevalhubmcproute-777de6', 'labels': {'evalhub.trustyai.opendatahub.io/tenant': 'true'}}, 'spec': {}} [ocp_resources Namespace] 2026-07-04T19:05:31.018667Z [info ] Wait for Namespace test-evalhub-tenant-a-testevalhubmcproute-777de6 status to be Active [ocp_resources Namespace] 2026-07-04T19:05:31.018832Z [info ] Waiting for 120 seconds [0:02:00], retry every 1 seconds. (Function: ocp_resources.resource.wait_for_status.lambda: self.exists) [timeout_sampler] 2026-07-04T19:05:31.033328Z [info ] Status of Namespace test-evalhub-tenant-a-testevalhubmcproute-777de6 is Active [ocp_resources Namespace] 2026-07-04T19:05:31.033465Z [info ] Elapsed time: 8.702278137207031e-05 [0:00:00.000087] [timeout_sampler] 2026-07-04T19:05:31.033566Z [info ] Executing class fixture: evalhub_mcp_mt_cr [conftest] 2026-07-04T19:05:31.033668Z [info ] kind: CustomResourceDefinition api version: apiextensions.k8s.io/v1 [ocp_resources.resource] 2026-07-04T19:05:31.059144Z [info ] kind: EvalHub api version: trustyai.opendatahub.io/v1alpha1 [ocp_resources.resource] 2026-07-04T19:05:31.059382Z [info ] Create EvalHub evalhub-mcp-mt  [ocp_resources EvalHub] 2026-07-04T19:05:31.059506Z [info ] Posting {'apiVersion': 'trustyai.opendatahub.io/v1alpha1', 'kind': 'EvalHub', 'metadata': {'name': 'evalhub-mcp-mt', 'namespace': 'test-evalhub-mcp-route'}, 'spec': {'collections': ['leaderboard-v2'], 'database': {'type': 'sqlite'}}} [ocp_resources EvalHub] 2026-07-04T19:05:31.103959Z [info ] Wait until EvalHub evalhub-mcp-mt is created [ocp_resources EvalHub] 2026-07-04T19:05:31.104095Z [info ] Waiting for 300 seconds [0:05:00], retry every 1 seconds. (Function: ocp_resources.resource.wait.lambda: self.exists) [timeout_sampler] 2026-07-04T19:05:31.121046Z [info ] Elapsed time: 3.4809112548828125e-05 [0:00:00.000035] [timeout_sampler] 2026-07-04T19:05:31.121217Z [info ] Executing class fixture: evalhub_mcp_service_account [conftest] 2026-07-04T19:05:31.120370Z [info ] Waiting for ServiceAccount 'evalhub-mcp-mt-service' in namespace 'test-evalhub-mcp-route' [tests.ai_safety.evalhub.utils] name=tests.ai_safety.evalhub.utils 2026-07-04T19:05:31.121486Z [info ] Waiting for 120 seconds [0:02:00], retry every 10 seconds. (Function: tests.ai_safety.evalhub.utils._sa_exists) [timeout_sampler] 2026-07-04T19:05:41.154103Z [info ] ServiceAccount 'evalhub-mcp-mt-service' found in namespace 'test-evalhub-mcp-route' [tests.ai_safety.evalhub.utils] name=tests.ai_safety.evalhub.utils 2026-07-04T19:05:41.155799Z [info ] Elapsed time: 10.018970251083374 [0:00:10.018970] [timeout_sampler] 2026-07-04T19:05:41.155948Z [info ] Executing class fixture: evalhub_mcp_mt_cr_with_auth [conftest] 2026-07-04T19:05:41.156076Z [info ] Running oc create token -n test-evalhub-mcp-route evalhub-mcp-mt-service command [pyhelper_utils.shell] 2026-07-04T19:05:41.328496Z [info ] Create Secret evalhub-mcp-mt-mcp-token [ocp_resources Secret] 2026-07-04T19:05:41.328746Z [info ] Posting {'apiVersion': 'v1', 'kind': 'Secret', 'metadata': {'name': 'evalhub-mcp-mt-mcp-token', 'namespace': 'test-evalhub-mcp-route'}, 'stringData': '*******'} [ocp_resources Secret] 2026-07-04T19:05:41.372666Z [info ] Update EvalHub evalhub-mcp-mt: {'metadata': {'name': 'evalhub-mcp-mt', 'namespace': 'test-evalhub-mcp-route'}, 'spec': {'mcp': {'enabled': True, 'replicas': 1, 'authSecret': 'evalhub-mcp-mt-mcp-token', 'env': [{'name': 'EVALHUB_TENANT', 'value': 'test-evalhub-tenant-a-testevalhubmcproute-777de6'}]}}} [ocp_resources EvalHub] 2026-07-04T19:05:41.391677Z [info ] Wait until EvalHub evalhub-mcp-mt is created [ocp_resources EvalHub] 2026-07-04T19:05:41.391887Z [info ] Waiting for 300 seconds [0:05:00], retry every 1 seconds. (Function: ocp_resources.resource.wait.lambda: self.exists) [timeout_sampler] 2026-07-04T19:05:41.405164Z [info ] Elapsed time: 4.267692565917969e-05 [0:00:00.000043] [timeout_sampler] 2026-07-04T19:05:41.405392Z [info ] Executing class fixture: evalhub_mcp_mt_deployment [conftest] 2026-07-04T19:05:41.405526Z [info ] kind: Deployment api version: apps/v1 [ocp_resources.resource] 2026-07-04T19:05:41.405646Z [info ] Waiting for 300 seconds [0:05:00], retry every 1 seconds. (Function: ocp_resources.deployment.wait_for_replicas.lambda: self.exists) [timeout_sampler] 2026-07-04T19:05:43.448812Z [info ] Elapsed time: 2.028477907180786 [0:00:02.028478] [timeout_sampler] 2026-07-04T19:05:43.448995Z [info ] Waiting for 297.95665740966797 seconds [0:04:57.956657], retry every 1 seconds. (Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance) [timeout_sampler] ERROR _ ERROR at setup of TestEvalHubMcpRoute.test_evalhub_mcp_service_exposes_https_port[model_namespace0] _ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1232, in execute result: FixtureValue = ihook.pytest_fixture_setup( ~~~~~~~~~~~~~~~~~~~~~~~~~~^ fixturedef=self, request=request ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pytest_asyncio/plugin.py", line 924, in pytest_fixture_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/setuponly.py", line 36, in pytest_fixture_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1328, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1005, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/mcp/conftest.py", line 222, in evalhub_mcp_mt_deployment deployment.wait_for_replicas(timeout=Timeout.TIMEOUT_5MIN) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/deployment.py", line 371, in wait_for_replicas for sample in samples: ^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/timeout_sampler/__init__.py", line 182, in __iter__ raise TimeoutExpiredError(self._get_exception_log(exp=last_exp), last_exp=last_exp) timeout_sampler.TimeoutExpiredError: Timed Out: 297.95665740966797 Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance Last exception: N/A. _ 4 of 10 completed, 0 Pass, 0 Fail, 0 Skip, 0 XPass, 0 XFail, 4 Error, 0 ReRun _ tests/ai_safety/evalhub/mcp/test_evalhub_mcp_route.py::TestEvalHubMcpRoute::test_evalhub_mcp_route_targets_service[model_namespace0]  TEST: TestEvalHubMcpRoute.test_evalhub_mcp_service_exposes_https_port[model_namespace0] [setup] STATUS: ERROR 2026-07-04T19:10:42.263062Z [error ] Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1232, in execute result: FixtureValue = ihook.pytest_fixture_setup( ~~~~~~~~~~~~~~~~~~~~~~~~~~^ fixturedef=self, request=request ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pytest_asyncio/plugin.py", line 924, in pytest_fixture_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/setuponly.py", line 36, in pytest_fixture_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1328, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1005, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/mcp/conftest.py", line 222, in evalhub_mcp_mt_deployment deployment.wait_for_replicas(timeout=Timeout.TIMEOUT_5MIN) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/deployment.py", line 371, in wait_for_replicas for sample in samples: ^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/timeout_sampler/__init__.py", line 182, in __iter__ raise TimeoutExpiredError(self._get_exception_log(exp=last_exp), last_exp=last_exp) timeout_sampler.TimeoutExpiredError: Timed Out: 297.95665740966797 Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance Last exception: N/A. [conftest] ------------------------------------------------------- TEARDOWN -------------------------------------------------------  ------------------------------- test_evalhub_mcp_route_targets_service[model_namespace0] ------------------------------- -------------------------------------------------------- SETUP -------------------------------------------------------- ERROR _ ERROR at setup of TestEvalHubMcpRoute.test_evalhub_mcp_route_targets_service[model_namespace0] _ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1181, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1204, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1328, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1005, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/mcp/conftest.py", line 222, in evalhub_mcp_mt_deployment deployment.wait_for_replicas(timeout=Timeout.TIMEOUT_5MIN) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/deployment.py", line 371, in wait_for_replicas for sample in samples: ^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/timeout_sampler/__init__.py", line 182, in __iter__ raise TimeoutExpiredError(self._get_exception_log(exp=last_exp), last_exp=last_exp) timeout_sampler.TimeoutExpiredError: Timed Out: 297.95665740966797 Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance Last exception: N/A. _ 5 of 10 completed, 0 Pass, 0 Fail, 0 Skip, 0 XPass, 0 XFail, 5 Error, 0 ReRun _ tests/ai_safety/evalhub/mcp/test_evalhub_mcp_route.py::TestEvalHubMcpRoute::test_evalhub_mcp_configmap_exists[model_namespace0]  TEST: TestEvalHubMcpRoute.test_evalhub_mcp_route_targets_service[model_namespace0] [setup] STATUS: ERROR 2026-07-04T19:10:42.268679Z [error ] Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1181, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1204, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1328, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1005, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/mcp/conftest.py", line 222, in evalhub_mcp_mt_deployment deployment.wait_for_replicas(timeout=Timeout.TIMEOUT_5MIN) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/deployment.py", line 371, in wait_for_replicas for sample in samples: ^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/timeout_sampler/__init__.py", line 182, in __iter__ raise TimeoutExpiredError(self._get_exception_log(exp=last_exp), last_exp=last_exp) timeout_sampler.TimeoutExpiredError: Timed Out: 297.95665740966797 Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance Last exception: N/A. [conftest] ------------------------------------------------------- TEARDOWN -------------------------------------------------------  --------------------------------- test_evalhub_mcp_configmap_exists[model_namespace0] --------------------------------- -------------------------------------------------------- SETUP -------------------------------------------------------- --------------------------------------------------------- CALL --------------------------------------------------------- FAILED ___ TestEvalHubMcpRoute.test_evalhub_mcp_configmap_exists[model_namespace0] ____ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 184, in pytest_runtest_call item.runtest() ~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1707, in runtest self.ihook.pytest_pyfunc_call(pyfuncitem=self) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 167, in pytest_pyfunc_call result = testfunction(**testargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/mcp/test_evalhub_mcp_route.py", line 86, in test_evalhub_mcp_configmap_exists assert "config.yaml" in data, f"Expected config.yaml in {configmap_name}" AssertionError: Expected config.yaml in evalhub-mcp-mt-mcp-config assert 'config.yaml' in {'auth.yaml': 'authorization:\n'\n ' endpoints:\n'\n ' - path: /*\n'\n ' mappings:\n'\n ' - resources:\n'\n ' - resourceAttributes:\n'\n ' namespace: "test-evalhub-mcp-route"\n'\n ' apiGroup: trustyai.opendatahub.io\n'\n ' resource: evalhubs\n'\n ' subresource: proxy\n'\n ' name: "evalhub-mcp-mt"\n'\n ' verb: get\n'\n ' - resourceAttributes:\n'\n ' namespace: "test-evalhub-mcp-route"\n'\n ' apiGroup: trustyai.opendatahub.io\n'\n ' resource: evalhubs\n'\n ' subresource: proxy\n'\n ' name: "evalhub-mcp-mt"\n'\n ' verb: create\n',\n 'config.yaml': 'base_url: '\n 'https://evalhub-mcp-mt.test-evalhub-mcp-route.svc.cluster.local:8443\n'\n 'ca_cert_path: /etc/evalhub-mcp/ca/service-ca.crt\n'\n 'host: 127.0.0.1\n'\n 'port: 8445\n'\n 'transport: http\n'} _ 6 of 10 completed, 0 Pass, 1 Fail, 0 Skip, 0 XPass, 0 XFail, 5 Error, 0 ReRun _ tests/ai_safety/evalhub/mcp/test_evalhub_mcp_route.py::TestEvalHubMcpRoute::test_evalhub_mcp_auth_secret_exists[model_namespace0]  TEST: TestEvalHubMcpRoute.test_evalhub_mcp_configmap_exists[model_namespace0] STATUS: FAILED 2026-07-04T19:10:42.299121Z [error ] Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 184, in pytest_runtest_call item.runtest() ~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1707, in runtest self.ihook.pytest_pyfunc_call(pyfuncitem=self) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 167, in pytest_pyfunc_call result = testfunction(**testargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/mcp/test_evalhub_mcp_route.py", line 86, in test_evalhub_mcp_configmap_exists assert "config.yaml" in data, f"Expected config.yaml in {configmap_name}" AssertionError: Expected config.yaml in evalhub-mcp-mt-mcp-config assert 'config.yaml' in {'auth.yaml': 'authorization:\n'\n ' endpoints:\n'\n ' - path: /*\n'\n ' mappings:\n'\n ' - resources:\n'\n ' - resourceAttributes:\n'\n ' namespace: "test-evalhub-mcp-route"\n'\n ' apiGroup: trustyai.opendatahub.io\n'\n ' resource: evalhubs\n'\n ' subresource: proxy\n'\n ' name: "evalhub-mcp-mt"\n'\n ' verb: get\n'\n ' - resourceAttributes:\n'\n ' namespace: "test-evalhub-mcp-route"\n'\n ' apiGroup: trustyai.opendatahub.io\n'\n ' resource: evalhubs\n'\n ' subresource: proxy\n'\n ' name: "evalhub-mcp-mt"\n'\n ' verb: create\n',\n 'config.yaml': 'base_url: '\n 'https://evalhub-mcp-mt.test-evalhub-mcp-route.svc.cluster.local:8443\n'\n 'ca_cert_path: /etc/evalhub-mcp/ca/service-ca.crt\n'\n 'host: 127.0.0.1\n'\n 'port: 8445\n'\n 'transport: http\n'} [conftest] ------------------------------------------------------- TEARDOWN -------------------------------------------------------  -------------------------------- test_evalhub_mcp_auth_secret_exists[model_namespace0] -------------------------------- -------------------------------------------------------- SETUP -------------------------------------------------------- --------------------------------------------------------- CALL --------------------------------------------------------- PASSED TEST: TestEvalHubMcpRoute.test_evalhub_mcp_auth_secret_exists[model_namespace0] STATUS: PASSED ------------------------------------------------------- TEARDOWN ------------------------------------------------------- 2026-07-04T19:10:42.325716Z [info ] Delete Secret evalhub-mcp-mt-mcp-token [ocp_resources Secret] 2026-07-04T19:10:42.356136Z [info ] Deleting {'kind': 'Secret', 'apiVersion': 'v1', 'metadata': {'name': 'evalhub-mcp-mt-mcp-token', 'namespace': 'test-evalhub-mcp-route', 'uid': 'fbcafbbd-f795-434d-ab2a-826c8ab8d43a', 'resourceVersion': '378386987', 'creationTimestamp': '2026-07-04T19:05:41Z', 'managedFields': [{'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2026-07-04T19:05:41Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:data': {'.': {}, 'f:token': {}}, 'f:type': {}}}]}, 'data': '*******', 'type': 'Opaque'} [ocp_resources Secret] 2026-07-04T19:10:42.372605Z [info ] Wait until Secret evalhub-mcp-mt-mcp-token is deleted [ocp_resources Secret] 2026-07-04T19:10:42.372747Z [info ] Waiting for 240 seconds [0:04:00], retry every 1 seconds. (Function: ocp_resources.resource.wait_deleted.lambda: self.exists) [timeout_sampler] 2026-07-04T19:10:42.396931Z [info ] Elapsed time: 7.581710815429688e-05 [0:00:00.000076] [timeout_sampler] 2026-07-04T19:10:42.397136Z [info ] Delete EvalHub evalhub-mcp-mt  [ocp_resources EvalHub] 2026-07-04T19:10:42.429673Z [info ] Deleting {'apiVersion': 'trustyai.opendatahub.io/v1alpha1', 'kind': 'EvalHub', 'metadata': {'creationTimestamp': '2026-07-04T19:05:31Z', 'finalizers': ['trustyai.opendatahub.io/evalhub-finalizer'], 'generation': 2, 'managedFields': [{'apiVersion': 'trustyai.opendatahub.io/v1alpha1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:finalizers': {'.': {}, 'v:"trustyai.opendatahub.io/evalhub-finalizer"': {}}}}, 'manager': 'manager', 'operation': 'Update', 'time': '2026-07-04T19:05:31Z'}, {'apiVersion': 'trustyai.opendatahub.io/v1alpha1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:spec': {'.': {}, 'f:collections': {}, 'f:database': {'.': {}, 'f:maxIdleConns': {}, 'f:maxOpenConns': {}, 'f:type': {}}, 'f:mcp': {'.': {}, 'f:authSecret': {}, 'f:enabled': {}, 'f:env': {}, 'f:replicas': {}, 'f:transport': {}}, 'f:providers': {}, 'f:replicas': {}}}, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': '2026-07-04T19:05:41Z'}, {'apiVersion': 'trustyai.opendatahub.io/v1alpha1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:status': {'.': {}, 'f:activeCollections': {}, 'f:activeProviders': {}, 'f:conditions': {}, 'f:lastUpdateTime': {}, 'f:mcp': {'.': {}, 'f:conditions': {}, 'f:phase': {}, 'f:ready': {}}, 'f:phase': {}, 'f:ready': {}, 'f:readyReplicas': {}, 'f:replicas': {}, 'f:url': {}}}, 'manager': 'manager', 'operation': 'Update', 'subresource': 'status', 'time': '2026-07-04T19:10:42Z'}], 'name': 'evalhub-mcp-mt', 'namespace': 'test-evalhub-mcp-route', 'resourceVersion': '378392139', 'uid': '4f8acd4b-b437-48e1-8e6d-7b43e282c416'}, 'spec': {'collections': ['leaderboard-v2'], 'database': {'maxIdleConns': 5, 'maxOpenConns': 25, 'type': 'sqlite'}, 'mcp': {'authSecret': 'evalhub-mcp-mt-mcp-token', 'enabled': True, 'env': [{'name': 'EVALHUB_TENANT', 'value': 'test-evalhub-tenant-a-testevalhubmcproute-777de6'}], 'replicas': 1, 'transport': 'http'}, 'providers': ['garak', 'garak-kfp', 'lm-evaluation-harness'], 'replicas': 1}, 'status': {'activeCollections': ['leaderboard-v2'], 'activeProviders': ['garak', 'garak-kfp', 'lm-evaluation-harness'], 'conditions': [{'lastTransitionTime': '2026-07-04T19:10:42Z', 'message': 'All replicas are ready', 'reason': 'DeploymentReady', 'status': 'True', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-04T19:10:42Z', 'reason': 'MonitoringReady', 'status': 'False', 'type': 'MonitoringDegraded'}], 'lastUpdateTime': '2026-07-04T19:10:42Z', 'mcp': {'conditions': [{'lastTransitionTime': '2026-07-04T19:05:43Z', 'message': 'Route reconciliation succeeded', 'observedGeneration': 2, 'reason': 'MCPRouteReady', 'status': 'True', 'type': 'RouteReady'}, {'lastTransitionTime': '2026-07-04T19:05:43Z', 'message': '', 'observedGeneration': 2, 'reason': 'ReconcileComplete', 'status': 'True', 'type': 'Reconciled'}, {'lastTransitionTime': '2026-07-04T19:05:43Z', 'message': 'Waiting for MCP deployment (0/1 replicas ready)', 'observedGeneration': 2, 'reason': 'DeploymentNotReady', 'status': 'False', 'type': 'Ready'}], 'phase': 'Pending', 'ready': False}, 'phase': 'Ready', 'ready': 'True', 'readyReplicas': 1, 'replicas': 1, 'url': 'https://evalhub-mcp-mt.test-evalhub-mcp-route.svc.cluster.local:8443'}} [ocp_resources EvalHub] 2026-07-04T19:10:42.450624Z [info ] Wait until EvalHub evalhub-mcp-mt is deleted [ocp_resources EvalHub] 2026-07-04T19:10:42.450767Z [info ] Waiting for 240 seconds [0:04:00], retry every 1 seconds. (Function: ocp_resources.resource.wait_deleted.lambda: self.exists) [timeout_sampler] 2026-07-04T19:10:43.489908Z [info ] Elapsed time: 1.0133330821990967 [0:00:01.013333] [timeout_sampler] 2026-07-04T19:10:43.506877Z [info ] Delete Namespace test-evalhub-tenant-a-testevalhubmcproute-777de6 [ocp_resources Namespace] 2026-07-04T19:10:43.543410Z [info ] Deleting {'kind': 'Namespace', 'apiVersion': 'v1', 'metadata': {'name': 'test-evalhub-tenant-a-testevalhubmcproute-777de6', 'uid': '9f9bb845-2d0e-4b3f-9cf7-cf04ba45d145', 'resourceVersion': '378386661', 'creationTimestamp': '2026-07-04T19:05:30Z', 'labels': {'evalhub.trustyai.opendatahub.io/tenant': 'true', 'kubernetes.io/metadata.name': 'test-evalhub-tenant-a-testevalhubmcproute-777de6', 'pod-security.kubernetes.io/audit': 'restricted', 'pod-security.kubernetes.io/audit-version': 'latest', 'pod-security.kubernetes.io/warn': 'restricted', 'pod-security.kubernetes.io/warn-version': 'latest'}, 'annotations': {'openshift.io/sa.scc.mcs': 's0:c35,c0', 'openshift.io/sa.scc.supplemental-groups': '1001190000/10000', 'openshift.io/sa.scc.uid-range': '1001190000/10000', 'security.openshift.io/MinimallySufficientPodSecurityStandard': 'restricted'}, 'managedFields': [{'manager': 'pod-security-admission-label-synchronization-controller', 'operation': 'Apply', 'apiVersion': 'v1', 'time': '2026-07-04T19:05:31Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'f:security.openshift.io/MinimallySufficientPodSecurityStandard': {}}, 'f:labels': {'f:pod-security.kubernetes.io/audit': {}, 'f:pod-security.kubernetes.io/audit-version': {}, 'f:pod-security.kubernetes.io/warn': {}, 'f:pod-security.kubernetes.io/warn-version': {}}}}}, {'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2026-07-04T19:05:30Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:labels': {'.': {}, 'f:evalhub.trustyai.opendatahub.io/tenant': {}, 'f:kubernetes.io/metadata.name': {}}}}}, {'manager': 'cluster-policy-controller', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2026-07-04T19:05:31Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'.': {}, 'f:openshift.io/sa.scc.mcs': {}, 'f:openshift.io/sa.scc.supplemental-groups': {}, 'f:openshift.io/sa.scc.uid-range': {}}}}}]}, 'spec': {'finalizers': ['kubernetes']}, 'status': {'phase': 'Active'}} [ocp_resources Namespace] 2026-07-04T19:10:43.569700Z [info ] Wait until Namespace test-evalhub-tenant-a-testevalhubmcproute-777de6 is deleted [ocp_resources Namespace] 2026-07-04T19:10:43.569835Z [info ] Waiting for 240 seconds [0:04:00], retry every 1 seconds. (Function: ocp_resources.resource.wait_deleted.lambda: self.exists) [timeout_sampler] 2026-07-04T19:10:51.744076Z [info ] Elapsed time: 8.158472299575806 [0:00:08.158472] [timeout_sampler] 2026-07-04T19:10:51.861660Z [info ] Delete Namespace test-evalhub-mcp-route [ocp_resources Namespace] 2026-07-04T19:10:51.890113Z [info ] Deleting {'kind': 'Namespace', 'apiVersion': 'v1', 'metadata': {'name': 'test-evalhub-mcp-route', 'uid': 'c5797a6a-36bd-4c54-b2ad-9da4d6093d81', 'resourceVersion': '378386643', 'creationTimestamp': '2026-07-04T19:05:30Z', 'labels': {'kubernetes.io/metadata.name': 'test-evalhub-mcp-route', 'pod-security.kubernetes.io/audit': 'restricted', 'pod-security.kubernetes.io/audit-version': 'latest', 'pod-security.kubernetes.io/warn': 'restricted', 'pod-security.kubernetes.io/warn-version': 'latest'}, 'annotations': {'openshift.io/sa.scc.mcs': 's0:c34,c29', 'openshift.io/sa.scc.supplemental-groups': '1001180000/10000', 'openshift.io/sa.scc.uid-range': '1001180000/10000', 'security.openshift.io/MinimallySufficientPodSecurityStandard': 'restricted'}, 'managedFields': [{'manager': 'pod-security-admission-label-synchronization-controller', 'operation': 'Apply', 'apiVersion': 'v1', 'time': '2026-07-04T19:05:30Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'f:security.openshift.io/MinimallySufficientPodSecurityStandard': {}}, 'f:labels': {'f:pod-security.kubernetes.io/audit': {}, 'f:pod-security.kubernetes.io/audit-version': {}, 'f:pod-security.kubernetes.io/warn': {}, 'f:pod-security.kubernetes.io/warn-version': {}}}}}, {'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2026-07-04T19:05:30Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:labels': {'.': {}, 'f:kubernetes.io/metadata.name': {}}}}}, {'manager': 'cluster-policy-controller', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2026-07-04T19:05:30Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'.': {}, 'f:openshift.io/sa.scc.mcs': {}, 'f:openshift.io/sa.scc.supplemental-groups': {}, 'f:openshift.io/sa.scc.uid-range': {}}}}}]}, 'spec': {'finalizers': ['kubernetes']}, 'status': {'phase': 'Active'}} [ocp_resources Namespace] 2026-07-04T19:10:51.910630Z [info ] Wait until Namespace test-evalhub-mcp-route is deleted [ocp_resources Namespace] 2026-07-04T19:10:51.910794Z [info ] Waiting for 240 seconds [0:04:00], retry every 1 seconds. (Function: ocp_resources.resource.wait_deleted.lambda: self.exists) [timeout_sampler] _ 7 of 10 completed, 1 Pass, 1 Fail, 0 Skip, 0 XPass, 0 XFail, 5 Error, 0 ReRun _ tests/ai_safety/evalhub/test_evalhub_conversion.py::test_evalhub_crd_serves_both_versions 2026-07-04T19:11:24.397937Z [info ] Elapsed time: 32.461649894714355 [0:00:32.461650] [timeout_sampler]  ---------------------------------------- test_evalhub_crd_serves_both_versions ---------------------------------------- -------------------------------------------------------- SETUP -------------------------------------------------------- --------------------------------------------------------- CALL --------------------------------------------------------- 2026-07-04T19:11:24.398777Z [info ] kind: CustomResourceDefinition api version: apiextensions.k8s.io/v1 [ocp_resources.resource] FAILED ____________________ test_evalhub_crd_serves_both_versions _____________________ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 184, in pytest_runtest_call item.runtest() ~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1707, in runtest self.ihook.pytest_pyfunc_call(pyfuncitem=self) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 167, in pytest_pyfunc_call result = testfunction(**testargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/test_evalhub_conversion.py", line 36, in test_evalhub_crd_serves_both_versions assert "v1" in served_versions, f"v1 not served; served versions: {served_versions}" AssertionError: v1 not served; served versions: {'v1alpha1'} assert 'v1' in {'v1alpha1'} _ 8 of 10 completed, 1 Pass, 2 Fail, 0 Skip, 0 XPass, 0 XFail, 5 Error, 0 ReRun _ tests/ai_safety/evalhub/test_evalhub_conversion.py::test_evalhub_crd_conversion_strategy_is_webhook  TEST: test_evalhub_crd_serves_both_versions STATUS: FAILED 2026-07-04T19:11:24.460129Z [error ] Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 184, in pytest_runtest_call item.runtest() ~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1707, in runtest self.ihook.pytest_pyfunc_call(pyfuncitem=self) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 167, in pytest_pyfunc_call result = testfunction(**testargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/test_evalhub_conversion.py", line 36, in test_evalhub_crd_serves_both_versions assert "v1" in served_versions, f"v1 not served; served versions: {served_versions}" AssertionError: v1 not served; served versions: {'v1alpha1'} assert 'v1' in {'v1alpha1'} [conftest] ------------------------------------------------------- TEARDOWN -------------------------------------------------------  ----------------------------------- test_evalhub_crd_conversion_strategy_is_webhook ----------------------------------- -------------------------------------------------------- SETUP -------------------------------------------------------- --------------------------------------------------------- CALL --------------------------------------------------------- FAILED _______________ test_evalhub_crd_conversion_strategy_is_webhook ________________ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 184, in pytest_runtest_call item.runtest() ~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1707, in runtest self.ihook.pytest_pyfunc_call(pyfuncitem=self) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 167, in pytest_pyfunc_call result = testfunction(**testargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/test_evalhub_conversion.py", line 54, in test_evalhub_crd_conversion_strategy_is_webhook assert conversion.get("strategy") == "Webhook", ( f"Expected conversion strategy 'Webhook', got '{conversion.get('strategy')}'" ) AssertionError: Expected conversion strategy 'Webhook', got 'None' assert 'None' == 'Webhook' - Webhook + None _ 9 of 10 completed, 1 Pass, 3 Fail, 0 Skip, 0 XPass, 0 XFail, 5 Error, 0 ReRun _ tests/ai_safety/evalhub/test_evalhub_deployment.py::test_evalhub_crd_exists  TEST: test_evalhub_crd_conversion_strategy_is_webhook STATUS: FAILED 2026-07-04T19:11:24.509466Z [error ] Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 184, in pytest_runtest_call item.runtest() ~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1707, in runtest self.ihook.pytest_pyfunc_call(pyfuncitem=self) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 167, in pytest_pyfunc_call result = testfunction(**testargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/test_evalhub_conversion.py", line 54, in test_evalhub_crd_conversion_strategy_is_webhook assert conversion.get("strategy") == "Webhook", ( f"Expected conversion strategy 'Webhook', got '{conversion.get('strategy')}'" ) AssertionError: Expected conversion strategy 'Webhook', got 'None' assert 'None' == 'Webhook' - Webhook + None [conftest] ------------------------------------------------------- TEARDOWN -------------------------------------------------------  ----------------------------------------------- test_evalhub_crd_exists ----------------------------------------------- -------------------------------------------------------- SETUP -------------------------------------------------------- --------------------------------------------------------- CALL --------------------------------------------------------- PASSED TEST: test_evalhub_crd_exists STATUS: PASSED ------------------------------------------------------- TEARDOWN ------------------------------------------------------- _ 10 of 10 completed, 2 Pass, 3 Fail, 0 Skip, 0 XPass, 0 XFail, 5 Error, 0 ReRun _ = 3 failed, 2 passed, 221 deselected, 452 warnings, 5 errors in 710.27s (0:11:50) = ==================================== ERRORS ==================================== _ ERROR at setup of TestEvalHubMcpDeployment.test_evalhub_mcp_pod_has_expected_containers[model_namespace0] _ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1232, in execute result: FixtureValue = ihook.pytest_fixture_setup( ~~~~~~~~~~~~~~~~~~~~~~~~~~^ fixturedef=self, request=request ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pytest_asyncio/plugin.py", line 924, in pytest_fixture_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/setuponly.py", line 36, in pytest_fixture_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1328, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1005, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/mcp/conftest.py", line 222, in evalhub_mcp_mt_deployment deployment.wait_for_replicas(timeout=Timeout.TIMEOUT_5MIN) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/deployment.py", line 371, in wait_for_replicas for sample in samples: ^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/timeout_sampler/__init__.py", line 182, in __iter__ raise TimeoutExpiredError(self._get_exception_log(exp=last_exp), last_exp=last_exp) timeout_sampler.TimeoutExpiredError: Timed Out: 298.971563577652 Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance Last exception: N/A. _ ERROR at setup of TestEvalHubMcpDeployment.test_evalhub_mcp_pod_labels_match_operator[model_namespace0] _ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1204, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1328, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1005, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/mcp/conftest.py", line 222, in evalhub_mcp_mt_deployment deployment.wait_for_replicas(timeout=Timeout.TIMEOUT_5MIN) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/deployment.py", line 371, in wait_for_replicas for sample in samples: ^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/timeout_sampler/__init__.py", line 182, in __iter__ raise TimeoutExpiredError(self._get_exception_log(exp=last_exp), last_exp=last_exp) timeout_sampler.TimeoutExpiredError: Timed Out: 298.971563577652 Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance Last exception: N/A. _ ERROR at setup of TestEvalHubMcpDeployment.test_evalhub_mcp_configmap_mounted_in_pod[model_namespace0] _ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1204, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1328, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1005, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/mcp/conftest.py", line 222, in evalhub_mcp_mt_deployment deployment.wait_for_replicas(timeout=Timeout.TIMEOUT_5MIN) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/deployment.py", line 371, in wait_for_replicas for sample in samples: ^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/timeout_sampler/__init__.py", line 182, in __iter__ raise TimeoutExpiredError(self._get_exception_log(exp=last_exp), last_exp=last_exp) timeout_sampler.TimeoutExpiredError: Timed Out: 298.971563577652 Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance Last exception: N/A. _ ERROR at setup of TestEvalHubMcpRoute.test_evalhub_mcp_service_exposes_https_port[model_namespace0] _ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1232, in execute result: FixtureValue = ihook.pytest_fixture_setup( ~~~~~~~~~~~~~~~~~~~~~~~~~~^ fixturedef=self, request=request ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pytest_asyncio/plugin.py", line 924, in pytest_fixture_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/setuponly.py", line 36, in pytest_fixture_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1328, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1005, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/mcp/conftest.py", line 222, in evalhub_mcp_mt_deployment deployment.wait_for_replicas(timeout=Timeout.TIMEOUT_5MIN) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/deployment.py", line 371, in wait_for_replicas for sample in samples: ^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/timeout_sampler/__init__.py", line 182, in __iter__ raise TimeoutExpiredError(self._get_exception_log(exp=last_exp), last_exp=last_exp) timeout_sampler.TimeoutExpiredError: Timed Out: 297.95665740966797 Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance Last exception: N/A. _ ERROR at setup of TestEvalHubMcpRoute.test_evalhub_mcp_route_targets_service[model_namespace0] _ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 170, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 536, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1710, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 806, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 630, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1181, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 726, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1204, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1328, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1005, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/mcp/conftest.py", line 222, in evalhub_mcp_mt_deployment deployment.wait_for_replicas(timeout=Timeout.TIMEOUT_5MIN) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/deployment.py", line 371, in wait_for_replicas for sample in samples: ^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/timeout_sampler/__init__.py", line 182, in __iter__ raise TimeoutExpiredError(self._get_exception_log(exp=last_exp), last_exp=last_exp) timeout_sampler.TimeoutExpiredError: Timed Out: 297.95665740966797 Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance Last exception: N/A. =================================== FAILURES =================================== ___ TestEvalHubMcpRoute.test_evalhub_mcp_configmap_exists[model_namespace0] ____ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 184, in pytest_runtest_call item.runtest() ~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1707, in runtest self.ihook.pytest_pyfunc_call(pyfuncitem=self) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 167, in pytest_pyfunc_call result = testfunction(**testargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/mcp/test_evalhub_mcp_route.py", line 86, in test_evalhub_mcp_configmap_exists assert "config.yaml" in data, f"Expected config.yaml in {configmap_name}" AssertionError: Expected config.yaml in evalhub-mcp-mt-mcp-config assert 'config.yaml' in {'auth.yaml': 'authorization:\n'\n ' endpoints:\n'\n ' - path: /*\n'\n ' mappings:\n'\n ' - resources:\n'\n ' - resourceAttributes:\n'\n ' namespace: "test-evalhub-mcp-route"\n'\n ' apiGroup: trustyai.opendatahub.io\n'\n ' resource: evalhubs\n'\n ' subresource: proxy\n'\n ' name: "evalhub-mcp-mt"\n'\n ' verb: get\n'\n ' - resourceAttributes:\n'\n ' namespace: "test-evalhub-mcp-route"\n'\n ' apiGroup: trustyai.opendatahub.io\n'\n ' resource: evalhubs\n'\n ' subresource: proxy\n'\n ' name: "evalhub-mcp-mt"\n'\n ' verb: create\n',\n 'config.yaml': 'base_url: '\n 'https://evalhub-mcp-mt.test-evalhub-mcp-route.svc.cluster.local:8443\n'\n 'ca_cert_path: /etc/evalhub-mcp/ca/service-ca.crt\n'\n 'host: 127.0.0.1\n'\n 'port: 8445\n'\n 'transport: http\n'} ____________________ test_evalhub_crd_serves_both_versions _____________________ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 184, in pytest_runtest_call item.runtest() ~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1707, in runtest self.ihook.pytest_pyfunc_call(pyfuncitem=self) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 167, in pytest_pyfunc_call result = testfunction(**testargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/test_evalhub_conversion.py", line 36, in test_evalhub_crd_serves_both_versions assert "v1" in served_versions, f"v1 not served; served versions: {served_versions}" AssertionError: v1 not served; served versions: {'v1alpha1'} assert 'v1' in {'v1alpha1'} _______________ test_evalhub_crd_conversion_strategy_is_webhook ________________ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 361, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 250, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 184, in pytest_runtest_call item.runtest() ~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1707, in runtest self.ihook.pytest_pyfunc_call(pyfuncitem=self) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 167, in pytest_pyfunc_call result = testfunction(**testargs) File "/home/odh/opendatahub-tests/tests/ai_safety/evalhub/test_evalhub_conversion.py", line 54, in test_evalhub_crd_conversion_strategy_is_webhook assert conversion.get("strategy") == "Webhook", ( f"Expected conversion strategy 'Webhook', got '{conversion.get('strategy')}'" ) AssertionError: Expected conversion strategy 'Webhook', got 'None' assert 'None' == 'Webhook' - Webhook + None =============================== warnings summary =============================== tests/ai_safety/evalhub/mcp/test_evalhub_mcp_deployment.py: 66 warnings tests/ai_safety/evalhub/mcp/test_evalhub_mcp_route.py: 378 warnings tests/ai_safety/evalhub/test_evalhub_conversion.py: 6 warnings tests/ai_safety/evalhub/test_evalhub_deployment.py: 2 warnings /home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/urllib3/connectionpool.py:1110: InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.ods-qe-psi-07.osp.rh-ods.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html - generated xml file: /workspace/tests-shared/tests-payload/results/ai-safety-evalhub-smoke.xml - =========================== short test summary info ============================ FAILED tests/ai_safety/evalhub/mcp/test_evalhub_mcp_route.py::TestEvalHubMcpRoute::test_evalhub_mcp_configmap_exists[model_namespace0] - AssertionError: Expected config.yaml in evalhub-mcp-mt-mcp-config assert 'config.yaml' in {'auth.yaml': 'authorization:\n'\n ' endpoints:\n'\n ' - path: /*\n'\n ' mappings:\n'\n ' - resources:\n'\n ' - resourceAttributes:\n'\n ' namespace: "test-evalhub-mcp-route"\n'\n ' apiGroup: trustyai.opendatahub.io\n'\n ' resource: evalhubs\n'\n ' subresource: proxy\n'\n ' name: "evalhub-mcp-mt"\n'\n ' verb: get\n'\n ' - resourceAttributes:\n'\n ' namespace: "test-evalhub-mcp-route"\n'\n ' apiGroup: trustyai.opendatahub.io\n'\n ' resource: evalhubs\n'\n ' subresource: proxy\n'\n ' name: "evalhub-mcp-mt"\n'\n ' verb: create\n',\n 'config.yaml': 'base_url: '\n 'https://evalhub-mcp-mt.test-evalhub-mcp-route.svc.cluster.local:8443\n'\n 'ca_cert_path: /etc/evalhub-mcp/ca/service-ca.crt\n'\n 'host: 127.0.0.1\n'\n 'port: 8445\n'\n 'transport: http\n'} FAILED tests/ai_safety/evalhub/test_evalhub_conversion.py::test_evalhub_crd_serves_both_versions - AssertionError: v1 not served; served versions: {'v1alpha1'} assert 'v1' in {'v1alpha1'} FAILED tests/ai_safety/evalhub/test_evalhub_conversion.py::test_evalhub_crd_conversion_strategy_is_webhook - AssertionError: Expected conversion strategy 'Webhook', got 'None' assert 'None' == 'Webhook' - Webhook + None ERROR tests/ai_safety/evalhub/mcp/test_evalhub_mcp_deployment.py::TestEvalHubMcpDeployment::test_evalhub_mcp_pod_has_expected_containers[model_namespace0] - timeout_sampler.TimeoutExpiredError: Timed Out: 298.971563577652 Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance Last exception: N/A. ERROR tests/ai_safety/evalhub/mcp/test_evalhub_mcp_deployment.py::TestEvalHubMcpDeployment::test_evalhub_mcp_pod_labels_match_operator[model_namespace0] - timeout_sampler.TimeoutExpiredError: Timed Out: 298.971563577652 Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance Last exception: N/A. ERROR tests/ai_safety/evalhub/mcp/test_evalhub_mcp_deployment.py::TestEvalHubMcpDeployment::test_evalhub_mcp_configmap_mounted_in_pod[model_namespace0] - timeout_sampler.TimeoutExpiredError: Timed Out: 298.971563577652 Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance Last exception: N/A. ERROR tests/ai_safety/evalhub/mcp/test_evalhub_mcp_route.py::TestEvalHubMcpRoute::test_evalhub_mcp_service_exposes_https_port[model_namespace0] - timeout_sampler.TimeoutExpiredError: Timed Out: 297.95665740966797 Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance Last exception: N/A. ERROR tests/ai_safety/evalhub/mcp/test_evalhub_mcp_route.py::TestEvalHubMcpRoute::test_evalhub_mcp_route_targets_service[model_namespace0] - timeout_sampler.TimeoutExpiredError: Timed Out: 297.95665740966797 Function: ocp_resources.deployment.wait_for_replicas.lambda: self.instance Last exception: N/A. = 3 failed, 2 passed, 221 deselected, 452 warnings, 5 errors in 710.27s (0:11:50) =