--- BVT pytest attempt 2026-06-14T13:17:33.534234+00:00 (/.local/bin/uv run pytest...) --- 2026-06-14T13:17:35.011727Z [info ] Writing tests log to /home/odh/opendatahub-tests/results/pytest-tests.log [conftest] 2026-06-14T13:17:35.020855Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.021059Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.021213Z [info ] PRAGMA main.table_info("OpenDataHubTestTable") [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.021359Z [info ] PRAGMA main.table_info("OpenDataHubTestTable") [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.021786Z [info ] [raw sql] ()  [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.022462Z [info ] [raw sql] ()  [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.022626Z [info ] PRAGMA temp.table_info("OpenDataHubTestTable") [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.022764Z [info ] PRAGMA temp.table_info("OpenDataHubTestTable") [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.022920Z [info ] [raw sql] ()  [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.023209Z [info ] [raw sql] ()  [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.023348Z [info ]  CREATE TABLE "OpenDataHubTestTable" ( id INTEGER NOT NULL, test_name VARCHAR(500) NOT NULL, start_time INTEGER NOT NULL, PRIMARY KEY (id) )  [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.023468Z [info ]  CREATE TABLE "OpenDataHubTestTable" ( id INTEGER NOT NULL, test_name VARCHAR(500) NOT NULL, start_time INTEGER NOT NULL, PRIMARY KEY (id) )  [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.023596Z [info ] [no key 0.00005s] ()  [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.023726Z [info ] [no key 0.00005s] ()  [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.023841Z [info ] COMMIT  [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.024169Z [info ] COMMIT  [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.027469Z [info ] Trying to get client via new_client_from_config [ocp_resources.resource] 2026-06-14T13:17:35.074611Z [info ] kind: DataScienceCluster api version: datasciencecluster.opendatahub.io/v2 [ocp_resources.resource] 2026-06-14T13:17:35.090064Z [info ] kind: DSCInitialization api version: dscinitialization.opendatahub.io/v2 [ocp_resources.resource] 2026-06-14T13:17:35.098033Z [info ] Running model registry tests against default namespace [conftest] 2026-06-14T13:17:35.098245Z [info ] kind: DataScienceCluster api version: datasciencecluster.opendatahub.io/v2 [ocp_resources.resource] ============================= test session starts ============================== platform linux -- Python 3.14.5, pytest-9.0.3, pluggy-1.6.0 -- /home/odh/opendatahub-tests/.venv/bin/python cachedir: .pytest_cache metadata: {'Python': '3.14.5', 'Platform': 'Linux-5.14.0-427.115.1.el9_4.x86_64-x86_64-with-glibc2.42', 'Packages': {'pytest': '9.0.3', 'pluggy': '1.6.0'}, 'Plugins': {'anyio': '4.13.0', 'asyncio': '1.3.0', 'dependency': '0.6.1', 'html': '4.2.0', 'metadata': '3.1.1', 'order': '1.3.0', 'progress': '1.4.0', 'testconfig': '0.2.0', 'xdist': '3.8.0', 'syrupy': '5.1.0'}} rootdir: /home/odh/opendatahub-tests configfile: pytest.ini plugins: anyio-4.13.0, asyncio-1.3.0, dependency-0.6.1, html-4.2.0, metadata-3.1.1, order-1.3.0, progress-1.4.0, testconfig-0.2.0, xdist-3.8.0, syrupy-5.1.0 asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function collecting ... collected 105 items / 94 deselected / 11 selected tests/model_serving/maas_billing/maas_api_key/test_api_key_auth_policy_validation.py::TestAuthPolicyApiKeyValidation::test_auth_policy_callback_url_uses_correct_namespace  --------------------------------- test_auth_policy_callback_url_uses_correct_namespace --------------------------------- -------------------------------------------------------- SETUP -------------------------------------------------------- 2026-06-14T13:17:35.348864Z [info ] (OpenDataHubTestTable|OpenDataHubTestTable) _post_configure_properties() started [sqlalchemy.orm.mapper.Mapper] 2026-06-14T13:17:35.349080Z [info ] (OpenDataHubTestTable|OpenDataHubTestTable) initialize prop id [sqlalchemy.orm.mapper.Mapper] 2026-06-14T13:17:35.349227Z [info ] (OpenDataHubTestTable|OpenDataHubTestTable) initialize prop test_name [sqlalchemy.orm.mapper.Mapper] 2026-06-14T13:17:35.349371Z [info ] (OpenDataHubTestTable|OpenDataHubTestTable) initialize prop start_time [sqlalchemy.orm.mapper.Mapper] 2026-06-14T13:17:35.349525Z [info ] (OpenDataHubTestTable|OpenDataHubTestTable) _post_configure_properties() complete [sqlalchemy.orm.mapper.Mapper] 2026-06-14T13:17:35.350042Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.350180Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.350295Z [info ] INSERT INTO "OpenDataHubTestTable" (test_name, start_time) VALUES (?, ?) [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.350406Z [info ] INSERT INTO "OpenDataHubTestTable" (test_name, start_time) VALUES (?, ?) [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.350549Z [info ] [generated in 0.00012s] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_api_key_auth_policy_validation.py::test_auth_policy_callback_url_uses_correct_namespace', 1781443055) [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.350679Z [info ] [generated in 0.00012s] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_api_key_auth_policy_validation.py::test_auth_policy_callback_url_uses_correct_namespace', 1781443055) [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.350805Z [info ] COMMIT  [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.351281Z [info ] COMMIT  [sqlalchemy.engine.Engine] 2026-06-14T13:17:35.355464Z [info ] Executing session fixture: event_loop_policy [conftest] 2026-06-14T13:17:35.355631Z [info ] Executing session fixture: _syrupy_apply_ide_patches [conftest] 2026-06-14T13:17:35.357048Z [info ] Executing session fixture: admin_client [conftest] 2026-06-14T13:17:35.357199Z [info ] Trying to get client via new_client_from_config [ocp_resources.resource] 2026-06-14T13:17:35.357348Z [info ] Executing session fixture: dsc_resource [conftest] 2026-06-14T13:17:35.357487Z [info ] kind: DataScienceCluster api version: datasciencecluster.opendatahub.io/v2 [ocp_resources.resource] 2026-06-14T13:17:35.388829Z [info ] Executing session fixture: tmp_path_factory [conftest] 2026-06-14T13:17:35.389074Z [info ] Executing session fixture: tests_tmp_dir [conftest] 2026-06-14T13:17:35.389792Z [info ] Executing session fixture: os_path_environment [conftest] 2026-06-14T13:17:35.390005Z [info ] Executing session fixture: tmpdir_factory [conftest] 2026-06-14T13:17:35.390166Z [info ] Executing session fixture: bin_directory [conftest] 2026-06-14T13:17:35.390976Z [info ] Executing session fixture: oc_binary_path [conftest] 2026-06-14T13:17:35.395190Z [info ] kind: ConsoleCLIDownload api version: console.openshift.io/v1 [ocp_resources.resource] 2026-06-14T13:17:35.402429Z [info ] All oc console cli download links: ['https://downloads-openshift-console.apps.19b18154-ecba-4af6-9a35-53ce643898a9.prod.konfluxeaas.com/amd64/linux/oc.tar'] [utilities.infra] name=utilities.infra 2026-06-14T13:17:35.402587Z [info ] Downloading archive using: url=https://downloads-openshift-console.apps.19b18154-ecba-4af6-9a35-53ce643898a9.prod.konfluxeaas.com/amd64/linux/oc.tar [utilities.infra] name=utilities.infra 2026-06-14T13:17:36.247729Z [info ] Extract the downloaded archive. [utilities.infra] name=utilities.infra 2026-06-14T13:17:36.329925Z [info ] Downloaded file: ['oc']  [utilities.infra] name=utilities.infra 2026-06-14T13:17:36.350420Z [info ] Executing session fixture: bin_directory_to_os_path [conftest] 2026-06-14T13:17:36.348840Z [info ] OC binary path: /tmp/pytest-ETB8SNRa7LPBtSe7ejPGbj/bin0/oc [tests.conftest] name=tests.conftest 2026-06-14T13:17:36.348922Z [info ] Adding /tmp/pytest-ETB8SNRa7LPBtSe7ejPGbj/bin0 to $PATH [tests.conftest] name=tests.conftest 2026-06-14T13:17:36.350981Z [info ] Executing session fixture: nodes [conftest] 2026-06-14T13:17:36.596376Z [info ] Executing session fixture: dsci_resource [conftest] 2026-06-14T13:17:36.596545Z [info ] kind: DSCInitialization api version: dscinitialization.opendatahub.io/v2 [ocp_resources.resource] 2026-06-14T13:17:36.602038Z [info ] Executing session fixture: record_testsuite_property [conftest] 2026-06-14T13:17:36.602244Z [info ] Executing session fixture: junitxml_plugin [conftest] 2026-06-14T13:17:36.602385Z [info ] Executing session fixture: cluster_sanity_scope_session [conftest] 2026-06-14T13:17:36.601505Z [info ] Check cluster sanity.  [utilities.infra] name=utilities.infra 2026-06-14T13:17:36.602648Z [info ] Verify all nodes are in a healthy condition. [ocp_utilities.infra] 2026-06-14T13:17:36.615700Z [info ] Verify all nodes are schedulable. [ocp_utilities.infra] 2026-06-14T13:17:36.628848Z [info ] Waiting for 120 seconds [0:02:00], retry every 5 seconds. (Function: utilities.infra.wait_for_dsci_status_ready Kwargs: {'dsci_resource': }) [timeout_sampler] 2026-06-14T13:17:36.628397Z [info ] Wait for DSCI default-dsci to be in Ready status. [utilities.infra] name=utilities.infra 2026-06-14T13:17:36.629115Z [info ] Get DSCInitialization default-dsci status [ocp_resources DSCInitialization] 2026-06-14T13:17:36.633359Z [info ] Elapsed time: 8.0108642578125e-05 [0:00:00.000080] [timeout_sampler] 2026-06-14T13:17:36.633597Z [info ] Waiting for 120 seconds [0:02:00], retry every 5 seconds. (Function: utilities.infra.wait_for_dsc_status_ready Kwargs: {'dsc_resource': }) [timeout_sampler] 2026-06-14T13:17:36.632927Z [info ] Wait for DSC default-dsc are Ready. [utilities.infra] name=utilities.infra 2026-06-14T13:17:36.633954Z [info ] Get DataScienceCluster default-dsc status [ocp_resources DataScienceCluster] 2026-06-14T13:17:36.638531Z [info ] Elapsed time: 6.389617919921875e-05 [0:00:00.000064] [timeout_sampler] 2026-06-14T13:17:36.638691Z [info ] Executing session fixture: autouse_fixtures [conftest] 2026-06-14T13:17:36.638909Z [info ] Executing session fixture: maas_gateway_api_hostname [conftest] 2026-06-14T13:17:36.639029Z [info ] kind: Ingress api version: config.openshift.io/v1 [ocp_resources.resource] 2026-06-14T13:17:36.646727Z [info ] Executing session fixture: maas_gateway_api [conftest] 2026-06-14T13:17:36.660431Z [info ] kind: Gateway api version: gateway.networking.k8s.io/v1 [ocp_resources.resource] 2026-06-14T13:17:36.665647Z [info ] Creating gateway openshift-ingress/maas-default-gateway [conftest] name=conftest 2026-06-14T13:17:36.668354Z [info ] Create Gateway maas-default-gateway [ocp_resources Gateway] 2026-06-14T13:17:36.668581Z [info ] Posting {'apiVersion': 'gateway.networking.k8s.io/v1', 'kind': 'Gateway', 'metadata': {'name': 'maas-default-gateway', 'labels': {'app.kubernetes.io/name': 'maas', 'app.kubernetes.io/instance': 'maas-default-gateway', 'app.kubernetes.io/component': 'gateway', 'opendatahub.io/managed': 'false'}, 'annotations': {'opendatahub.io/managed': 'false', 'security.opendatahub.io/authorino-tls-bootstrap': 'true'}, 'namespace': 'openshift-ingress'}, 'spec': {'gatewayClassName': 'openshift-default', 'listeners': [{'name': 'http', 'hostname': 'maas.apps.19b18154-ecba-4af6-9a35-53ce643898a9.prod.konfluxeaas.com', 'port': 80, 'protocol': 'HTTP', 'allowedRoutes': {'namespaces': {'from': 'All'}}}, {'name': 'https', 'hostname': 'maas.apps.19b18154-ecba-4af6-9a35-53ce643898a9.prod.konfluxeaas.com', 'port': 443, 'protocol': 'HTTPS', 'allowedRoutes': {'namespaces': {'from': 'All'}}, 'tls': {'mode': 'Terminate', 'certificateRefs': [{'group': '', 'kind': 'Secret', 'name': 'data-science-gateway-service-tls'}]}}]}} [ocp_resources Gateway] 2026-06-14T13:17:36.681588Z [info ] Wait until Gateway maas-default-gateway is created [ocp_resources Gateway] 2026-06-14T13:17:36.681740Z [info ] Waiting for 240 seconds [0:04:00], retry every 1 seconds. (Function: ocp_resources.resource.wait.lambda: self.exists) [timeout_sampler] 2026-06-14T13:17:36.686773Z [info ] Elapsed time: 3.600120544433594e-05 [0:00:00.000036] [timeout_sampler] 2026-06-14T13:17:36.686985Z [info ] Executing session fixture: pytestconfig [conftest] 2026-06-14T13:17:36.687101Z [info ] Executing session fixture: use_unprivileged_client [conftest] 2026-06-14T13:17:36.687204Z [info ] Executing session fixture: kubconfig_filepath [conftest] 2026-06-14T13:17:36.687304Z [info ] Executing session fixture: cluster_authentication [conftest] 2026-06-14T13:17:36.687409Z [info ] kind: Authentication api version: config.openshift.io/v1 [ocp_resources.resource] 2026-06-14T13:17:36.694186Z [info ] Executing session fixture: is_byoidc [conftest] 2026-06-14T13:17:36.698665Z [info ] Executing session fixture: non_admin_user_password [conftest] ERROR TEST: TestAuthPolicyApiKeyValidation.test_auth_policy_callback_url_uses_correct_namespace [setup] STATUS: ERROR 2026-06-14T13:17:36.723968Z [error ] Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) kubernetes.client.exceptions.ApiException: (404) Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 353, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 245, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 165, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 523, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1723, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 707, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 539, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1110, in execute result: FixtureValue = ihook.pytest_fixture_setup( ~~~~~~~~~~~~~~~~~~~~~~~~~~^ fixturedef=self, request=request ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pytest_asyncio/plugin.py", line 728, in pytest_fixture_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/setuponly.py", line 36, in pytest_fixture_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1202, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 915, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/conftest.py", line 445, in non_admin_user_password if users_Secret := list( ~~~~^ Secret.get( ^^^^^^^^^^^ ...<3 lines>... ) ^ ): ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 1690, in _get _resources = cls._prepare_resources(*args, client=_client, singular_name=singular_name, **kwargs) # type: ignore[misc] File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 881, in _prepare_resources return client.resources.get( ~~~~~~~~~~~~~~~~~~~~~ ...<2 lines>... **get_kwargs, ~~~~~~~~~~~~~ ).get(*args, **kwargs, timeout_seconds=cls.timeout_seconds) ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 112, in get return self.request('get', path, **kwargs) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 57, in inner raise api_exception(e) kubernetes.dynamic.exceptions.NotFoundError: 404 Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' Original traceback: File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) [conftest] 2026-06-14T13:17:36.724171Z [info ] Must-gather collection is enabled for /home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_api_key_auth_policy_validation.py::test_auth_policy_callback_url_uses_correct_namespace. [conftest] 2026-06-14T13:17:36.724347Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:17:36.724493Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:17:36.728253Z [info ] SELECT "OpenDataHubTestTable".start_time AS "OpenDataHubTestTable_start_time" FROM "OpenDataHubTestTable" WHERE "OpenDataHubTestTable".test_name = ? LIMIT ? OFFSET ? [sqlalchemy.engine.Engine] 2026-06-14T13:17:36.728409Z [info ] SELECT "OpenDataHubTestTable".start_time AS "OpenDataHubTestTable_start_time" FROM "OpenDataHubTestTable" WHERE "OpenDataHubTestTable".test_name = ? LIMIT ? OFFSET ? [sqlalchemy.engine.Engine] 2026-06-14T13:17:36.728579Z [info ] [generated in 0.00019s] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_api_key_auth_policy_validation.py::test_auth_policy_callback_url_uses_correct_namespace', 1, 0) [sqlalchemy.engine.Engine] 2026-06-14T13:17:36.728733Z [info ] [generated in 0.00019s] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_api_key_auth_policy_validation.py::test_auth_policy_callback_url_uses_correct_namespace', 1, 0) [sqlalchemy.engine.Engine] 2026-06-14T13:17:36.728858Z [info ] ROLLBACK  [sqlalchemy.engine.Engine] 2026-06-14T13:17:36.728992Z [info ] ROLLBACK  [sqlalchemy.engine.Engine] 2026-06-14T13:17:36.729096Z [info ] Trying to get client via new_client_from_config [ocp_resources.resource] 2026-06-14T13:17:36.781056Z [info ] kind: Subscription api version: operators.coreos.com/v1alpha1 [ocp_resources.resource] 2026-06-14T13:17:36.785497Z [info ] Checking subscription rhods-operator [utilities.infra] name=utilities.infra 2026-06-14T13:17:36.790270Z [info ] Expected CSV: rhods-operator.3.4.0 [utilities.infra] name=utilities.infra 2026-06-14T13:17:36.794190Z [info ] Trying to get client via new_client_from_config [ocp_resources.resource] 2026-06-14T13:17:36.798148Z [info ] kind: ClusterServiceVersion api version: operators.coreos.com/v1alpha1 [ocp_resources.resource] 2026-06-14T13:17:36.860950Z [info ] Running oc adm must-gather --dest-dir=/home/odh/opendatahub-tests/results/must-gather-collected/model_serving/maas_billing/maas_api_key/test_api_key_auth_policy_validation/TestAuthPolicyApiKeyValidation/test_auth_policy_callback_url_uses_correct_namespace/pytest_exception_interact --since=300s --image=registry.redhat.io/rhoai/odh-must-gather-rhel9@sha256:3de5ff9eee7e9b8e6650f61dbae642db3d3ccb813afb804397dc60d18f150ea6 command [pyhelper_utils.shell] 2026-06-14T13:23:04.709800Z [info ] /home/odh/opendatahub-tests/results/must-gather-collected/model_serving/maas_billing/maas_api_key/test_api_key_auth_policy_validation/TestAuthPolicyApiKeyValidation/test_auth_policy_callback_url_uses_correct_namespace/pytest_exception_interact/mg-1781443055.zip is collected successfully [utilities.must_gather_collector] name=utilities.must_gather_collector _ ERROR at setup of TestAuthPolicyApiKeyValidation.test_auth_policy_callback_url_uses_correct_namespace _ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) kubernetes.client.exceptions.ApiException: (404) Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 353, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 245, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 165, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 523, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1723, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 707, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 539, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1110, in execute result: FixtureValue = ihook.pytest_fixture_setup( ~~~~~~~~~~~~~~~~~~~~~~~~~~^ fixturedef=self, request=request ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pytest_asyncio/plugin.py", line 728, in pytest_fixture_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 53, in run_old_style_hookwrapper return result.get_result() ~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_result.py", line 103, in get_result raise exc.with_traceback(tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 38, in run_old_style_hookwrapper res = yield ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/setuponly.py", line 36, in pytest_fixture_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1202, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 915, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/conftest.py", line 445, in non_admin_user_password if users_Secret := list( ~~~~^ Secret.get( ^^^^^^^^^^^ ...<3 lines>... ) ^ ): ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 1690, in _get _resources = cls._prepare_resources(*args, client=_client, singular_name=singular_name, **kwargs) # type: ignore[misc] File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 881, in _prepare_resources return client.resources.get( ~~~~~~~~~~~~~~~~~~~~~ ...<2 lines>... **get_kwargs, ~~~~~~~~~~~~~ ).get(*args, **kwargs, timeout_seconds=cls.timeout_seconds) ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 112, in get return self.request('get', path, **kwargs) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 57, in inner raise api_exception(e) kubernetes.dynamic.exceptions.NotFoundError: 404 Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' Original traceback: File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) _ 1 of 11 completed, 0 Pass, 0 Fail, 0 Skip, 0 XPass, 0 XFail, 1 Error, 0 ReRun _ tests/model_serving/maas_billing/maas_api_key/test_api_key_auth_policy_validation.py::TestAuthPolicyApiKeyValidation::test_api_key_can_list_models[ocp_token_for_actor0] ------------------------------------------------------- TEARDOWN -------------------------------------------------------  ---------------------------------- test_api_key_can_list_models[ocp_token_for_actor0] ---------------------------------- -------------------------------------------------------- SETUP -------------------------------------------------------- 2026-06-14T13:23:04.713097Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:23:04.713537Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:23:04.713660Z [info ] INSERT INTO "OpenDataHubTestTable" (test_name, start_time) VALUES (?, ?) [sqlalchemy.engine.Engine] 2026-06-14T13:23:04.713774Z [info ] INSERT INTO "OpenDataHubTestTable" (test_name, start_time) VALUES (?, ?) [sqlalchemy.engine.Engine] 2026-06-14T13:23:04.715027Z [info ] [cached since 329.4s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_api_key_auth_policy_validation.py::test_api_key_can_list_models[ocp_token_for_actor0]', 1781443384) [sqlalchemy.engine.Engine] 2026-06-14T13:23:04.715211Z [info ] [cached since 329.4s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_api_key_auth_policy_validation.py::test_api_key_can_list_models[ocp_token_for_actor0]', 1781443384) [sqlalchemy.engine.Engine] 2026-06-14T13:23:04.715342Z [info ] COMMIT  [sqlalchemy.engine.Engine] 2026-06-14T13:23:04.715462Z [info ] COMMIT  [sqlalchemy.engine.Engine] ERROR TEST: TestAuthPolicyApiKeyValidation.test_api_key_can_list_models[ocp_token_for_actor0] [setup] STATUS: ERROR 2026-06-14T13:23:04.721032Z [error ] Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) kubernetes.client.exceptions.ApiException: (404) Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 353, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 245, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 165, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 523, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1723, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 707, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 539, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1091, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1202, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 915, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/conftest.py", line 445, in non_admin_user_password if users_Secret := list( ~~~~^ Secret.get( ^^^^^^^^^^^ ...<3 lines>... ) ^ ): ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 1690, in _get _resources = cls._prepare_resources(*args, client=_client, singular_name=singular_name, **kwargs) # type: ignore[misc] File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 881, in _prepare_resources return client.resources.get( ~~~~~~~~~~~~~~~~~~~~~ ...<2 lines>... **get_kwargs, ~~~~~~~~~~~~~ ).get(*args, **kwargs, timeout_seconds=cls.timeout_seconds) ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 112, in get return self.request('get', path, **kwargs) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 57, in inner raise api_exception(e) kubernetes.dynamic.exceptions.NotFoundError: 404 Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' Original traceback: File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) [conftest] 2026-06-14T13:23:04.724166Z [info ] Must-gather collection is enabled for /home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_api_key_auth_policy_validation.py::test_api_key_can_list_models[ocp_token_for_actor0]. [conftest] 2026-06-14T13:23:04.724343Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:23:04.724719Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:23:04.724876Z [info ] SELECT "OpenDataHubTestTable".start_time AS "OpenDataHubTestTable_start_time" FROM "OpenDataHubTestTable" WHERE "OpenDataHubTestTable".test_name = ? LIMIT ? OFFSET ? [sqlalchemy.engine.Engine] 2026-06-14T13:23:04.725084Z [info ] SELECT "OpenDataHubTestTable".start_time AS "OpenDataHubTestTable_start_time" FROM "OpenDataHubTestTable" WHERE "OpenDataHubTestTable".test_name = ? LIMIT ? OFFSET ? [sqlalchemy.engine.Engine] 2026-06-14T13:23:04.728761Z [info ] [cached since 328s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_api_key_auth_policy_validation.py::test_api_key_can_list_models[ocp_token_for_actor0]', 1, 0) [sqlalchemy.engine.Engine] 2026-06-14T13:23:04.729135Z [info ] [cached since 328s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_api_key_auth_policy_validation.py::test_api_key_can_list_models[ocp_token_for_actor0]', 1, 0) [sqlalchemy.engine.Engine] 2026-06-14T13:23:04.729279Z [info ] ROLLBACK  [sqlalchemy.engine.Engine] 2026-06-14T13:23:04.729403Z [info ] ROLLBACK  [sqlalchemy.engine.Engine] 2026-06-14T13:23:04.733233Z [info ] Trying to get client via new_client_from_config [ocp_resources.resource] 2026-06-14T13:23:04.762054Z [info ] Checking subscription rhods-operator [utilities.infra] name=utilities.infra 2026-06-14T13:23:04.765942Z [info ] Expected CSV: rhods-operator.3.4.0 [utilities.infra] name=utilities.infra 2026-06-14T13:23:04.773464Z [info ] kind: ClusterServiceVersion api version: operators.coreos.com/v1alpha1 [ocp_resources.resource] 2026-06-14T13:23:04.830562Z [info ] Running oc adm must-gather --dest-dir=/home/odh/opendatahub-tests/results/must-gather-collected/model_serving/maas_billing/maas_api_key/test_api_key_auth_policy_validation/TestAuthPolicyApiKeyValidation/test_api_key_can_list_models_ocp_token_for_actor0_/pytest_exception_interact --since=300s --image=registry.redhat.io/rhoai/odh-must-gather-rhel9@sha256:3de5ff9eee7e9b8e6650f61dbae642db3d3ccb813afb804397dc60d18f150ea6 command [pyhelper_utils.shell] _ ERROR at setup of TestAuthPolicyApiKeyValidation.test_api_key_can_list_models[ocp_token_for_actor0] _ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) kubernetes.client.exceptions.ApiException: (404) Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 353, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 245, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 165, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 523, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1723, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 707, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 539, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1091, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1202, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 915, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/conftest.py", line 445, in non_admin_user_password if users_Secret := list( ~~~~^ Secret.get( ^^^^^^^^^^^ ...<3 lines>... ) ^ ): ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 1690, in _get _resources = cls._prepare_resources(*args, client=_client, singular_name=singular_name, **kwargs) # type: ignore[misc] File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 881, in _prepare_resources return client.resources.get( ~~~~~~~~~~~~~~~~~~~~~ ...<2 lines>... **get_kwargs, ~~~~~~~~~~~~~ ).get(*args, **kwargs, timeout_seconds=cls.timeout_seconds) ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 112, in get return self.request('get', path, **kwargs) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 57, in inner raise api_exception(e) kubernetes.dynamic.exceptions.NotFoundError: 404 Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' Original traceback: File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) _ 2 of 11 completed, 0 Pass, 0 Fail, 0 Skip, 0 XPass, 0 XFail, 2 Error, 0 ReRun _ tests/model_serving/maas_billing/maas_api_key/test_api_key_crud.py::TestAPIKeyCRUD::test_create_api_key[ocp_token_for_actor0] 2026-06-14T13:28:24.310445Z [info ] /home/odh/opendatahub-tests/results/must-gather-collected/model_serving/maas_billing/maas_api_key/test_api_key_auth_policy_validation/TestAuthPolicyApiKeyValidation/test_api_key_can_list_models_ocp_token_for_actor0_/pytest_exception_interact/mg-1781443384.zip is collected successfully [utilities.must_gather_collector] name=utilities.must_gather_collector ------------------------------------------------------- TEARDOWN -------------------------------------------------------  -------------------------------------- test_create_api_key[ocp_token_for_actor0] -------------------------------------- -------------------------------------------------------- SETUP -------------------------------------------------------- 2026-06-14T13:28:24.316109Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:28:24.316228Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:28:24.316406Z [info ] INSERT INTO "OpenDataHubTestTable" (test_name, start_time) VALUES (?, ?) [sqlalchemy.engine.Engine] 2026-06-14T13:28:24.316527Z [info ] INSERT INTO "OpenDataHubTestTable" (test_name, start_time) VALUES (?, ?) [sqlalchemy.engine.Engine] 2026-06-14T13:28:24.316689Z [info ] [cached since 649s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_api_key_crud.py::test_create_api_key[ocp_token_for_actor0]', 1781443704) [sqlalchemy.engine.Engine] 2026-06-14T13:28:24.317141Z [info ] [cached since 649s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_api_key_crud.py::test_create_api_key[ocp_token_for_actor0]', 1781443704) [sqlalchemy.engine.Engine] 2026-06-14T13:28:24.317280Z [info ] COMMIT  [sqlalchemy.engine.Engine] 2026-06-14T13:28:24.317397Z [info ] COMMIT  [sqlalchemy.engine.Engine] ERROR TEST: TestAPIKeyCRUD.test_create_api_key[ocp_token_for_actor0] [setup] STATUS: ERROR 2026-06-14T13:28:24.325531Z [error ] Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) kubernetes.client.exceptions.ApiException: (404) Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 353, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 245, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 165, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 523, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1723, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 707, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 539, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1091, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1202, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 915, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/conftest.py", line 445, in non_admin_user_password if users_Secret := list( ~~~~^ Secret.get( ^^^^^^^^^^^ ...<3 lines>... ) ^ ): ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 1690, in _get _resources = cls._prepare_resources(*args, client=_client, singular_name=singular_name, **kwargs) # type: ignore[misc] File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 881, in _prepare_resources return client.resources.get( ~~~~~~~~~~~~~~~~~~~~~ ...<2 lines>... **get_kwargs, ~~~~~~~~~~~~~ ).get(*args, **kwargs, timeout_seconds=cls.timeout_seconds) ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 112, in get return self.request('get', path, **kwargs) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 57, in inner raise api_exception(e) kubernetes.dynamic.exceptions.NotFoundError: 404 Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' Original traceback: File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) [conftest] 2026-06-14T13:28:24.329412Z [info ] Must-gather collection is enabled for /home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_api_key_crud.py::test_create_api_key[ocp_token_for_actor0]. [conftest] 2026-06-14T13:28:24.329571Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:28:24.329692Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:28:24.329859Z [info ] SELECT "OpenDataHubTestTable".start_time AS "OpenDataHubTestTable_start_time" FROM "OpenDataHubTestTable" WHERE "OpenDataHubTestTable".test_name = ? LIMIT ? OFFSET ? [sqlalchemy.engine.Engine] 2026-06-14T13:28:24.333985Z [info ] SELECT "OpenDataHubTestTable".start_time AS "OpenDataHubTestTable_start_time" FROM "OpenDataHubTestTable" WHERE "OpenDataHubTestTable".test_name = ? LIMIT ? OFFSET ? [sqlalchemy.engine.Engine] 2026-06-14T13:28:24.334169Z [info ] [cached since 647.6s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_api_key_crud.py::test_create_api_key[ocp_token_for_actor0]', 1, 0) [sqlalchemy.engine.Engine] 2026-06-14T13:28:24.334323Z [info ] [cached since 647.6s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_api_key_crud.py::test_create_api_key[ocp_token_for_actor0]', 1, 0) [sqlalchemy.engine.Engine] 2026-06-14T13:28:24.334432Z [info ] ROLLBACK  [sqlalchemy.engine.Engine] 2026-06-14T13:28:24.334534Z [info ] ROLLBACK  [sqlalchemy.engine.Engine] 2026-06-14T13:28:24.334636Z [info ] Trying to get client via new_client_from_config [ocp_resources.resource] 2026-06-14T13:28:24.368783Z [info ] Checking subscription rhods-operator [utilities.infra] name=utilities.infra 2026-06-14T13:28:24.373268Z [info ] Expected CSV: rhods-operator.3.4.0 [utilities.infra] name=utilities.infra 2026-06-14T13:28:24.380316Z [info ] kind: ClusterServiceVersion api version: operators.coreos.com/v1alpha1 [ocp_resources.resource] 2026-06-14T13:28:24.445113Z [info ] Running oc adm must-gather --dest-dir=/home/odh/opendatahub-tests/results/must-gather-collected/model_serving/maas_billing/maas_api_key/test_api_key_crud/TestAPIKeyCRUD/test_create_api_key_ocp_token_for_actor0_/pytest_exception_interact --since=300s --image=registry.redhat.io/rhoai/odh-must-gather-rhel9@sha256:3de5ff9eee7e9b8e6650f61dbae642db3d3ccb813afb804397dc60d18f150ea6 command [pyhelper_utils.shell] 2026-06-14T13:33:44.690979Z [info ] /home/odh/opendatahub-tests/results/must-gather-collected/model_serving/maas_billing/maas_api_key/test_api_key_crud/TestAPIKeyCRUD/test_create_api_key_ocp_token_for_actor0_/pytest_exception_interact/mg-1781443704.zip is collected successfully [utilities.must_gather_collector] name=utilities.must_gather_collector ------------------------------------------------------- TEARDOWN ------------------------------------------------------- __ ERROR at setup of TestAPIKeyCRUD.test_create_api_key[ocp_token_for_actor0] __ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) kubernetes.client.exceptions.ApiException: (404) Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 353, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 245, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 165, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 523, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1723, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 707, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 539, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1091, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1202, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 915, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/conftest.py", line 445, in non_admin_user_password if users_Secret := list( ~~~~^ Secret.get( ^^^^^^^^^^^ ...<3 lines>... ) ^ ): ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 1690, in _get _resources = cls._prepare_resources(*args, client=_client, singular_name=singular_name, **kwargs) # type: ignore[misc] File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 881, in _prepare_resources return client.resources.get( ~~~~~~~~~~~~~~~~~~~~~ ...<2 lines>... **get_kwargs, ~~~~~~~~~~~~~ ).get(*args, **kwargs, timeout_seconds=cls.timeout_seconds) ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 112, in get return self.request('get', path, **kwargs) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 57, in inner raise api_exception(e) kubernetes.dynamic.exceptions.NotFoundError: 404 Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' Original traceback: File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) _ 3 of 11 completed, 0 Pass, 0 Fail, 0 Skip, 0 XPass, 0 XFail, 3 Error, 0 ReRun _ tests/model_serving/maas_billing/maas_api_key/test_api_key_crud.py::TestAPIKeyCRUD::test_revoke_api_key[ocp_token_for_actor0]  -------------------------------------- test_revoke_api_key[ocp_token_for_actor0] -------------------------------------- -------------------------------------------------------- SETUP -------------------------------------------------------- 2026-06-14T13:33:44.693309Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:33:44.693433Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:33:44.693560Z [info ] INSERT INTO "OpenDataHubTestTable" (test_name, start_time) VALUES (?, ?) [sqlalchemy.engine.Engine] 2026-06-14T13:33:44.693679Z [info ] INSERT INTO "OpenDataHubTestTable" (test_name, start_time) VALUES (?, ?) [sqlalchemy.engine.Engine] 2026-06-14T13:33:44.693816Z [info ] [cached since 969.3s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_api_key_crud.py::test_revoke_api_key[ocp_token_for_actor0]', 1781444024) [sqlalchemy.engine.Engine] 2026-06-14T13:33:44.693976Z [info ] [cached since 969.3s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_api_key_crud.py::test_revoke_api_key[ocp_token_for_actor0]', 1781444024) [sqlalchemy.engine.Engine] 2026-06-14T13:33:44.695317Z [info ] COMMIT  [sqlalchemy.engine.Engine] 2026-06-14T13:33:44.695475Z [info ] COMMIT  [sqlalchemy.engine.Engine] ERROR TEST: TestAPIKeyCRUD.test_revoke_api_key[ocp_token_for_actor0] [setup] STATUS: ERROR 2026-06-14T13:33:44.700627Z [error ] Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) kubernetes.client.exceptions.ApiException: (404) Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 353, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 245, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 165, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 523, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1723, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 707, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 539, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1091, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1202, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 915, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/conftest.py", line 445, in non_admin_user_password if users_Secret := list( ~~~~^ Secret.get( ^^^^^^^^^^^ ...<3 lines>... ) ^ ): ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 1690, in _get _resources = cls._prepare_resources(*args, client=_client, singular_name=singular_name, **kwargs) # type: ignore[misc] File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 881, in _prepare_resources return client.resources.get( ~~~~~~~~~~~~~~~~~~~~~ ...<2 lines>... **get_kwargs, ~~~~~~~~~~~~~ ).get(*args, **kwargs, timeout_seconds=cls.timeout_seconds) ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 112, in get return self.request('get', path, **kwargs) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 57, in inner raise api_exception(e) kubernetes.dynamic.exceptions.NotFoundError: 404 Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' Original traceback: File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) [conftest] 2026-06-14T13:33:44.700766Z [info ] Must-gather collection is enabled for /home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_api_key_crud.py::test_revoke_api_key[ocp_token_for_actor0]. [conftest] 2026-06-14T13:33:44.700879Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:33:44.705146Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:33:44.705358Z [info ] SELECT "OpenDataHubTestTable".start_time AS "OpenDataHubTestTable_start_time" FROM "OpenDataHubTestTable" WHERE "OpenDataHubTestTable".test_name = ? LIMIT ? OFFSET ? [sqlalchemy.engine.Engine] 2026-06-14T13:33:44.705569Z [info ] SELECT "OpenDataHubTestTable".start_time AS "OpenDataHubTestTable_start_time" FROM "OpenDataHubTestTable" WHERE "OpenDataHubTestTable".test_name = ? LIMIT ? OFFSET ? [sqlalchemy.engine.Engine] 2026-06-14T13:33:44.705787Z [info ] [cached since 968s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_api_key_crud.py::test_revoke_api_key[ocp_token_for_actor0]', 1, 0) [sqlalchemy.engine.Engine] 2026-06-14T13:33:44.706027Z [info ] [cached since 968s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_api_key_crud.py::test_revoke_api_key[ocp_token_for_actor0]', 1, 0) [sqlalchemy.engine.Engine] 2026-06-14T13:33:44.706206Z [info ] ROLLBACK  [sqlalchemy.engine.Engine] 2026-06-14T13:33:44.706451Z [info ] ROLLBACK  [sqlalchemy.engine.Engine] 2026-06-14T13:33:44.706711Z [info ] Trying to get client via new_client_from_config [ocp_resources.resource] 2026-06-14T13:33:44.743639Z [info ] Checking subscription rhods-operator [utilities.infra] name=utilities.infra 2026-06-14T13:33:44.747104Z [info ] Expected CSV: rhods-operator.3.4.0 [utilities.infra] name=utilities.infra 2026-06-14T13:33:44.753646Z [info ] kind: ClusterServiceVersion api version: operators.coreos.com/v1alpha1 [ocp_resources.resource] 2026-06-14T13:33:44.808762Z [info ] Running oc adm must-gather --dest-dir=/home/odh/opendatahub-tests/results/must-gather-collected/model_serving/maas_billing/maas_api_key/test_api_key_crud/TestAPIKeyCRUD/test_revoke_api_key_ocp_token_for_actor0_/pytest_exception_interact --since=300s --image=registry.redhat.io/rhoai/odh-must-gather-rhel9@sha256:3de5ff9eee7e9b8e6650f61dbae642db3d3ccb813afb804397dc60d18f150ea6 command [pyhelper_utils.shell] 2026-06-14T13:39:03.046502Z [info ] /home/odh/opendatahub-tests/results/must-gather-collected/model_serving/maas_billing/maas_api_key/test_api_key_crud/TestAPIKeyCRUD/test_revoke_api_key_ocp_token_for_actor0_/pytest_exception_interact/mg-1781444024.zip is collected successfully [utilities.must_gather_collector] name=utilities.must_gather_collector __ ERROR at setup of TestAPIKeyCRUD.test_revoke_api_key[ocp_token_for_actor0] __ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) kubernetes.client.exceptions.ApiException: (404) Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 353, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 245, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 165, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 523, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1723, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 707, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 539, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1091, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1202, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 915, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/conftest.py", line 445, in non_admin_user_password if users_Secret := list( ~~~~^ Secret.get( ^^^^^^^^^^^ ...<3 lines>... ) ^ ): ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 1690, in _get _resources = cls._prepare_resources(*args, client=_client, singular_name=singular_name, **kwargs) # type: ignore[misc] File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 881, in _prepare_resources return client.resources.get( ~~~~~~~~~~~~~~~~~~~~~ ...<2 lines>... **get_kwargs, ~~~~~~~~~~~~~ ).get(*args, **kwargs, timeout_seconds=cls.timeout_seconds) ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 112, in get return self.request('get', path, **kwargs) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 57, in inner raise api_exception(e) kubernetes.dynamic.exceptions.NotFoundError: 404 Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' Original traceback: File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) _ 4 of 11 completed, 0 Pass, 0 Fail, 0 Skip, 0 XPass, 0 XFail, 4 Error, 0 ReRun _ tests/model_serving/maas_billing/maas_api_key/test_gateway_deny_default.py::TestGatewayDenyByDefault::test_gateway_default_auth_in_gateway_namespace_and_accepted ------------------------------------------------------- TEARDOWN -------------------------------------------------------  ----------------------------- test_gateway_default_auth_in_gateway_namespace_and_accepted ----------------------------- -------------------------------------------------------- SETUP -------------------------------------------------------- 2026-06-14T13:39:03.049144Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:39:03.049261Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:39:03.049372Z [info ] INSERT INTO "OpenDataHubTestTable" (test_name, start_time) VALUES (?, ?) [sqlalchemy.engine.Engine] 2026-06-14T13:39:03.049478Z [info ] INSERT INTO "OpenDataHubTestTable" (test_name, start_time) VALUES (?, ?) [sqlalchemy.engine.Engine] 2026-06-14T13:39:03.049624Z [info ] [cached since 1288s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_gateway_deny_default.py::test_gateway_default_auth_in_gateway_namespace_and_accepted', 1781444343) [sqlalchemy.engine.Engine] 2026-06-14T13:39:03.049763Z [info ] [cached since 1288s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_gateway_deny_default.py::test_gateway_default_auth_in_gateway_namespace_and_accepted', 1781444343) [sqlalchemy.engine.Engine] 2026-06-14T13:39:03.050207Z [info ] COMMIT  [sqlalchemy.engine.Engine] 2026-06-14T13:39:03.050319Z [info ] COMMIT  [sqlalchemy.engine.Engine] ERROR TEST: TestGatewayDenyByDefault.test_gateway_default_auth_in_gateway_namespace_and_accepted [setup] STATUS: ERROR 2026-06-14T13:39:03.059461Z [error ] Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) kubernetes.client.exceptions.ApiException: (404) Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 353, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 245, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 165, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 523, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1723, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 707, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 539, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1091, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1202, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 915, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/conftest.py", line 445, in non_admin_user_password if users_Secret := list( ~~~~^ Secret.get( ^^^^^^^^^^^ ...<3 lines>... ) ^ ): ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 1690, in _get _resources = cls._prepare_resources(*args, client=_client, singular_name=singular_name, **kwargs) # type: ignore[misc] File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 881, in _prepare_resources return client.resources.get( ~~~~~~~~~~~~~~~~~~~~~ ...<2 lines>... **get_kwargs, ~~~~~~~~~~~~~ ).get(*args, **kwargs, timeout_seconds=cls.timeout_seconds) ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 112, in get return self.request('get', path, **kwargs) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 57, in inner raise api_exception(e) kubernetes.dynamic.exceptions.NotFoundError: 404 Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' Original traceback: File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) [conftest] 2026-06-14T13:39:03.059627Z [info ] Must-gather collection is enabled for /home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_gateway_deny_default.py::test_gateway_default_auth_in_gateway_namespace_and_accepted. [conftest] 2026-06-14T13:39:03.059812Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:39:03.060133Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:39:03.060258Z [info ] SELECT "OpenDataHubTestTable".start_time AS "OpenDataHubTestTable_start_time" FROM "OpenDataHubTestTable" WHERE "OpenDataHubTestTable".test_name = ? LIMIT ? OFFSET ? [sqlalchemy.engine.Engine] 2026-06-14T13:39:03.060406Z [info ] SELECT "OpenDataHubTestTable".start_time AS "OpenDataHubTestTable_start_time" FROM "OpenDataHubTestTable" WHERE "OpenDataHubTestTable".test_name = ? LIMIT ? OFFSET ? [sqlalchemy.engine.Engine] 2026-06-14T13:39:03.060548Z [info ] [cached since 1286s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_gateway_deny_default.py::test_gateway_default_auth_in_gateway_namespace_and_accepted', 1, 0) [sqlalchemy.engine.Engine] 2026-06-14T13:39:03.060677Z [info ] [cached since 1286s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_gateway_deny_default.py::test_gateway_default_auth_in_gateway_namespace_and_accepted', 1, 0) [sqlalchemy.engine.Engine] 2026-06-14T13:39:03.060790Z [info ] ROLLBACK  [sqlalchemy.engine.Engine] 2026-06-14T13:39:03.064230Z [info ] ROLLBACK  [sqlalchemy.engine.Engine] 2026-06-14T13:39:03.064360Z [info ] Trying to get client via new_client_from_config [ocp_resources.resource] 2026-06-14T13:39:03.095676Z [info ] Checking subscription rhods-operator [utilities.infra] name=utilities.infra 2026-06-14T13:39:03.099325Z [info ] Expected CSV: rhods-operator.3.4.0 [utilities.infra] name=utilities.infra 2026-06-14T13:39:03.106162Z [info ] kind: ClusterServiceVersion api version: operators.coreos.com/v1alpha1 [ocp_resources.resource] 2026-06-14T13:39:03.161072Z [info ] Running oc adm must-gather --dest-dir=/home/odh/opendatahub-tests/results/must-gather-collected/model_serving/maas_billing/maas_api_key/test_gateway_deny_default/TestGatewayDenyByDefault/test_gateway_default_auth_in_gateway_namespace_and_accepted/pytest_exception_interact --since=300s --image=registry.redhat.io/rhoai/odh-must-gather-rhel9@sha256:3de5ff9eee7e9b8e6650f61dbae642db3d3ccb813afb804397dc60d18f150ea6 command [pyhelper_utils.shell] 2026-06-14T13:44:24.193850Z [info ] /home/odh/opendatahub-tests/results/must-gather-collected/model_serving/maas_billing/maas_api_key/test_gateway_deny_default/TestGatewayDenyByDefault/test_gateway_default_auth_in_gateway_namespace_and_accepted/pytest_exception_interact/mg-1781444343.zip is collected successfully [utilities.must_gather_collector] name=utilities.must_gather_collector _ ERROR at setup of TestGatewayDenyByDefault.test_gateway_default_auth_in_gateway_namespace_and_accepted _ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) kubernetes.client.exceptions.ApiException: (404) Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 353, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 245, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 165, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 523, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1723, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 707, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 539, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1091, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1202, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 915, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/conftest.py", line 445, in non_admin_user_password if users_Secret := list( ~~~~^ Secret.get( ^^^^^^^^^^^ ...<3 lines>... ) ^ ): ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 1690, in _get _resources = cls._prepare_resources(*args, client=_client, singular_name=singular_name, **kwargs) # type: ignore[misc] File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 881, in _prepare_resources return client.resources.get( ~~~~~~~~~~~~~~~~~~~~~ ...<2 lines>... **get_kwargs, ~~~~~~~~~~~~~ ).get(*args, **kwargs, timeout_seconds=cls.timeout_seconds) ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 112, in get return self.request('get', path, **kwargs) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 57, in inner raise api_exception(e) kubernetes.dynamic.exceptions.NotFoundError: 404 Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' Original traceback: File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) _ 5 of 11 completed, 0 Pass, 0 Fail, 0 Skip, 0 XPass, 0 XFail, 5 Error, 0 ReRun _ tests/model_serving/maas_billing/maas_api_key/test_gateway_deny_default.py::TestGatewayDenyByDefault::test_unconfigured_model_denies_unauthenticated_request ------------------------------------------------------- TEARDOWN -------------------------------------------------------  -------------------------------- test_unconfigured_model_denies_unauthenticated_request -------------------------------- -------------------------------------------------------- SETUP -------------------------------------------------------- 2026-06-14T13:44:24.197205Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:44:24.197355Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:44:24.197529Z [info ] INSERT INTO "OpenDataHubTestTable" (test_name, start_time) VALUES (?, ?) [sqlalchemy.engine.Engine] 2026-06-14T13:44:24.197897Z [info ] INSERT INTO "OpenDataHubTestTable" (test_name, start_time) VALUES (?, ?) [sqlalchemy.engine.Engine] 2026-06-14T13:44:24.198053Z [info ] [cached since 1609s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_gateway_deny_default.py::test_unconfigured_model_denies_unauthenticated_request', 1781444664) [sqlalchemy.engine.Engine] 2026-06-14T13:44:24.199363Z [info ] [cached since 1609s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_gateway_deny_default.py::test_unconfigured_model_denies_unauthenticated_request', 1781444664) [sqlalchemy.engine.Engine] 2026-06-14T13:44:24.199532Z [info ] COMMIT  [sqlalchemy.engine.Engine] 2026-06-14T13:44:24.199653Z [info ] COMMIT  [sqlalchemy.engine.Engine] ERROR TEST: TestGatewayDenyByDefault.test_unconfigured_model_denies_unauthenticated_request [setup] STATUS: ERROR 2026-06-14T13:44:24.208795Z [error ] Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) kubernetes.client.exceptions.ApiException: (404) Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 353, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 245, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 165, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 523, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1723, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 707, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 539, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1091, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1202, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 915, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/conftest.py", line 445, in non_admin_user_password if users_Secret := list( ~~~~^ Secret.get( ^^^^^^^^^^^ ...<3 lines>... ) ^ ): ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 1690, in _get _resources = cls._prepare_resources(*args, client=_client, singular_name=singular_name, **kwargs) # type: ignore[misc] File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 881, in _prepare_resources return client.resources.get( ~~~~~~~~~~~~~~~~~~~~~ ...<2 lines>... **get_kwargs, ~~~~~~~~~~~~~ ).get(*args, **kwargs, timeout_seconds=cls.timeout_seconds) ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 112, in get return self.request('get', path, **kwargs) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 57, in inner raise api_exception(e) kubernetes.dynamic.exceptions.NotFoundError: 404 Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' Original traceback: File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) [conftest] 2026-06-14T13:44:24.209019Z [info ] Must-gather collection is enabled for /home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_gateway_deny_default.py::test_unconfigured_model_denies_unauthenticated_request. [conftest] 2026-06-14T13:44:24.209212Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:44:24.209360Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:44:24.209510Z [info ] SELECT "OpenDataHubTestTable".start_time AS "OpenDataHubTestTable_start_time" FROM "OpenDataHubTestTable" WHERE "OpenDataHubTestTable".test_name = ? LIMIT ? OFFSET ? [sqlalchemy.engine.Engine] 2026-06-14T13:44:24.209636Z [info ] SELECT "OpenDataHubTestTable".start_time AS "OpenDataHubTestTable_start_time" FROM "OpenDataHubTestTable" WHERE "OpenDataHubTestTable".test_name = ? LIMIT ? OFFSET ? [sqlalchemy.engine.Engine] 2026-06-14T13:44:24.212597Z [info ] [cached since 1607s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_gateway_deny_default.py::test_unconfigured_model_denies_unauthenticated_request', 1, 0) [sqlalchemy.engine.Engine] 2026-06-14T13:44:24.212744Z [info ] [cached since 1607s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_api_key/test_gateway_deny_default.py::test_unconfigured_model_denies_unauthenticated_request', 1, 0) [sqlalchemy.engine.Engine] 2026-06-14T13:44:24.212860Z [info ] ROLLBACK  [sqlalchemy.engine.Engine] 2026-06-14T13:44:24.213153Z [info ] ROLLBACK  [sqlalchemy.engine.Engine] 2026-06-14T13:44:24.213267Z [info ] Trying to get client via new_client_from_config [ocp_resources.resource] 2026-06-14T13:44:24.251191Z [info ] Checking subscription rhods-operator [utilities.infra] name=utilities.infra 2026-06-14T13:44:24.255618Z [info ] Expected CSV: rhods-operator.3.4.0 [utilities.infra] name=utilities.infra 2026-06-14T13:44:24.262157Z [info ] kind: ClusterServiceVersion api version: operators.coreos.com/v1alpha1 [ocp_resources.resource] 2026-06-14T13:44:24.322538Z [info ] Running oc adm must-gather --dest-dir=/home/odh/opendatahub-tests/results/must-gather-collected/model_serving/maas_billing/maas_api_key/test_gateway_deny_default/TestGatewayDenyByDefault/test_unconfigured_model_denies_unauthenticated_request/pytest_exception_interact --since=300s --image=registry.redhat.io/rhoai/odh-must-gather-rhel9@sha256:3de5ff9eee7e9b8e6650f61dbae642db3d3ccb813afb804397dc60d18f150ea6 command [pyhelper_utils.shell] 2026-06-14T13:49:42.663824Z [info ] /home/odh/opendatahub-tests/results/must-gather-collected/model_serving/maas_billing/maas_api_key/test_gateway_deny_default/TestGatewayDenyByDefault/test_unconfigured_model_denies_unauthenticated_request/pytest_exception_interact/mg-1781444664.zip is collected successfully [utilities.must_gather_collector] name=utilities.must_gather_collector _ ERROR at setup of TestGatewayDenyByDefault.test_unconfigured_model_denies_unauthenticated_request _ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) kubernetes.client.exceptions.ApiException: (404) Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 353, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 245, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 165, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 523, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1723, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 707, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 539, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1091, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1202, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 915, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/conftest.py", line 445, in non_admin_user_password if users_Secret := list( ~~~~^ Secret.get( ^^^^^^^^^^^ ...<3 lines>... ) ^ ): ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 1690, in _get _resources = cls._prepare_resources(*args, client=_client, singular_name=singular_name, **kwargs) # type: ignore[misc] File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 881, in _prepare_resources return client.resources.get( ~~~~~~~~~~~~~~~~~~~~~ ...<2 lines>... **get_kwargs, ~~~~~~~~~~~~~ ).get(*args, **kwargs, timeout_seconds=cls.timeout_seconds) ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 112, in get return self.request('get', path, **kwargs) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 57, in inner raise api_exception(e) kubernetes.dynamic.exceptions.NotFoundError: 404 Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' Original traceback: File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) _ 6 of 11 completed, 0 Pass, 0 Fail, 0 Skip, 0 XPass, 0 XFail, 6 Error, 0 ReRun _ tests/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement.py::TestMaaSAuthPolicyEnforcementTinyLlama::test_authorized_user_gets_200[ocp_token_for_actor0] ------------------------------------------------------- TEARDOWN -------------------------------------------------------  --------------------------------- test_authorized_user_gets_200[ocp_token_for_actor0] --------------------------------- -------------------------------------------------------- SETUP -------------------------------------------------------- 2026-06-14T13:49:42.666497Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:49:42.666619Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:49:42.666735Z [info ] INSERT INTO "OpenDataHubTestTable" (test_name, start_time) VALUES (?, ?) [sqlalchemy.engine.Engine] 2026-06-14T13:49:42.666860Z [info ] INSERT INTO "OpenDataHubTestTable" (test_name, start_time) VALUES (?, ?) [sqlalchemy.engine.Engine] 2026-06-14T13:49:42.667022Z [info ] [cached since 1927s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement.py::test_authorized_user_gets_200[ocp_token_for_actor0]', 1781444982) [sqlalchemy.engine.Engine] 2026-06-14T13:49:42.667154Z [info ] [cached since 1927s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement.py::test_authorized_user_gets_200[ocp_token_for_actor0]', 1781444982) [sqlalchemy.engine.Engine] 2026-06-14T13:49:42.667623Z [info ] COMMIT  [sqlalchemy.engine.Engine] 2026-06-14T13:49:42.667747Z [info ] COMMIT  [sqlalchemy.engine.Engine] ERROR TEST: TestMaaSAuthPolicyEnforcementTinyLlama.test_authorized_user_gets_200[ocp_token_for_actor0] [setup] STATUS: ERROR 2026-06-14T13:49:42.677223Z [error ] Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) kubernetes.client.exceptions.ApiException: (404) Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 353, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 245, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 165, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 523, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1723, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 707, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 539, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1091, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1202, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 915, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/conftest.py", line 445, in non_admin_user_password if users_Secret := list( ~~~~^ Secret.get( ^^^^^^^^^^^ ...<3 lines>... ) ^ ): ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 1690, in _get _resources = cls._prepare_resources(*args, client=_client, singular_name=singular_name, **kwargs) # type: ignore[misc] File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 881, in _prepare_resources return client.resources.get( ~~~~~~~~~~~~~~~~~~~~~ ...<2 lines>... **get_kwargs, ~~~~~~~~~~~~~ ).get(*args, **kwargs, timeout_seconds=cls.timeout_seconds) ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 112, in get return self.request('get', path, **kwargs) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 57, in inner raise api_exception(e) kubernetes.dynamic.exceptions.NotFoundError: 404 Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' Original traceback: File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) [conftest] 2026-06-14T13:49:42.677441Z [info ] Must-gather collection is enabled for /home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement.py::test_authorized_user_gets_200[ocp_token_for_actor0]. [conftest] 2026-06-14T13:49:42.677601Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:49:42.677731Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:49:42.677846Z [info ] SELECT "OpenDataHubTestTable".start_time AS "OpenDataHubTestTable_start_time" FROM "OpenDataHubTestTable" WHERE "OpenDataHubTestTable".test_name = ? LIMIT ? OFFSET ? [sqlalchemy.engine.Engine] 2026-06-14T13:49:42.681224Z [info ] SELECT "OpenDataHubTestTable".start_time AS "OpenDataHubTestTable_start_time" FROM "OpenDataHubTestTable" WHERE "OpenDataHubTestTable".test_name = ? LIMIT ? OFFSET ? [sqlalchemy.engine.Engine] 2026-06-14T13:49:42.681403Z [info ] [cached since 1926s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement.py::test_authorized_user_gets_200[ocp_token_for_actor0]', 1, 0) [sqlalchemy.engine.Engine] 2026-06-14T13:49:42.681564Z [info ] [cached since 1926s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement.py::test_authorized_user_gets_200[ocp_token_for_actor0]', 1, 0) [sqlalchemy.engine.Engine] 2026-06-14T13:49:42.681690Z [info ] ROLLBACK  [sqlalchemy.engine.Engine] 2026-06-14T13:49:42.681798Z [info ] ROLLBACK  [sqlalchemy.engine.Engine] 2026-06-14T13:49:42.681923Z [info ] Trying to get client via new_client_from_config [ocp_resources.resource] 2026-06-14T13:49:42.714273Z [info ] Checking subscription rhods-operator [utilities.infra] name=utilities.infra 2026-06-14T13:49:42.718985Z [info ] Expected CSV: rhods-operator.3.4.0 [utilities.infra] name=utilities.infra 2026-06-14T13:49:42.725644Z [info ] kind: ClusterServiceVersion api version: operators.coreos.com/v1alpha1 [ocp_resources.resource] 2026-06-14T13:49:42.781731Z [info ] Running oc adm must-gather --dest-dir=/home/odh/opendatahub-tests/results/must-gather-collected/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement/TestMaaSAuthPolicyEnforcementTinyLlama/test_authorized_user_gets_200_ocp_token_for_actor0_/pytest_exception_interact --since=300s --image=registry.redhat.io/rhoai/odh-must-gather-rhel9@sha256:3de5ff9eee7e9b8e6650f61dbae642db3d3ccb813afb804397dc60d18f150ea6 command [pyhelper_utils.shell] 2026-06-14T13:55:01.342251Z [info ] /home/odh/opendatahub-tests/results/must-gather-collected/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement/TestMaaSAuthPolicyEnforcementTinyLlama/test_authorized_user_gets_200_ocp_token_for_actor0_/pytest_exception_interact/mg-1781444982.zip is collected successfully [utilities.must_gather_collector] name=utilities.must_gather_collector ------------------------------------------------------- TEARDOWN ------------------------------------------------------- _ ERROR at setup of TestMaaSAuthPolicyEnforcementTinyLlama.test_authorized_user_gets_200[ocp_token_for_actor0] _ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) kubernetes.client.exceptions.ApiException: (404) Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 353, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 245, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 165, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 523, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1723, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 707, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 539, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1091, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1202, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 915, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/conftest.py", line 445, in non_admin_user_password if users_Secret := list( ~~~~^ Secret.get( ^^^^^^^^^^^ ...<3 lines>... ) ^ ): ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 1690, in _get _resources = cls._prepare_resources(*args, client=_client, singular_name=singular_name, **kwargs) # type: ignore[misc] File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 881, in _prepare_resources return client.resources.get( ~~~~~~~~~~~~~~~~~~~~~ ...<2 lines>... **get_kwargs, ~~~~~~~~~~~~~ ).get(*args, **kwargs, timeout_seconds=cls.timeout_seconds) ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 112, in get return self.request('get', path, **kwargs) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 57, in inner raise api_exception(e) kubernetes.dynamic.exceptions.NotFoundError: 404 Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' Original traceback: File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) _ 7 of 11 completed, 0 Pass, 0 Fail, 0 Skip, 0 XPass, 0 XFail, 7 Error, 0 ReRun _ tests/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement.py::TestMaaSAuthPolicyEnforcementTinyLlama::test_no_auth_header_gets_401  --------------------------------------------- test_no_auth_header_gets_401 --------------------------------------------- -------------------------------------------------------- SETUP -------------------------------------------------------- 2026-06-14T13:55:01.344945Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:55:01.345128Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:55:01.345285Z [info ] INSERT INTO "OpenDataHubTestTable" (test_name, start_time) VALUES (?, ?) [sqlalchemy.engine.Engine] 2026-06-14T13:55:01.345433Z [info ] INSERT INTO "OpenDataHubTestTable" (test_name, start_time) VALUES (?, ?) [sqlalchemy.engine.Engine] 2026-06-14T13:55:01.345599Z [info ] [cached since 2246s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement.py::test_no_auth_header_gets_401', 1781445301) [sqlalchemy.engine.Engine] 2026-06-14T13:55:01.345757Z [info ] [cached since 2246s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement.py::test_no_auth_header_gets_401', 1781445301) [sqlalchemy.engine.Engine] 2026-06-14T13:55:01.347125Z [info ] COMMIT  [sqlalchemy.engine.Engine] 2026-06-14T13:55:01.347297Z [info ] COMMIT  [sqlalchemy.engine.Engine] ERROR TEST: TestMaaSAuthPolicyEnforcementTinyLlama.test_no_auth_header_gets_401 [setup] STATUS: ERROR 2026-06-14T13:55:01.352542Z [error ] Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) kubernetes.client.exceptions.ApiException: (404) Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 353, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 245, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 165, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 523, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1723, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 707, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 539, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1091, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1202, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 915, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/conftest.py", line 445, in non_admin_user_password if users_Secret := list( ~~~~^ Secret.get( ^^^^^^^^^^^ ...<3 lines>... ) ^ ): ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 1690, in _get _resources = cls._prepare_resources(*args, client=_client, singular_name=singular_name, **kwargs) # type: ignore[misc] File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 881, in _prepare_resources return client.resources.get( ~~~~~~~~~~~~~~~~~~~~~ ...<2 lines>... **get_kwargs, ~~~~~~~~~~~~~ ).get(*args, **kwargs, timeout_seconds=cls.timeout_seconds) ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 112, in get return self.request('get', path, **kwargs) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 57, in inner raise api_exception(e) kubernetes.dynamic.exceptions.NotFoundError: 404 Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' Original traceback: File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) [conftest] 2026-06-14T13:55:01.352724Z [info ] Must-gather collection is enabled for /home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement.py::test_no_auth_header_gets_401. [conftest] 2026-06-14T13:55:01.352867Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:55:01.355929Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T13:55:01.356103Z [info ] SELECT "OpenDataHubTestTable".start_time AS "OpenDataHubTestTable_start_time" FROM "OpenDataHubTestTable" WHERE "OpenDataHubTestTable".test_name = ? LIMIT ? OFFSET ? [sqlalchemy.engine.Engine] 2026-06-14T13:55:01.356263Z [info ] SELECT "OpenDataHubTestTable".start_time AS "OpenDataHubTestTable_start_time" FROM "OpenDataHubTestTable" WHERE "OpenDataHubTestTable".test_name = ? LIMIT ? OFFSET ? [sqlalchemy.engine.Engine] 2026-06-14T13:55:01.356432Z [info ] [cached since 2245s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement.py::test_no_auth_header_gets_401', 1, 0) [sqlalchemy.engine.Engine] 2026-06-14T13:55:01.356589Z [info ] [cached since 2245s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement.py::test_no_auth_header_gets_401', 1, 0) [sqlalchemy.engine.Engine] 2026-06-14T13:55:01.356734Z [info ] ROLLBACK  [sqlalchemy.engine.Engine] 2026-06-14T13:55:01.356871Z [info ] ROLLBACK  [sqlalchemy.engine.Engine] 2026-06-14T13:55:01.357048Z [info ] Trying to get client via new_client_from_config [ocp_resources.resource] 2026-06-14T13:55:01.396114Z [info ] Checking subscription rhods-operator [utilities.infra] name=utilities.infra 2026-06-14T13:55:01.399977Z [info ] Expected CSV: rhods-operator.3.4.0 [utilities.infra] name=utilities.infra 2026-06-14T13:55:01.406657Z [info ] kind: ClusterServiceVersion api version: operators.coreos.com/v1alpha1 [ocp_resources.resource] 2026-06-14T13:55:01.466247Z [info ] Running oc adm must-gather --dest-dir=/home/odh/opendatahub-tests/results/must-gather-collected/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement/TestMaaSAuthPolicyEnforcementTinyLlama/test_no_auth_header_gets_401/pytest_exception_interact --since=300s --image=registry.redhat.io/rhoai/odh-must-gather-rhel9@sha256:3de5ff9eee7e9b8e6650f61dbae642db3d3ccb813afb804397dc60d18f150ea6 command [pyhelper_utils.shell] 2026-06-14T14:00:21.422627Z [info ] /home/odh/opendatahub-tests/results/must-gather-collected/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement/TestMaaSAuthPolicyEnforcementTinyLlama/test_no_auth_header_gets_401/pytest_exception_interact/mg-1781445301.zip is collected successfully [utilities.must_gather_collector] name=utilities.must_gather_collector _ ERROR at setup of TestMaaSAuthPolicyEnforcementTinyLlama.test_no_auth_header_gets_401 _ Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) kubernetes.client.exceptions.ApiException: (404) Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 353, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 245, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 165, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 523, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1723, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 707, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 539, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1091, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1202, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 915, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/conftest.py", line 445, in non_admin_user_password if users_Secret := list( ~~~~^ Secret.get( ^^^^^^^^^^^ ...<3 lines>... ) ^ ): ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 1690, in _get _resources = cls._prepare_resources(*args, client=_client, singular_name=singular_name, **kwargs) # type: ignore[misc] File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 881, in _prepare_resources return client.resources.get( ~~~~~~~~~~~~~~~~~~~~~ ...<2 lines>... **get_kwargs, ~~~~~~~~~~~~~ ).get(*args, **kwargs, timeout_seconds=cls.timeout_seconds) ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 112, in get return self.request('get', path, **kwargs) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 57, in inner raise api_exception(e) kubernetes.dynamic.exceptions.NotFoundError: 404 Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' Original traceback: File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) _ 8 of 11 completed, 0 Pass, 0 Fail, 0 Skip, 0 XPass, 0 XFail, 8 Error, 0 ReRun _ tests/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement.py::TestMaaSAuthPolicyEnforcementTinyLlama::test_invalid_token_gets_401 ------------------------------------------------------- TEARDOWN -------------------------------------------------------  --------------------------------------------- test_invalid_token_gets_401 --------------------------------------------- -------------------------------------------------------- SETUP -------------------------------------------------------- 2026-06-14T14:00:21.426135Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T14:00:21.426282Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T14:00:21.426426Z [info ] INSERT INTO "OpenDataHubTestTable" (test_name, start_time) VALUES (?, ?) [sqlalchemy.engine.Engine] 2026-06-14T14:00:21.426785Z [info ] INSERT INTO "OpenDataHubTestTable" (test_name, start_time) VALUES (?, ?) [sqlalchemy.engine.Engine] 2026-06-14T14:00:21.428173Z [info ] [cached since 2566s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement.py::test_invalid_token_gets_401', 1781445621) [sqlalchemy.engine.Engine] 2026-06-14T14:00:21.428335Z [info ] [cached since 2566s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement.py::test_invalid_token_gets_401', 1781445621) [sqlalchemy.engine.Engine] 2026-06-14T14:00:21.428474Z [info ] COMMIT  [sqlalchemy.engine.Engine] 2026-06-14T14:00:21.428615Z [info ] COMMIT  [sqlalchemy.engine.Engine] ERROR TEST: TestMaaSAuthPolicyEnforcementTinyLlama.test_invalid_token_gets_401 [setup] STATUS: ERROR 2026-06-14T14:00:21.437519Z [error ] Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) kubernetes.client.exceptions.ApiException: (404) Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 353, in from_call result: TResult | None = func() ~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 245, in lambda: runtest_hook(item=item, **kwds), ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_hooks.py", line 512, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall raise exception File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 139, in _multicall teardown.throw(exception) ~~~~~~~~~~~~~~^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/capture.py", line 895, in pytest_runtest_setup return (yield) ^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall res = hook_impl.function(*args) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 165, in pytest_runtest_setup item.session._setupstate.setup(item) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/runner.py", line 523, in setup col.setup() ~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/python.py", line 1723, in setup self._request._fillfixtures() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 707, in _fillfixtures item.funcargs[argname] = self.getfixturevalue(argname) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 539, in getfixturevalue fixturedef = self._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1068, in execute fixturedef = request._get_active_fixturedef(argname) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 627, in _get_active_fixturedef fixturedef.execute(request=subrequest) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1091, in execute raise exc.with_traceback(exc_tb) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 1202, in pytest_fixture_setup result = call_fixture_func(fixturefunc, request, kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/_pytest/fixtures.py", line 915, in call_fixture_func fixture_result = fixturefunc(**kwargs) File "/home/odh/opendatahub-tests/tests/conftest.py", line 445, in non_admin_user_password if users_Secret := list( ~~~~^ Secret.get( ^^^^^^^^^^^ ...<3 lines>... ) ^ ): ^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 1690, in _get _resources = cls._prepare_resources(*args, client=_client, singular_name=singular_name, **kwargs) # type: ignore[misc] File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/ocp_resources/resource.py", line 881, in _prepare_resources return client.resources.get( ~~~~~~~~~~~~~~~~~~~~~ ...<2 lines>... **get_kwargs, ~~~~~~~~~~~~~ ).get(*args, **kwargs, timeout_seconds=cls.timeout_seconds) ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 112, in get return self.request('get', path, **kwargs) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 57, in inner raise api_exception(e) kubernetes.dynamic.exceptions.NotFoundError: 404 Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'bc5fad7e-f4f3-4f0d-9335-d852dd07a0b1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'X-Kubernetes-Pf-Flowschema-Uid': '5b0fc478-8963-41dd-8607-9275c943c7fa', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd41002bd-961e-4bc1-8269-04737b11ea8f', 'Date': 'Sun, 14 Jun 2026 13:17:36 GMT', 'Content-Length': '190'}) HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \\"openldap\\" not found","reason":"NotFound","details":{"name":"openldap","kind":"secrets"},"code":404}\n' Original traceback: File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 55, in inner resp = func(self, *args, **kwargs) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/dynamic/client.py", line 277, in request api_response = self.client.call_api( path, ...<11 lines>... _request_timeout=params.get('_request_timeout') ) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ path_params, query_params, header_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... _return_http_data_only, collection_formats, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content, _request_timeout, _host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( method, url, query_params=query_params, headers=header_params, post_params=post_params, body=body, _preload_content=_preload_content, _request_timeout=_request_timeout) File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ~~~~~~~~~~~~~~~~~~~~^^^^^ query_params=query_params, ^^^^^^^^^^^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ headers=headers) ^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ~~~~~~~~~~~~^^^^^^^^^^^^ headers=headers, ^^^^^^^^^^^^^^^^ _preload_content=_preload_content, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ _request_timeout=_request_timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ query_params=query_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/odh/opendatahub-tests/.venv/lib64/python3.14/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) [conftest] 2026-06-14T14:00:21.437754Z [info ] Must-gather collection is enabled for /home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement.py::test_invalid_token_gets_401. [conftest] 2026-06-14T14:00:21.437919Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T14:00:21.441589Z [info ] BEGIN (implicit)  [sqlalchemy.engine.Engine] 2026-06-14T14:00:21.441739Z [info ] SELECT "OpenDataHubTestTable".start_time AS "OpenDataHubTestTable_start_time" FROM "OpenDataHubTestTable" WHERE "OpenDataHubTestTable".test_name = ? LIMIT ? OFFSET ? [sqlalchemy.engine.Engine] 2026-06-14T14:00:21.441866Z [info ] SELECT "OpenDataHubTestTable".start_time AS "OpenDataHubTestTable_start_time" FROM "OpenDataHubTestTable" WHERE "OpenDataHubTestTable".test_name = ? LIMIT ? OFFSET ? [sqlalchemy.engine.Engine] 2026-06-14T14:00:21.442022Z [info ] [cached since 2565s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement.py::test_invalid_token_gets_401', 1, 0) [sqlalchemy.engine.Engine] 2026-06-14T14:00:21.442141Z [info ] [cached since 2565s ago] ('/home/odh/opendatahub-tests/tests/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement.py::test_invalid_token_gets_401', 1, 0) [sqlalchemy.engine.Engine] 2026-06-14T14:00:21.442252Z [info ] ROLLBACK  [sqlalchemy.engine.Engine] 2026-06-14T14:00:21.442355Z [info ] ROLLBACK  [sqlalchemy.engine.Engine] 2026-06-14T14:00:21.442455Z [info ] Trying to get client via new_client_from_config [ocp_resources.resource] 2026-06-14T14:00:21.470561Z [info ] Checking subscription rhods-operator [utilities.infra] name=utilities.infra 2026-06-14T14:00:21.475592Z [info ] Expected CSV: rhods-operator.3.4.0 [utilities.infra] name=utilities.infra 2026-06-14T14:00:21.484303Z [info ] kind: ClusterServiceVersion api version: operators.coreos.com/v1alpha1 [ocp_resources.resource] 2026-06-14T14:00:21.544656Z [info ] Running oc adm must-gather --dest-dir=/home/odh/opendatahub-tests/results/must-gather-collected/model_serving/maas_billing/maas_subscription/test_maas_auth_enforcement/TestMaaSAuthPolicyEnforcementTinyLlama/test_invalid_token_gets_401/pytest_exception_interact --since=300s --image=registry.redhat.io/rhoai/odh-must-gather-rhel9@sha256:3de5ff9eee7e9b8e6650f61dbae642db3d3ccb813afb804397dc60d18f150ea6 command [pyhelper_utils.shell]