/workspace/source/test/e2e/tests/test_api_keys.py:930: CronJob maas-api-key-cleanup not found in opendatahub: Error from server (NotFound): cronjobs.batch "maas-api-key-cleanup" not found/workspace/source/test/e2e/tests/test_api_keys.py:977: NetworkPolicy maas-api-cleanup-restrict not found in opendatahub: Error from server (NotFound): networkpolicies.networking.k8s.io "maas-api-cleanup-restrict" not found/workspace/source/test/e2e/tests/test_api_keys.py:1096: Cannot find maas-api pod in opendatahub: error: error executing jsonpath "{.items[0].metadata.name}": Error executing template: array index out of bounds: index 0, length 0. Printing more information for debugging the template: template was: {.items[0].metadata.name} object given to jsonpath engine was: map[string]interface {}{"apiVersion":"v1", "items":[]interface {}{}, "kind":"List", "metadata":map[string]interface {}{"resourceVersion":""}}/workspace/source/test/e2e/tests/test_namespace_scoping.py:212: test_namespace_scoping validates single-tenant dormant mode; skipped when ENABLE_TENANT_NAMESPACE_DISCOVERY=true/workspace/source/test/e2e/tests/test_namespace_scoping.py:245: test_namespace_scoping validates single-tenant dormant mode; skipped when ENABLE_TENANT_NAMESPACE_DISCOVERY=true/workspace/source/test/e2e/tests/test_namespace_scoping.py:283: test_namespace_scoping validates single-tenant dormant mode; skipped when ENABLE_TENANT_NAMESPACE_DISCOVERY=true/workspace/source/test/e2e/tests/test_namespace_scoping.py:320: test_namespace_scoping validates single-tenant dormant mode; skipped when ENABLE_TENANT_NAMESPACE_DISCOVERY=true/workspace/source/test/e2e/tests/test_namespace_scoping.py:377: test_namespace_scoping validates single-tenant dormant mode; skipped when ENABLE_TENANT_NAMESPACE_DISCOVERY=true/workspace/source/test/e2e/tests/test_namespace_scoping.py:453: test_namespace_scoping validates single-tenant dormant mode; skipped when ENABLE_TENANT_NAMESPACE_DISCOVERY=true/workspace/source/test/e2e/tests/test_subscription.py:1056: gateway-only mode: per-model AuthPolicy is not createdself = <test_tenant.TestTenantLifecycle object at 0x7fc30c7e4dc0> def test_tenant_ready_and_phase_healthy(self): st = _wait_tenant_ready() > assert st is not None, "Tenant Ready did not become True in time." E AssertionError: Tenant Ready did not become True in time. E assert None is not None test/e2e/tests/test_tenant.py:115: AssertionErrorself = <test_tenant.TestTenantLifecycle object at 0x7fc30c7e4d00> def test_payload_processing_deployed_with_active_tenant(self): st = _wait_tenant_ready() > assert st is not None, "Tenant not Ready; skip workload checks." E AssertionError: Tenant not Ready; skip workload checks. E assert None is not None test/e2e/tests/test_tenant.py:124: AssertionError/workspace/source/test/e2e/tests/test_tenant_namespace_discovery.py:186: OIDC_ISSUER_URL not set; per-tenant OIDC E2E requires external OIDC deployself = <test_tenant_namespace_discovery.TestTenantNamespaceDiscovery object at 0x7fc30c78abe0> def test_tenant_admin_rbac_is_namespace_scoped(self): """1.6: Tenant-admin Role from AITenant bootstrap is scoped to the tenant namespace.""" case = new_discovery_case() sa_name = f"e2e-mt-rbac-{case['suffix']}" role_name = f"aitenant-{case['tenant_label_name']}-tenant-admin" other_ns = f"e2e-mt-other-{case['suffix']}" try: apply_gateway_fixture(case["gateway_name"], fixture_label=case["tenant_label_name"]) > bootstrap_aitenant_tenant(case) test/e2e/tests/test_tenant_namespace_discovery.py:260: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ test/e2e/tests/multitenancy_helpers.py:773: in bootstrap_aitenant_tenant wait_for_httproute_accepted( test/e2e/tests/multitenancy_helpers.py:674: in wait_for_httproute_accepted return wait_for_json("httproute", route_name, namespace, predicate=_predicate, timeout=timeout, interval=interval) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ kind = 'httproute', name = 'maas-api-route-e2e-mt-9099fcfd' namespace = 'odh-ai-gateway-infra' def wait_for_json( kind: str, name: str, namespace: Optional[str] = None, *, predicate=None, timeout: int = 180, interval: int = 5, ) -> dict: deadline = time.time() + timeout last_obj = None while time.time() < deadline: obj = get_json_or_none(kind, name, namespace) if obj is not None: last_obj = obj if predicate is None or predicate(obj): return obj time.sleep(interval) > raise AssertionError( f"{kind}/{name} in {namespace or '<cluster>'} did not satisfy condition. Last object: {last_obj}" ) E AssertionError: httproute/maas-api-route-e2e-mt-9099fcfd in odh-ai-gateway-infra did not satisfy condition. Last object: None test/e2e/tests/multitenancy_helpers.py:227: AssertionError/workspace/source/test/e2e/tests/test_tenant_namespace_discovery.py:358: Dormant-mode test mutates controller flags; set ENABLE_TENANT_DISCOVERY_DORMANT_E2E=true/workspace/source/test/e2e/tests/test_gateway_scoped_authpolicy.py:170: gateway-default-auth not present (maas-gateway-auth is active); scoping is validated by unit tests@pytest.fixture(scope="module") def tenant_cases(): require_aitenant_crd() case_a = new_named_tenant_case("e2e-api-a") case_b = new_named_tenant_case("e2e-api-b") try: for case in (case_a, case_b): > bootstrap_aitenant_tenant(case) test/e2e/tests/test_multi_tenant_maas_api.py:46: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ test/e2e/tests/multitenancy_helpers.py:773: in bootstrap_aitenant_tenant wait_for_httproute_accepted( test/e2e/tests/multitenancy_helpers.py:674: in wait_for_httproute_accepted return wait_for_json("httproute", route_name, namespace, predicate=_predicate, timeout=timeout, interval=interval) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ kind = 'httproute', name = 'maas-api-route-e2e-api-a-8d5c89' namespace = 'odh-ai-gateway-infra' def wait_for_json( kind: str, name: str, namespace: Optional[str] = None, *, predicate=None, timeout: int = 180, interval: int = 5, ) -> dict: deadline = time.time() + timeout last_obj = None while time.time() < deadline: obj = get_json_or_none(kind, name, namespace) if obj is not None: last_obj = obj if predicate is None or predicate(obj): return obj time.sleep(interval) > raise AssertionError( f"{kind}/{name} in {namespace or '<cluster>'} did not satisfy condition. Last object: {last_obj}" ) E AssertionError: httproute/maas-api-route-e2e-api-a-8d5c89 in odh-ai-gateway-infra did not satisfy condition. Last object: None test/e2e/tests/multitenancy_helpers.py:227: AssertionError@pytest.fixture(scope="module") def tenant_cases(): require_aitenant_crd() case_a = new_named_tenant_case("e2e-api-a") case_b = new_named_tenant_case("e2e-api-b") try: for case in (case_a, case_b): > bootstrap_aitenant_tenant(case) test/e2e/tests/test_multi_tenant_maas_api.py:46: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ test/e2e/tests/multitenancy_helpers.py:773: in bootstrap_aitenant_tenant wait_for_httproute_accepted( test/e2e/tests/multitenancy_helpers.py:674: in wait_for_httproute_accepted return wait_for_json("httproute", route_name, namespace, predicate=_predicate, timeout=timeout, interval=interval) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ kind = 'httproute', name = 'maas-api-route-e2e-api-a-8d5c89' namespace = 'odh-ai-gateway-infra' def wait_for_json( kind: str, name: str, namespace: Optional[str] = None, *, predicate=None, timeout: int = 180, interval: int = 5, ) -> dict: deadline = time.time() + timeout last_obj = None while time.time() < deadline: obj = get_json_or_none(kind, name, namespace) if obj is not None: last_obj = obj if predicate is None or predicate(obj): return obj time.sleep(interval) > raise AssertionError( f"{kind}/{name} in {namespace or '<cluster>'} did not satisfy condition. Last object: {last_obj}" ) E AssertionError: httproute/maas-api-route-e2e-api-a-8d5c89 in odh-ai-gateway-infra did not satisfy condition. Last object: None test/e2e/tests/multitenancy_helpers.py:227: AssertionError