/workspace/source/test/e2e/tests/test_namespace_scoping.py:212: test_namespace_scoping validates single-tenant dormant mode; skipped when ENABLE_TENANT_NAMESPACE_DISCOVERY=true/workspace/source/test/e2e/tests/test_namespace_scoping.py:245: test_namespace_scoping validates single-tenant dormant mode; skipped when ENABLE_TENANT_NAMESPACE_DISCOVERY=true/workspace/source/test/e2e/tests/test_namespace_scoping.py:283: test_namespace_scoping validates single-tenant dormant mode; skipped when ENABLE_TENANT_NAMESPACE_DISCOVERY=true/workspace/source/test/e2e/tests/test_namespace_scoping.py:320: test_namespace_scoping validates single-tenant dormant mode; skipped when ENABLE_TENANT_NAMESPACE_DISCOVERY=true/workspace/source/test/e2e/tests/test_namespace_scoping.py:377: test_namespace_scoping validates single-tenant dormant mode; skipped when ENABLE_TENANT_NAMESPACE_DISCOVERY=true/workspace/source/test/e2e/tests/test_namespace_scoping.py:453: test_namespace_scoping validates single-tenant dormant mode; skipped when ENABLE_TENANT_NAMESPACE_DISCOVERY=true/workspace/source/test/e2e/tests/test_subscription.py:1054: gateway-only mode: per-model AuthPolicy is not createdself = <test_models_endpoint.TestModelsEndpoint object at 0x7f11aa70dca0> def test_deduplication_same_model_multiple_refs(self): """ Test 6: Same modelRef listed twice should deduplicate to 1 entry. Creates a subscription with the SAME modelRef listed TWICE (different rate limits). The API deduplicates by (model ID, URL) and returns only 1 entry since both references point to the same backend service. The response includes subscription information showing which subscription(s) provide access to the model. """ log.info("Test 6: Same modelRef twice should deduplicate (INTENDED behavior)") sa_name = "e2e-models-dedup-sa" sa_ns = "default" maas_ns = _ns() subscription_name = "e2e-dedup-subscription" auth_policy_name = "e2e-dedup-auth" api_key = None try: # Create SA with its own token sa_token = _create_sa_token(sa_name, namespace=sa_ns) sa_user = _sa_to_user(sa_name, namespace=sa_ns) # Create auth policy that grants access to the model log.info(f"Creating auth policy with access to {MODEL_REF}") auth_policy_cr = { "apiVersion": "maas.opendatahub.io/v1alpha1", "kind": "MaaSAuthPolicy", "metadata": { "name": auth_policy_name, "namespace": maas_ns, }, "spec": { "modelRefs": [{"name": MODEL_REF, "namespace": MODEL_NAMESPACE}], "subjects": { "users": [sa_user], "groups": [], }, }, } subprocess.run( ["oc", "apply", "-f", "-"], input=json.dumps(auth_policy_cr), text=True, check=True, ) # Create subscription with the SAME model ref TWICE (guaranteed duplicates) log.info(f"Creating subscription with {MODEL_REF} listed twice (to test deduplication)") subscription_cr = { "apiVersion": "maas.opendatahub.io/v1alpha1", "kind": "MaaSSubscription", "metadata": { "name": subscription_name, "namespace": maas_ns, }, "spec": { "owner": { "users": [sa_user], "groups": [], }, "modelRefs": [ { "name": MODEL_REF, "namespace": MODEL_NAMESPACE, "tokenRateLimits": [{"limit": 100, "window": "1m"}], }, { "name": MODEL_REF, # Same model ref again - guarantees duplicate "namespace": MODEL_NAMESPACE, "tokenRateLimits": [{"limit": 200, "window": "1m"}], }, ], }, } subprocess.run( ["oc", "apply", "-f", "-"], input=json.dumps(subscription_cr), text=True, check=True, ) # Wait for subscription to reconcile before creating API key _wait_for_maas_subscription_phase(subscription_name, namespace=maas_ns) # Create API key bound to our test subscription api_key = _create_api_key(sa_token, name="e2e-dedup-test-key", subscription=subscription_name) # Wait for reconciliation _wait_reconcile() # Query /v1/models with our custom subscription log.info(f"Querying /v1/models with subscription: {subscription_name}") r = _get_models_with_gateway_retry( headers={ "Authorization": f"Bearer {api_key}", "x-maas-subscription": subscription_name, }, ) assert r.status_code == 200, f"Expected 200, got {r.status_code}: {r.text}" data = r.json() models = data.get("data") or [] # Models should be a list assert isinstance(models, list), "Models should be a list" # Get model IDs from response model_ids = [m["id"] for m in models] unique_ids = set(model_ids) log.info(f"#x1F4CA API Response: {len(models)} total model(s), {len(unique_ids)} unique ID(s)") log.info(f" Model IDs: {model_ids}") log.info(f" Unique IDs: {unique_ids}") # Should all be the same model ID (we only referenced one modelRef) > assert len(unique_ids) == 1, \ f"Expected only 1 unique model ID (same modelRef listed twice), got {len(unique_ids)}: {unique_ids}" E AssertionError: Expected only 1 unique model ID (same modelRef listed twice), got 0: set() E assert 0 == 1 E + where 0 = len(set()) test/e2e/tests/test_models_endpoint.py:693: AssertionError/workspace/source/test/e2e/tests/test_tenant_namespace_discovery.py:186: OIDC_ISSUER_URL not set; per-tenant OIDC E2E requires external OIDC deploy/workspace/source/test/e2e/tests/test_tenant_namespace_discovery.py:353: Dormant-mode test mutates controller flags; set ENABLE_TENANT_DISCOVERY_DORMANT_E2E=true/workspace/source/test/e2e/tests/test_gateway_scoped_authpolicy.py:170: gateway-default-auth not present (maas-gateway-auth is active); scoping is validated by unit tests/workspace/source/test/e2e/tests/test_tenant_auth_isolation.py:209: Per-tenant OIDC tokens unavailable — set OIDC_TOKEN_URL (tenant-a) and OIDC_TOKEN_URL_TENANT_B (tenant-b), or OIDC_TOKEN_TENANT_A / OIDC_TOKEN_TENANT_B/workspace/source/test/e2e/tests/test_tenant_discovery.py:134: Skipping when Gateway uses ClusterIP + OpenShift Route (unsupported configuration). This mixes incompatible routing paradigms. Gateway has no external hostname in spec.listeners, so /v1/tenants returns an error. Supported configuration: LoadBalancer service with hostname in spec.listeners./workspace/source/test/e2e/tests/test_tenant_discovery.py:210: Skipping when Gateway uses ClusterIP + OpenShift Route (unsupported configuration). This mixes incompatible routing paradigms. Gateway has no external hostname in spec.listeners, so /v1/tenants returns an error. Supported configuration: LoadBalancer service with hostname in spec.listeners./workspace/source/test/e2e/tests/test_tenant_discovery_isolation.py:131: Skipping when Gateway uses ClusterIP + OpenShift Route (unsupported configuration). This mixes incompatible routing paradigms. Gateway has no external hostname in spec.listeners, so /v1/tenants returns an error. Supported configuration: LoadBalancer service with hostname in spec.listeners./workspace/source/test/e2e/tests/test_tenant_discovery_isolation.py:179: Skipping when Gateway uses ClusterIP + OpenShift Route (unsupported configuration). This mixes incompatible routing paradigms. Gateway has no external hostname in spec.listeners, so /v1/tenants returns an error. Supported configuration: LoadBalancer service with hostname in spec.listeners./workspace/source/test/e2e/tests/test_tenant_discovery_isolation.py:266: Skipping when Gateway uses ClusterIP + OpenShift Route (unsupported configuration). This mixes incompatible routing paradigms. Gateway has no external hostname in spec.listeners, so /v1/tenants returns an error. Supported configuration: LoadBalancer service with hostname in spec.listeners.self = <test_per_tenant_ipp_isolation.TestPerTenantIPPRouting object at 0x7f11aa6513d0> routing_case = {'gateway_name': 'e2e-ipp-a-d91e94', 'model_name': 'ipp-route-d91e94', 'model_path': '/ai-tenant-e2e-ipp-a-d91e94/ipp-route-d91e94', 'policy_name': 'e2e-ipp-a-d91e94-policy', ...} ipp_tenant_cases = ({'gateway_name': 'e2e-ipp-a-d91e94', 'model_name': 'ipp-route-d91e94', 'model_path': '/ai-tenant-e2e-ipp-a-d91e94/ipp...1f67', 'policy_name': 'e2e-ipp-b-651f67-policy', 'subscription_name': 'e2e-ipp-b-651f67-sub', 'suffix': '651f67', ...}) def test_tenant_gateway_hits_tenant_ipp_only(self, routing_case, ipp_tenant_cases): _, case_b = ipp_tenant_cases default_names = per_tenant_ipp_names(DEFAULT_AITENANT_NAME) tenant_names = per_tenant_ipp_names(routing_case["tenant_label_name"]) other_names = per_tenant_ipp_names(case_b["tenant_label_name"]) gateway_url = _get_tenant_gateway_url(routing_case["gateway_name"]) api_key = _create_tenant_api_key( gateway_url, routing_case, f"{routing_case['model_name']}-sub", ) time.sleep(2) response = _post_hybrid_chat( gateway_url, routing_case["model_path"], api_key, ) assert response.status_code == 200, ( f"Tenant gateway hybrid BBR failed: {response.status_code} " f"{redact_sensitive(response.text[:500])}" ) time.sleep(2) tenant_logs = deployment_log_snapshot( tenant_names["processing_deployment"], since="1m" ) default_logs = deployment_log_snapshot( default_names["processing_deployment"], since="1m" ) other_logs = deployment_log_snapshot( other_names["processing_deployment"], since="1m" ) > assert ipp_logs_show_recent_activity(tenant_logs), ( f"Expected ext_proc activity in {tenant_names['processing_deployment']} logs" ) E AssertionError: Expected ext_proc activity in payload-processing-e2e-ipp-a-d91e94 logs E assert False E + where False = ipp_logs_show_recent_activity('') test/e2e/tests/test_per_tenant_ipp_isolation.py:376: AssertionError/workspace/source/test/e2e/tests/test_external_oidc.py:249: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:260: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:274: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:287: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:300: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:332: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:376: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:392: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:402: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:425: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:433: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:438: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:451: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:498: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:534: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:599: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:630: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:656: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:687: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:747: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:771: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:824: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:872: EXTERNAL_OIDC is not true/workspace/source/test/e2e/tests/test_external_oidc.py:926: EXTERNAL_OIDC not enabled/workspace/source/test/e2e/tests/test_external_oidc.py:978: EXTERNAL_OIDC not enabled/workspace/source/test/e2e/tests/test_external_oidc.py:1014: EXTERNAL_OIDC not enabled/workspace/source/test/e2e/tests/test_external_oidc.py:1068: EXTERNAL_OIDC not enabled/workspace/source/test/e2e/tests/test_external_oidc.py:1107: EXTERNAL_OIDC not enabled