2026-06-03T23:36:27.155875Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T23:36:27.135Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 20 - "100.64.0.2" "python-requests/2.32.5" "4122707a-dcb5-4fb6-8a6c-888f3b7e017a" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:39556 10.133.0.38:443 100.64.0.2:8540 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T23:36:41.199912Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:36:41.199964Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:36:41.214743Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:36:41.214882Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:36:41.215095Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:36:41.220028Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:36:41.220095Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T23:36:41.220106Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T23:36:41.197Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 45 329 22 - "100.64.0.2" "python-requests/2.32.5" "8534c070-6b14-4514-964b-08ee7063d2ef" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:44000 10.133.0.38:443 100.64.0.2:5658 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T23:36:49.255914Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:36:49.255968Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:36:49.272806Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:36:49.272960Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:36:49.283323Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:36:49.283370Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:36:49.290888Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:36:49.291032Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:36:49.291151Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:36:49.292438Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:36:49.292503Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:36:49.292514Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:36:49.293429Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:36:49.293482Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:36:49.293493Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T23:36:49.281Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 11 - "10.133.0.45" "Go-http-client/1.1" "792c1f64-7039-4d12-9e68-fc676dfe65bb" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.48:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:58934 10.133.0.38:443 10.133.0.45:47402 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-03T23:36:49.253Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 471 39 - "100.64.0.4" "python-requests/2.32.5" "bdd7c49b-bed8-45e2-aeca-fda956c17303" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:39556 10.133.0.38:443 100.64.0.4:54836 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T23:37:05.568322Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:05.568376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:05.583759Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:05.583912Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:05.584162Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:05.588011Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:05.588076Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T23:37:05.588087Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:05.612092Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:05.612143Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:05.622653Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:05.622830Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:05.623067Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:05.626351Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:05.626422Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:37:05.626433Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T23:37:05.566Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 297 21 - "100.64.0.3" "python-requests/2.32.5" "f70c95ef-e42c-4e48-b74c-bb6a664d5fb4" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:44000 10.133.0.38:443 100.64.0.3:3524 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:37:05.609Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 297 16 - "100.64.0.2" "python-requests/2.32.5" "f9b00797-1707-43e7-b01c-0669ec582803" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:39556 10.133.0.38:443 100.64.0.2:29580 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T23:37:13.660571Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:13.660622Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:13.664766Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:13.664894Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:13.667671Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:13.667735Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:13.674307Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:13.674439Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:13.674554Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:13.675818Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:13.675880Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:37:13.675891Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:13.676891Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:13.676952Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T23:37:13.676964Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:13.699002Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:13.699055Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:13.702852Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:13.702987Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:13.705637Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:13.705677Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:13.712349Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:13.712470Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:13.712588Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:13.714051Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:13.714116Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:37:13.714126Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:13.715120Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:13.715179Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T23:37:13.715190Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T23:37:13.666Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 9 - "10.133.0.45" "Go-http-client/1.1" "ad59eca9-94ed-4ad2-ad18-5f03b38732da" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.48:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:58934 10.133.0.38:443 10.133.0.45:47402 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-03T23:37:13.658Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 468 18 - "100.64.0.4" "python-requests/2.32.5" "7d91f1f9-3fbd-4b2a-9fd9-27f703f161c4" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:44000 10.133.0.38:443 100.64.0.4:51220 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:37:13.704Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 10 - "10.133.0.45" "Go-http-client/1.1" "3bf9a238-547a-4d2a-ab3f-d170f323d53a" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.132.0.34:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:40014 10.133.0.38:443 10.133.0.45:47402 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-03T23:37:13.696Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 345 18 - "100.64.0.3" "python-requests/2.32.5" "41674593-48bd-453f-913d-461d5de74a2c" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:44000 10.133.0.38:443 100.64.0.3:3532 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T23:37:32.362460Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:32.362517Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:32.378563Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:32.378782Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:32.381599Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:32.381638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:32.391289Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:32.391424Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:32.392626Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:32.392691Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:37:32.392718Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:32.394689Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:32.394739Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T23:37:32.380Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 12 - "10.133.0.45" "Go-http-client/1.1" "ed5b8d03-df4e-9c2b-b759-8ffc35f474a8" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.48:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:58934 10.133.0.38:443 10.133.0.45:47402 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 2026-06-03T23:37:32.403518Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:32.403674Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:32.405329Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:32.405399Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:37:32.405411Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:32.407366Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:32.407401Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:32.416516Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:32.416675Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:32.418441Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:32.418513Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:37:32.418524Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:32.419618Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:32.419725Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T23:37:32.419739Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T23:37:32.393Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 12 - "10.133.0.45" "Go-http-client/1.1" "e1182629-b9e1-4241-b37a-fa7717abdd68" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.132.0.34:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:40014 10.133.0.38:443 10.133.0.45:47402 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-03T23:37:32.405Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 12 - "10.133.0.45" "Go-http-client/1.1" "e73d9db0-360a-4ad6-a77f-94ccdf4a2911" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.132.0.37:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:47746 10.133.0.38:443 10.133.0.45:47402 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T23:37:32.360Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 1361 59 - "100.64.0.3" "python-requests/2.32.5" "1a54814b-2ffc-4656-aee0-242fa29ae523" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:44000 10.133.0.38:443 100.64.0.3:5308 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T23:37:53.133159Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:53.133208Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:53.147747Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:53.147930Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:53.151270Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:53.151310Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:53.161228Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:53.161391Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:53.162835Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:53.162904Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:37:53.162916Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:53.163969Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:53.164026Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T23:37:53.164037Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:37:53.189468Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:53.189520Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:53.200731Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:53.200900Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:53.203663Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:53.203717Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:53.213755Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:53.213911Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:53.215360Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:53.215428Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:37:53.215440Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:53.216455Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:37:53.216516Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:37:53.216528Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T23:37:53.149Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 13 - "10.133.0.45" "Go-http-client/1.1" "a481cf21-421b-4be3-a2bc-bd988d4373dd" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.48:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:58934 10.133.0.38:443 10.133.0.45:47402 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-03T23:37:53.131Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 470 32 - "100.64.0.2" "python-requests/2.32.5" "ac1466f4-efa7-40af-970e-85f18081c904" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:44000 10.133.0.38:443 100.64.0.2:10220 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:37:53.202Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 13 - "10.133.0.45" "Go-http-client/1.1" "2c5657d7-f5cb-4b98-967e-6465ad2d751f" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.132.0.34:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:40014 10.133.0.38:443 10.133.0.45:47402 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-03T23:37:53.187Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 347 29 - "100.64.0.4" "python-requests/2.32.5" "8cd65bed-2d96-4a1f-a0b0-559c5634f36b" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:39556 10.133.0.38:443 100.64.0.4:60286 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T23:38:02.266600Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:02.266650Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:02.268370Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:02.268404Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T23:38:02.264Z] "GET /maas-api/v1/models HTTP/1.1" 401 - - "-" 0 0 3 - "100.64.0.3" "python-requests/2.32.5" "9e3c0be2-eec3-4653-bf82-22e21c064cea" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.38:443 100.64.0.3:54810 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T23:38:18.885224Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:18.885275Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:18.902135Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:18.902294Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:18.902565Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:18.907309Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:18.907382Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:38:18.907394Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:18.926877Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:18.926923Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:18.934901Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:18.935022Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:18.935322Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:18.935357Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:18.936913Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:18.936942Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:18.937170Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:18.938962Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:18.939098Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:18.939270Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:18.939940Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:18.960547Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:18.960593Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:18.967726Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:18.967851Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:18.968150Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:18.968184Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:18.968849Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:18.968876Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:18.991754Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:18.991807Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:18.999111Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:18.999248Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:18.999602Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:18.999643Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:19.000258Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:19.000296Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:19.021150Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:19.021202Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:19.024889Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:19.025036Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:19.028297Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:19.028336Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:19.034907Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:19.035029Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:19.035144Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:19.038546Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:19.038616Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:38:19.038627Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:19.039934Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:19.039996Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:38:19.040007Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T23:38:18.882Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 331 24 - "100.64.0.2" "python-requests/2.32.5" "e21fb504-3700-49b3-be41-4601b7e1cc01" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:39556 10.133.0.38:443 100.64.0.2:23568 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:38:18.925Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 14 - "100.64.0.4" "python-requests/2.32.5" "b8e425bb-2459-451e-a57d-f98cd8fa45ef" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.132.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:55478 10.133.0.38:443 100.64.0.4:15972 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-03T23:38:18.959Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 9 - "100.64.0.3" "python-requests/2.32.5" "3be6f036-6f84-4dc6-a44a-ec13e1590d95" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.132.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:33678 10.133.0.38:443 100.64.0.3:2512 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-03T23:38:18.990Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 10 - "100.64.0.4" "python-requests/2.32.5" "f5d2ffd0-9623-45c3-9c21-1d747ca98d38" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.132.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:55478 10.133.0.38:443 100.64.0.4:21628 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-03T23:38:19.027Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 11 - "10.133.0.45" "Go-http-client/1.1" "689e7f23-8a65-4a63-b1ed-19fa3525ceb4" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.132.0.36:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:33806 10.133.0.38:443 10.133.0.45:47402 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T23:38:19.019Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 547 20 - "100.64.0.2" "python-requests/2.32.5" "7cdd46a8-f4c7-4395-b4ef-634a798d2d6f" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:39556 10.133.0.38:443 100.64.0.2:23580 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T23:38:27.907834Z info ads ADS: new connection for node:6 2026-06-03T23:38:27.910985Z info cache read certificate from file resource=file-root:system 2026-06-03T23:38:36.793624Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:36.793675Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:36.808689Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:36.808861Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:36.809111Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:36.812399Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:36.812467Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T23:38:36.812478Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:37.163681Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:37.163744Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:37.165070Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:37.165105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:37.189483Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:37.189527Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:37.190729Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:37.190764Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:38:37.215378Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:37.215421Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:37.222624Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:37.222762Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:37.223029Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:37.223066Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:37.224657Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:37.224683Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:37.224927Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:37.225850Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:38:37.225980Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:38:37.225993Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T23:38:36.791Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 322 20 - "100.64.0.2" "python-requests/2.32.5" "c0e7db2e-dbd2-4e30-bbd4-2be418407be2" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:44000 10.133.0.38:443 100.64.0.2:12500 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:38:37.161Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 3 - "100.64.0.3" "python-requests/2.32.5" "0ea8a48d-b9f0-4321-95f7-a661b16b6410" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.133.0.38:443 100.64.0.3:33768 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-06-03T23:38:37.187Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 3 - "100.64.0.4" "python-requests/2.32.5" "2f05cf59-38cc-424d-aa5b-5eb01a7993f3" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.133.0.38:443 100.64.0.4:28944 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-06-03T23:38:37.213Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 404 - - "-" 83 98 12 - "100.64.0.4" "python-requests/2.32.5" "470157e7-c47c-4585-9286-1a8d95fbb559" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.133.0.38:443 100.64.0.4:21524 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com llm.e2e-external-model.0 2026-06-03T23:39:14.862818Z info ads ADS: "" 6 terminated with stream closed 2026-06-03T23:39:16.696980Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:16.697030Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:16.700454Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:16.700572Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:16.700782Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:16.714437Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:16.714508Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:39:16.714518Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:16.816139Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:16.816188Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:16.847466Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:16.847510Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:16.878191Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:16.878240Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:16.879826Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:16.879858Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:16.901859Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:16.901902Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:16.903378Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:16.903410Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:17.312044Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.312092Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.315214Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.315323Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.315497Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.319071Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.319132Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T23:39:17.319144Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T23:39:16.694Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 19 - "100.64.0.2" "python-requests/2.32.5" "512e9480-ea07-451f-9e5e-a604f63e3ac4" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:39556 10.133.0.38:443 100.64.0.2:26130 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:39:16.814Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 37 0 33 - "100.64.0.3" "python-requests/2.32.5" "deb637eb-0c1f-4750-8752-f03ab81e635d" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.38:443 100.64.0.3:62922 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:39:16.876Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 35 0 3 - "100.64.0.4" "python-requests/2.32.5" "3c53e61b-51b5-42bd-ab54-7e980c529a8e" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.38:443 100.64.0.4:9082 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:39:16.899Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 36 0 3 - "100.64.0.2" "python-requests/2.32.5" "95497883-7935-4ce4-9f6a-93c6edc3436a" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.38:443 100.64.0.2:26144 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:39:17.309Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 28 312 9 - "100.64.0.3" "python-requests/2.32.5" "18d1120e-bd9b-4bb4-972d-07088c9be5b1" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:44000 10.133.0.38:443 100.64.0.3:62930 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T23:39:17.557969Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.558024Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.561036Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.561173Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.561383Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.564674Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.564768Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T23:39:17.564782Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.585589Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:17.585639Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:17.590203Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:17.590339Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:17.593738Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:17.593782Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:17.600247Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:17.600402Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:17.600517Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:17.602943Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:17.603011Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:39:17.603026Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:17.604308Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:17.604370Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:39:17.604386Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:17.623139Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.623184Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.630568Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.630738Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.631011Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.631045Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.632532Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.632558Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.632817Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.634789Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.634923Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.635096Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.635662Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.728442Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:17.728490Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:17.731296Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:17.731404Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:17.731574Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:17.734824Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:17.734882Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:39:17.734893Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:17.755606Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.755657Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.758418Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.758539Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.762977Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:17.763053Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T23:39:17.763066Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T23:39:17.555Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 9 - "100.64.0.4" "python-requests/2.32.5" "73f5a3b3-3ad6-4c33-a86b-61cb40c7525f" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:44000 10.133.0.38:443 100.64.0.4:9086 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:39:17.592Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 10 - "10.133.0.45" "Go-http-client/1.1" "0ee229cf-906e-4364-89a9-847f76ed4bbc" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.132.0.37:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:47746 10.133.0.38:443 10.133.0.45:47402 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T23:39:17.583Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 20 - "100.64.0.2" "python-requests/2.32.5" "d1dc87d7-13a5-432f-9ebe-5c8f6e93f57c" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:39556 10.133.0.38:443 100.64.0.2:26160 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:39:17.621Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 123 375 14 - "100.64.0.3" "python-requests/2.32.5" "cc8a1b3a-8b8d-47a4-a0eb-bd00fa5813c2" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.132.0.37:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:42224 10.133.0.38:443 100.64.0.3:62936 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-06-03T23:39:17.726Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 8 - "100.64.0.3" "python-requests/2.32.5" "bf262af9-3229-42e7-a4cd-f3bf47c32d1a" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:39556 10.133.0.38:443 100.64.0.3:13674 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:39:17.753Z] "DELETE /v1/api-keys/62a4ecdc-07d4-499e-97fd-7ce2828df27d HTTP/1.1" 200 - via_upstream - "-" 0 251 9 - "100.64.0.4" "python-requests/2.32.5" "0f3c30ae-1c7f-4db4-b3e5-5f6e3641e923" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:44000 10.133.0.38:443 100.64.0.4:9102 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T23:39:20.792478Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:20.792529Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:20.796513Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:20.796599Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:20.798048Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:20.798110Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T23:39:20.798121Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:20.947435Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:20.947499Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:20.962091Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:20.962151Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.137343Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.137394Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.140348Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.140485Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.140692Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.144241Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.144330Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T23:39:21.144347Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.168195Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.168249Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.171048Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.171189Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.171389Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.174492Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.174563Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T23:39:21.174579Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.272685Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.272761Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.275547Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.275678Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.275924Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.279061Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.279129Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:39:21.279141Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.302770Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.302817Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.305590Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.305745Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.309524Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.309589Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:39:21.309600Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.335814Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.335868Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.338659Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.338827Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.341253Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.341326Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T23:39:21.341342Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.439329Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.439382Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.442117Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.442244Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.442450Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.445659Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.445752Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T23:39:21.445770Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.464948Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.465000Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.468766Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.468889Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.471284Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.471319Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.478007Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.478142Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.478246Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.479745Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.479809Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:39:21.479820Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.480793Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.480844Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:39:21.480855Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T23:39:20.790Z] "GET /v1/models HTTP/1.1" 500 - via_upstream - "-" 0 96 7 - "100.64.0.2" "python-requests/2.32.5" "e301a899-be3b-4248-9f58-5bad7c393438" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:44000 10.133.0.38:443 100.64.0.2:26162 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:39:20.944Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 38 0 17 - "100.64.0.3" "python-requests/2.32.5" "ae17b9ff-a066-449a-99c3-ab9ba8c59a69" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.38:443 100.64.0.3:62940 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:39:21.135Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 34 319 9 - "100.64.0.4" "python-requests/2.32.5" "bb53e20d-7755-4614-92b0-515bd09d804e" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:44000 10.133.0.38:443 100.64.0.4:9114 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:39:21.166Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 32 316 8 - "100.64.0.2" "python-requests/2.32.5" "d564c86c-ff27-45da-baaa-2edec2ae1e7b" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:44000 10.133.0.38:443 100.64.0.2:26166 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:39:21.270Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 31 315 8 - "100.64.0.3" "python-requests/2.32.5" "48e1c67c-0dc1-42c8-9c5e-14670cfd3f68" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:39556 10.133.0.38:443 100.64.0.3:53340 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:39:21.300Z] "DELETE /v1/api-keys/b6b7a302-40da-44cb-97f9-8115921d07d6 HTTP/1.1" 200 - via_upstream - "-" 0 253 9 - "100.64.0.4" "python-requests/2.32.5" "2941062e-382b-4a48-8634-4b2327e29286" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:39556 10.133.0.38:443 100.64.0.4:60350 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:39:21.333Z] "DELETE /v1/api-keys/b6b7a302-40da-44cb-97f9-8115921d07d6 HTTP/1.1" 404 - via_upstream - "-" 0 29 7 - "100.64.0.2" "python-requests/2.32.5" "07784f2f-7f4d-4093-987a-8ad00a7a5b67" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:44000 10.133.0.38:443 100.64.0.2:14552 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:39:21.437Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 8 - "100.64.0.3" "python-requests/2.32.5" "d2789cf4-3ea6-47db-ab79-c1cd7f792d13" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:44000 10.133.0.38:443 100.64.0.3:53350 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:39:21.469Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.133.0.45" "Go-http-client/1.1" "2b330fdf-559f-4b1b-bc89-d9f73dc6e4da" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.132.0.37:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:47746 10.133.0.38:443 10.133.0.45:47402 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T23:39:21.463Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 17 - "100.64.0.4" "python-requests/2.32.5" "39ab6db7-2c0a-4b2d-b590-acdce0964f4f" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:39556 10.133.0.38:443 100.64.0.4:60352 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T23:39:21.573293Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.573357Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.576473Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.576656Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.576978Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.580550Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.580639Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:39:21.580658Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.603743Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.603803Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.608303Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.608489Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.610561Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.610637Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:39:21.610649Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.630575Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.630632Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.634608Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.634777Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.637920Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.637968Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.644491Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.644696Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.644888Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.646620Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.646738Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:39:21.646756Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.647937Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.648004Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T23:39:21.648020Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.760953Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.761010Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.763883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.764040Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.764278Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.767650Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.767755Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:39:21.767773Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.787136Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.787205Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.791073Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.791255Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.794440Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.794489Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.801235Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.801445Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.801615Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.803365Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.803448Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:39:21.803467Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.804578Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.804639Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:39:21.804652Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.826820Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.826870Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.830571Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.830721Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.833418Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.833455Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.840252Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.840409Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.840522Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.841999Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.842062Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:39:21.842074Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.843050Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.843109Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T23:39:21.843121Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T23:39:21.936683Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.936743Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.939526Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.939635Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.939840Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.942989Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T23:39:21.943049Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T23:39:21.943060Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T23:39:21.570Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 10 - "100.64.0.2" "python-requests/2.32.5" "1fd7133f-4567-417d-990d-494f7c3f9481" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:39556 10.133.0.38:443 100.64.0.2:14566 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:39:21.601Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 9 - "100.64.0.3" "python-requests/2.32.5" "7ede3a1c-023a-43e2-b31a-a0d0656ffddf" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:39556 10.133.0.38:443 100.64.0.3:53364 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:39:21.635Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 10 - "10.133.0.45" "Go-http-client/1.1" "7c0158da-9f71-433f-9948-a5d1d09d033b" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.132.0.37:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:47746 10.133.0.38:443 10.133.0.45:47402 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T23:39:21.628Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 19 - "100.64.0.2" "python-requests/2.32.5" "e0576272-a917-4cba-a29a-f218f64dab16" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:44000 10.133.0.38:443 100.64.0.2:9378 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:39:21.758Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 9 - "100.64.0.4" "python-requests/2.32.5" "a896b262-4937-4f0a-9751-793a9222acef" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:39556 10.133.0.38:443 100.64.0.4:60360 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:39:21.792Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 11 - "10.133.0.45" "Go-http-client/1.1" "9e994d81-4ad3-4ae0-9e8b-7bad31b88b85" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.132.0.37:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:47746 10.133.0.38:443 10.133.0.45:47402 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T23:39:21.784Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 20 - "100.64.0.2" "python-requests/2.32.5" "fb7b4852-6bfb-417c-82ae-2e6c0fa215bc" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:39556 10.133.0.38:443 100.64.0.2:14576 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:39:21.831Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 10 - "10.133.0.45" "Go-http-client/1.1" "3884c015-52a6-4897-9784-4f1ec9af03c7" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.132.0.37:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:47746 10.133.0.38:443 10.133.0.45:47402 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T23:39:21.824Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 18 - "100.64.0.4" "python-requests/2.32.5" "0ee3d420-76f5-4c2a-898c-64326e2cb306" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:39556 10.133.0.38:443 100.64.0.4:6908 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T23:39:21.934Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 36 321 8 - "100.64.0.3" "python-requests/2.32.5" "e5332438-11b2-451b-b63a-c3c9ef163d7a" "maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:44000 10.133.0.38:443 100.64.0.3:53372 maas.apps.f717e01e-8081-4124-81bf-46fd99b2f8c0.prod.konfluxeaas.com opendatahub.maas-api-route.1