[2026-06-03T21:20:03.580Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 20 - "100.64.0.4" "python-requests/2.32.5" "53d0fecd-44d1-4182-bfd8-7009cf814a5f" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:42096 10.134.0.24:443 100.64.0.4:2094 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T21:20:19.707517Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:19.707565Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:19.721517Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:19.721635Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:19.721815Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:19.725839Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:19.725900Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:20:19.725924Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T21:20:19.705Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 45 330 20 - "100.64.0.3" "python-requests/2.32.5" "ccf8f0f1-23de-4963-9e8d-46f627fd6b40" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:42096 10.134.0.24:443 100.64.0.3:35462 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T21:20:27.755095Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:27.755149Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:27.770628Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:27.770750Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:27.783070Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:27.783112Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:27.790239Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:27.790374Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:27.790482Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:27.792052Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:27.792117Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:20:27.792132Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:27.793239Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:27.793299Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:20:27.793315Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T21:20:27.781Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 10 - "10.133.0.51" "Go-http-client/1.1" "deab8566-a472-4f4a-bee6-b8e704d41199" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.55:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.24:36690 10.134.0.24:443 10.133.0.51:58806 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-03T21:20:27.753Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 471 40 - "100.64.0.2" "python-requests/2.32.5" "66cecb7d-c6a7-4293-be0e-8ec5a0e19b23" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:42096 10.134.0.24:443 100.64.0.2:46530 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T21:20:41.871335Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:41.871381Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:41.885641Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:41.885747Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:41.885947Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:41.889761Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:41.889821Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:20:41.889832Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:41.910452Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:20:41.910488Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:20:41.921015Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:20:41.921115Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:20:41.921267Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:20:41.924691Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:20:41.924744Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T21:20:41.924754Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T21:20:41.869Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 298 20 - "100.64.0.3" "python-requests/2.32.5" "85c2d949-8264-4274-a480-696c99f00631" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:42096 10.134.0.24:443 100.64.0.3:3652 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:20:41.908Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 297 16 - "100.64.0.4" "python-requests/2.32.5" "e2bb1c26-1414-4845-ac38-78f244d88671" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:42690 10.134.0.24:443 100.64.0.4:9460 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T21:20:49.949999Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:20:49.950045Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:20:49.954309Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:20:49.954414Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:20:49.957776Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:49.957816Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:49.964352Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:49.964467Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:49.964571Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:49.966157Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:49.966215Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:20:49.966226Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:49.967549Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:20:49.967612Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T21:20:49.967624Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:20:49.985149Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:49.985181Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:49.988775Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:49.988865Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:49.991460Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:49.991495Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:49.997742Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:49.997849Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:49.997961Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:50.000688Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:50.000743Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:20:50.000753Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:50.002625Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:20:50.002674Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:20:50.002684Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T21:20:49.955Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 10 - "10.133.0.51" "Go-http-client/1.1" "b0c00a23-b117-48e5-811d-93a8d39c1626" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.55:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.24:36690 10.134.0.24:443 10.133.0.51:58806 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-03T21:20:49.948Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 468 19 - "100.64.0.2" "python-requests/2.32.5" "4b4c6e9d-8f5d-4fc6-8571-388c366be950" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:42690 10.134.0.24:443 100.64.0.2:23566 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:20:49.989Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 10 - "10.133.0.51" "Go-http-client/1.1" "2281e2ed-e64b-47ee-9697-1e8ba55a9ac7" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.134.0.32:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.24:58382 10.134.0.24:443 10.133.0.51:58806 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-03T21:20:49.983Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 345 18 - "100.64.0.3" "python-requests/2.32.5" "f08fea65-18d4-455d-ae0b-d94ec9c186e8" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:42096 10.134.0.24:443 100.64.0.3:61292 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T21:21:08.533909Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.533976Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.549371Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.549505Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.552730Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.552764Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.562496Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.562616Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.564313Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.564375Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:21:08.564385Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.566384Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.566416Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.577842Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.577970Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.579496Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.579547Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:21:08.579557Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.581436Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.581467Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.591594Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.591691Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.593021Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.593067Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:21:08.593077Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.594204Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:08.594258Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:21:08.594270Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T21:21:08.551Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 13 - "10.133.0.51" "Go-http-client/1.1" "db147819-29c5-4bf1-9959-24c1f870a193" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.55:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.24:36690 10.134.0.24:443 10.133.0.51:58806 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-03T21:21:08.565Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 14 - "10.133.0.51" "Go-http-client/1.1" "a10a37b7-f87d-4f5c-bc24-9ffbb4881643" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.134.0.32:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.24:58382 10.134.0.24:443 10.133.0.51:58806 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-03T21:21:08.580Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 12 - "10.133.0.51" "Go-http-client/1.1" "b417553c-749e-42a1-944c-f3ff5faef66d" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.134.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.24:36742 10.134.0.24:443 10.133.0.51:58806 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T21:21:08.532Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 1361 62 - "100.64.0.4" "python-requests/2.32.5" "a6070c69-eab2-48a5-a04f-8e3623125c3e" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:42096 10.134.0.24:443 100.64.0.4:4388 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T21:21:27.081596Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.081644Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.097091Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.097208Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.100042Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.100076Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.111041Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.111148Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.112664Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.112722Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:21:27.112732Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.113862Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.113930Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:21:27.113942Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.130862Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.130895Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.140881Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.140995Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.143466Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.143497Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.153046Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.153150Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.154419Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.154473Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:21:27.154485Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.155526Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:27.155573Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:21:27.155583Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T21:21:27.098Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 14 - "10.133.0.51" "Go-http-client/1.1" "ac43134f-be8a-4a7f-966c-383b0240aba2" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.55:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.24:36690 10.134.0.24:443 10.133.0.51:58806 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-03T21:21:27.079Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 470 34 - "100.64.0.4" "python-requests/2.32.5" "f87811f2-293a-4b32-92b4-56bbe79baf60" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:42096 10.134.0.24:443 100.64.0.4:14156 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:21:27.142Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 12 - "10.133.0.51" "Go-http-client/1.1" "53c698c1-e918-4df7-8d33-1f830c7cb941" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.134.0.32:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.24:58382 10.134.0.24:443 10.133.0.51:58806 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-03T21:21:27.129Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 347 26 - "100.64.0.2" "python-requests/2.32.5" "50804cc3-8ea7-4863-bee0-5e9fec47288f" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:42096 10.134.0.24:443 100.64.0.2:64650 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T21:21:36.167505Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:36.167551Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:36.169108Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:36.169140Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T21:21:36.165Z] "GET /maas-api/v1/models HTTP/1.1" 401 - - "-" 0 0 3 - "100.64.0.3" "python-requests/2.32.5" "c0e73787-acca-489d-8fdb-96d6950b12f0" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.24:443 100.64.0.3:29694 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T21:21:52.778696Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.778745Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.794285Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.794389Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.794569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.798183Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.798241Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:21:52.798251Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.816956Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.816988Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.825125Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.825230Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.825456Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.825485Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.826732Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.826757Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.826982Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.828565Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.828694Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.828863Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.829455Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.845198Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.845231Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.852424Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.852528Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.852737Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.852765Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.853255Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.853279Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.873497Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:21:52.873532Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:21:52.880614Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:21:52.880721Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:21:52.880964Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:21:52.880996Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:21:52.881505Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:21:52.881527Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:21:52.905677Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:21:52.905710Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:21:52.909491Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:21:52.909576Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:21:52.914153Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.914186Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.920984Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.921093Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.921185Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.923819Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.923873Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:21:52.923883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:21:52.925428Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:21:52.925481Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T21:21:52.925493Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T21:21:52.776Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 331 21 - "100.64.0.4" "python-requests/2.32.5" "dcb05bd1-d484-43da-ae00-cadcf0e1552b" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:42096 10.134.0.24:443 100.64.0.4:1928 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:21:52.815Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 292 13 - "100.64.0.2" "python-requests/2.32.5" "bfd91595-a11e-4019-9d25-7091b41b2b42" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.24:48230 10.134.0.24:443 100.64.0.2:32098 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-03T21:21:52.844Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 9 - "100.64.0.3" "python-requests/2.32.5" "d1fe2ab1-ff32-4260-b198-5fe42b368921" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.24:48230 10.134.0.24:443 100.64.0.3:55216 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-03T21:21:52.872Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 9 - "100.64.0.2" "python-requests/2.32.5" "41e275bb-cba6-40c0-9235-37c92317880b" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.24:51310 10.134.0.24:443 100.64.0.2:34298 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-03T21:21:52.913Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 10 - "10.133.0.51" "Go-http-client/1.1" "794c03b2-590d-4fb5-80e3-465423f1dc8f" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.24:36282 10.134.0.24:443 10.133.0.51:58806 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T21:21:52.903Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 547 21 - "100.64.0.3" "python-requests/2.32.5" "f5def76c-abf2-47e3-9f26-531977b49a33" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:48940 10.134.0.24:443 100.64.0.3:47604 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T21:22:01.754120Z info ads ADS: new connection for node:6 2026-06-03T21:22:01.756586Z info cache read certificate from file resource=file-root:system 2026-06-03T21:22:10.608216Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:10.608272Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:10.623446Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:10.623572Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:10.623769Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:10.627771Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:10.627832Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T21:22:10.627840Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:10.970098Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:10.970154Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:10.971303Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:10.971334Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:10.991774Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:10.991813Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:10.992799Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:10.992831Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:11.010932Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:11.010971Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:11.017577Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:11.017692Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:11.017942Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:11.017973Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:11.019575Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:11.019599Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:11.019799Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:11.020708Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:11.020812Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T21:22:11.020823Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T21:22:10.606Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 321 21 - "100.64.0.4" "python-requests/2.32.5" "20510d9e-84d4-497e-8ca5-bfd7d58fb199" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:48940 10.134.0.24:443 100.64.0.4:4142 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:10.968Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 3 - "100.64.0.2" "python-requests/2.32.5" "993e54ba-a657-4e7e-9c18-3f6670b4398f" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.134.0.24:443 100.64.0.2:25290 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-06-03T21:22:10.990Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 2 - "100.64.0.4" "python-requests/2.32.5" "b6d3ba2a-bc0d-4632-ab06-77e743e56677" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.134.0.24:443 100.64.0.4:11442 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-06-03T21:22:11.009Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 404 - - "-" 83 98 11 - "100.64.0.2" "python-requests/2.32.5" "624fe263-9998-4a01-bfa5-cc11a0c41f22" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.134.0.24:443 100.64.0.2:44590 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com llm.e2e-external-model.0 2026-06-03T21:22:48.549152Z info ads ADS: "" 6 terminated with stream closed 2026-06-03T21:22:50.449213Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:50.449260Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:50.451994Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:50.452095Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:50.452256Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:50.467313Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:50.467373Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T21:22:50.467384Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T21:22:50.446Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 20 - "100.64.0.3" "python-requests/2.32.5" "56023892-e566-4aa0-b17f-ee08497d8834" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:48940 10.134.0.24:443 100.64.0.3:35754 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T21:22:50.549283Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:50.549321Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:50.576041Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:50.576074Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:50.597824Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:50.597864Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:50.599432Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:50.599464Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:50.616645Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:50.616684Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:50.618180Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:50.618220Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:50.977593Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:50.977640Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:50.980598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:50.980702Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:50.980869Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:50.984754Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:50.984813Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T21:22:50.984824Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.183537Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.183595Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.186348Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.186471Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.186655Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.190335Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.190406Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T21:22:51.190422Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.206688Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:51.206724Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:51.211256Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:51.211349Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:51.214505Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:51.214535Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:51.221192Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:51.221374Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:51.221494Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:51.223488Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:51.223546Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:22:51.223556Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:51.225099Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:51.225149Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:22:51.225160Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:51.242164Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.242203Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.249161Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.249301Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.249534Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.249576Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.251076Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.251107Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.251303Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.253028Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.253151Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.253314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.253825Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.345047Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:51.345084Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:51.347933Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:51.348027Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:51.348183Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:51.351619Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:51.351669Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:22:51.351679Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:51.371144Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.371180Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.373870Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.373980Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.378630Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:51.378677Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T21:22:51.378687Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T21:22:50.547Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 37 0 28 - "100.64.0.4" "python-requests/2.32.5" "a6fc4dc4-289d-43e3-856c-ea60245e0814" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.24:443 100.64.0.4:39356 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:50.595Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 35 0 3 - "100.64.0.3" "python-requests/2.32.5" "5a21c9c4-0633-49e8-b658-604a7e9cb821" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.24:443 100.64.0.3:52392 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:50.614Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 36 0 3 - "100.64.0.4" "python-requests/2.32.5" "683066d1-2156-4ef6-b074-97ab12990c97" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.24:443 100.64.0.4:20132 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:50.975Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 28 313 9 - "100.64.0.2" "python-requests/2.32.5" "6d9d2466-ecc4-4d92-a836-9ffa90dbb601" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:48940 10.134.0.24:443 100.64.0.2:29582 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:51.181Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 8 - "100.64.0.3" "python-requests/2.32.5" "3376b0b2-3b4e-4027-a84b-2bb36cdef862" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:48940 10.134.0.24:443 100.64.0.3:35770 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:51.213Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 10 - "10.133.0.51" "Go-http-client/1.1" "cb508e7e-04a3-4d43-93cd-51bf20e60c2a" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.134.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.24:36742 10.134.0.24:443 10.133.0.51:58806 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T21:22:51.205Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 20 - "100.64.0.4" "python-requests/2.32.5" "82d54c9b-e68d-4541-8955-283cdb8e2a5d" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:42096 10.134.0.24:443 100.64.0.4:39366 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:51.240Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 123 385 13 - "100.64.0.2" "python-requests/2.32.5" "3a121adb-2375-4daa-91d6-224ce0f50bb1" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.134.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.24:33786 10.134.0.24:443 100.64.0.2:29598 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-06-03T21:22:51.343Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 8 - "100.64.0.2" "python-requests/2.32.5" "a9876ebf-676d-4f3d-bc9e-c60d2d686cd9" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:42096 10.134.0.24:443 100.64.0.2:27274 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:51.369Z] "DELETE /v1/api-keys/abad9ac0-84c8-4240-9e70-c9cdc613074c HTTP/1.1" 200 - via_upstream - "-" 0 251 9 - "100.64.0.3" "python-requests/2.32.5" "38828c1b-1450-4f87-97d0-a336557ee920" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:48940 10.134.0.24:443 100.64.0.3:35782 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T21:22:54.398708Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.398775Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.402776Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.402861Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.404624Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.404689Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T21:22:54.404704Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T21:22:54.396Z] "GET /v1/models HTTP/1.1" 500 - via_upstream - "-" 0 96 8 - "100.64.0.4" "python-requests/2.32.5" "b126a1f4-9709-400b-83b4-682823eed73a" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:48940 10.134.0.24:443 100.64.0.4:39382 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T21:22:54.523794Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.523838Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.539751Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.539789Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.677441Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.677493Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.680158Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.680278Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.680465Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.684146Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.684205Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T21:22:54.684216Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.704634Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.704672Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.707537Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.707635Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.707788Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.711150Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.711205Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:22:54.711215Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.786402Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.786440Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.789032Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.789126Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.789275Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.792460Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.792511Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T21:22:54.792521Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.811342Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.811375Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.814152Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.814240Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.817998Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.818055Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T21:22:54.818068Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.837937Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.837974Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.840628Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.840724Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.842893Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.842956Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:22:54.842966Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.920638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.920678Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.923525Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.923618Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.923771Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.926958Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.927008Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:22:54.927018Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.950809Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.950845Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.954322Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.954423Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.956795Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.956828Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.963240Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.963360Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.963446Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.964712Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.964767Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:22:54.964777Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:54.965903Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:54.965967Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T21:22:54.965979Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.047540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.047585Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.050386Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.050482Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.050643Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.053880Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.053956Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:22:55.053966Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.074237Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.074281Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.077664Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.077777Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.079458Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.079518Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T21:22:55.079533Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.095811Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.095851Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.098984Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.099080Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.101556Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.101588Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.107455Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.107570Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.107654Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.108976Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.109023Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:22:55.109033Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.110230Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.110292Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T21:22:55.110307Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.189281Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.189319Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.192088Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.192191Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.192358Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.195971Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.196037Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T21:22:55.196051Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.213129Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.213162Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.216600Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.216687Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.219148Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.219181Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.225867Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.226005Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.226107Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.227458Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.227510Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:22:55.227520Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.228644Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.228696Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T21:22:55.228707Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.249067Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.249102Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.252703Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.252800Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.255216Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.255250Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.261782Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.261902Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.262015Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.263314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.263362Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:22:55.263371Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.264406Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.264453Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T21:22:55.264463Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T21:22:55.342578Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.342628Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.345510Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.345626Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.345807Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.349144Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T21:22:55.349208Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T21:22:55.349223Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T21:22:54.521Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 38 0 18 - "100.64.0.2" "python-requests/2.32.5" "413eef33-0fdd-4099-8f33-0e272e33e728" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.24:443 100.64.0.2:29610 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:54.675Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 34 319 8 - "100.64.0.3" "python-requests/2.32.5" "5df425f6-2dc8-42ea-a9a2-a2ed5a565949" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:48940 10.134.0.24:443 100.64.0.3:52402 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:54.702Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 32 317 8 - "100.64.0.4" "python-requests/2.32.5" "61d8f148-c21b-45d1-898a-62438333d2ba" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:42096 10.134.0.24:443 100.64.0.4:20134 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:54.784Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 31 315 7 - "100.64.0.3" "python-requests/2.32.5" "19a10f92-8c6f-4361-847c-20b4199cf706" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:48940 10.134.0.24:443 100.64.0.3:35794 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:54.809Z] "DELETE /v1/api-keys/e3805d0f-bcef-45c5-8e9a-2bb57a6f5047 HTTP/1.1" 200 - via_upstream - "-" 0 253 8 - "100.64.0.4" "python-requests/2.32.5" "69ba25a2-c489-4d14-a48d-1ab920524b0a" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:48940 10.134.0.24:443 100.64.0.4:39386 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:54.836Z] "DELETE /v1/api-keys/e3805d0f-bcef-45c5-8e9a-2bb57a6f5047 HTTP/1.1" 404 - via_upstream - "-" 0 29 6 - "100.64.0.2" "python-requests/2.32.5" "ad29bd7a-8352-4d55-ba7a-eb6df59e9845" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:42096 10.134.0.24:443 100.64.0.2:27284 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:54.918Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 8 - "100.64.0.3" "python-requests/2.32.5" "90a396e0-eba7-93b5-afca-28e0dd9cc9e3" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:42096 10.134.0.24:443 100.64.0.3:52414 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:54.955Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.133.0.51" "Go-http-client/1.1" "ac49f680-ae30-4337-a68e-35b32506b151" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.134.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.24:36742 10.134.0.24:443 10.133.0.51:58806 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T21:22:54.949Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 16 - "100.64.0.4" "python-requests/2.32.5" "84a08be5-9b99-4880-8e57-c14540a55dca" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:48940 10.134.0.24:443 100.64.0.4:20150 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:55.045Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 8 - "100.64.0.2" "python-requests/2.32.5" "b3addfe8-ff1b-46ff-94d8-c09fcd51ceae" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:42096 10.134.0.24:443 100.64.0.2:29616 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:55.072Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 6 - "100.64.0.3" "python-requests/2.32.5" "ab49cf2d-4500-422e-81b5-955426a92bd0" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:48940 10.134.0.24:443 100.64.0.3:35802 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:55.100Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.133.0.51" "Go-http-client/1.1" "ae7a7a61-f7cd-49d4-9c33-e90749a333b4" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.134.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.24:36742 10.134.0.24:443 10.133.0.51:58806 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T21:22:55.094Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 16 - "100.64.0.4" "python-requests/2.32.5" "dc913248-354b-4d37-9f29-7820e43068fc" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:48940 10.134.0.24:443 100.64.0.4:39392 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:55.187Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 8 - "100.64.0.2" "python-requests/2.32.5" "75ee430c-c4de-424f-b3d0-a21be52a8a13" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:48940 10.134.0.24:443 100.64.0.2:29620 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:55.217Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.133.0.51" "Go-http-client/1.1" "34c36a99-0767-4bf3-b75b-3b2bdc902a4a" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.134.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.24:36742 10.134.0.24:443 10.133.0.51:58806 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T21:22:55.211Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 17 - "100.64.0.2" "python-requests/2.32.5" "b807ab6c-ceaa-4e7b-b018-42aa602f0094" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:48940 10.134.0.24:443 100.64.0.2:27288 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:55.253Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.133.0.51" "Go-http-client/1.1" "18715381-1700-430b-ae76-f11f4c47d1bc" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.134.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.24:36742 10.134.0.24:443 10.133.0.51:58806 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T21:22:55.247Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 16 - "100.64.0.3" "python-requests/2.32.5" "26d6bb09-3a08-4c40-bb95-0178753447cd" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:42096 10.134.0.24:443 100.64.0.3:52420 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T21:22:55.340Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 36 320 8 - "100.64.0.4" "python-requests/2.32.5" "fe5d26a1-0130-4b8b-b71f-7e40350fda4d" "maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.24:48940 10.134.0.24:443 100.64.0.4:20164 maas.apps.fde2cff5-837d-4ac4-8786-1ae9207f1e1f.prod.konfluxeaas.com opendatahub.maas-api-route.1