2026-06-09T08:20:19.973155Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:19.994970Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:19.995006Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.002351Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.002480Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.002728Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.002762Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.003832Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.003857Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.004057Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.006670Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.006796Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.006973Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.007376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.031550Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:20.031583Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:20.032380Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:20.032417Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:20.055259Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:20.055291Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:20.055969Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:20.055996Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:20.096602Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:20.096634Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:20.101317Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:20.101376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T08:20:19.950Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 69 312 22 - "100.64.0.4" "python-requests/2.32.5" "e0921c16-46b1-4b07-b0cd-a361c30a6546" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.25:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.35:42302 10.134.0.35:443 100.64.0.4:53956 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T08:20:19.993Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 292 13 - "100.64.0.2" "python-requests/2.32.5" "40365c8c-6054-4d99-bd9b-5f3373dfd170" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.30:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:43210 10.134.0.35:443 100.64.0.2:49072 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T08:20:20.028Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 66 0 4 - "100.64.0.2" "python-requests/2.32.5" "1ba40dad-9b09-499d-af79-1bb9fc8abc62" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.2:23286 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T08:20:20.054Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 66 0 1 - "100.64.0.3" "python-requests/2.32.5" "366caf39-3038-4364-aa45-fa1c06f61685" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.3:56168 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T08:20:20.095Z] "POST /v1/completions HTTP/1.1" 403 - - "-" 66 98 6 - "100.64.0.3" "python-requests/2.32.5" "80c55586-02dc-40bf-83d5-52e718ca477a" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.31:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.3:16444 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.premium-simulated-simulated-premium-kserve-route.0 2026-06-09T08:20:20.346228Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.346273Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.356324Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.356451Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.356627Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.360145Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.360199Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-06-09T08:20:20.360210Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.376663Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.376697Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.385885Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.385975Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.390590Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.390639Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-06-09T08:20:20.390650Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.409642Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.409674Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.419108Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.419198Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.419360Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.422556Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.422602Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-06-09T08:20:20.422612Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.442576Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:20.442612Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:20.451995Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:20.452094Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:20.455633Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:20.455682Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T08:20:20.455692Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:20.475928Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.475962Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.485669Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.485756Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.485900Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.487946Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.487990Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-06-09T08:20:20.487997Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.725383Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.725426Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.732034Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.732164Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.732420Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.732453Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.732929Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.732954Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.733156Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.735027Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.735153Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.735315Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.735690Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.758905Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.758938Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.763728Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:20.763758Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T08:20:20.344Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 37 331 15 - "100.64.0.4" "python-requests/2.32.5" "51c669a6-6e04-444c-9b17-5b5799e8a3bc" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.25:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.35:42306 10.134.0.35:443 100.64.0.4:53972 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T08:20:20.375Z] "DELETE /v1/api-keys/6c405ea7-9b48-438c-9189-fc9cd051b514 HTTP/1.1" 200 - via_upstream - "-" 0 307 15 - "100.64.0.4" "python-requests/2.32.5" "d53bdf13-8cf9-40d6-9f48-602fd5623b82" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.25:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.35:42306 10.134.0.35:443 100.64.0.4:9522 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T08:20:20.408Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 82 325 14 - "100.64.0.2" "python-requests/2.32.5" "78574736-dafe-4cb8-b5c8-188a0fcf2985" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.25:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.35:42306 10.134.0.35:443 100.64.0.2:49078 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T08:20:20.441Z] "DELETE /v1/api-keys/45bbf733-4ba6-493a-9fa6-86de47046bef HTTP/1.1" 200 - via_upstream - "-" 0 301 14 - "100.64.0.3" "python-requests/2.32.5" "1b72a9dd-cbda-4d69-b90d-1c7c3608060d" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.25:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.35:42302 10.134.0.35:443 100.64.0.3:56176 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T08:20:20.474Z] "POST /v1/api-keys HTTP/1.1" 400 - via_upstream - "-" 114 91 13 - "100.64.0.4" "python-requests/2.32.5" "104b91ea-6e7c-4ac4-86c7-5a35b4fe071e" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.25:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.35:42306 10.134.0.35:443 100.64.0.4:53984 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T08:20:20.724Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 11 - "100.64.0.2" "python-requests/2.32.5" "f636faf5-5e72-4821-9f39-ff1da94525cc" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.30:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:43210 10.134.0.35:443 100.64.0.2:49086 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T08:20:20.757Z] "POST /v1/completions HTTP/1.1" 403 - - "-" 66 98 6 - "100.64.0.3" "python-requests/2.32.5" "88d998f2-b403-4b71-b8ca-062fe2a8acd4" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.31:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.3:56186 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.premium-simulated-simulated-premium-kserve-route.0 2026-06-09T08:20:28.904067Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:28.904113Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:28.909626Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:28.909669Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T08:20:28.902Z] "POST /v1/completions HTTP/1.1" 403 - - "-" 66 98 7 - "100.64.0.2" "python-requests/2.32.5" "13a729c9-1e63-489b-aa7c-412dac0db981" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.31:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:38078 10.134.0.35:443 100.64.0.2:42048 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.premium-simulated-simulated-premium-kserve-route.0 [2026-06-09T08:20:47.011Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 224 - "-" "-" "-" "-" "-" - - 10.134.0.35:443 100.64.0.3:22068 - - 2026-06-09T08:20:53.617880Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:53.617929Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:53.631578Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:53.631693Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:53.631894Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:53.635633Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:53.635689Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-06-09T08:20:53.635701Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:53.664297Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.664350Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.671000Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.671114Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.671368Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.671398Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.672889Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.672912Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.673099Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.686476Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.686604Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.686775Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.687197Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.808436Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:53.808468Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:53.814618Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:53.814782Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:53.815094Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:53.815140Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:53.815625Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:53.815664Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:53.815944Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:53.819118Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:53.819254Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:53.819444Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:53.819823Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:53.949581Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.949615Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.956490Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.956599Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.956821Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.956851Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.957256Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.957280Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.957505Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.959482Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.959601Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.959761Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:53.960146Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:54.081734Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:54.081771Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:54.089829Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:54.089943Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:54.090196Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:54.090229Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:54.090638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:54.090664Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:54.090877Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:54.092727Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:54.092850Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:54.093019Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:54.093398Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:20:54.215900Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:54.215943Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:54.222818Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:54.222933Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:54.223176Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:54.223208Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:54.223613Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:54.223637Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:54.223844Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:54.225583Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:54.225709Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:54.225873Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:54.226235Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T08:20:53.616Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 87 329 19 - "100.64.0.4" "python-requests/2.32.5" "c6736094-4d8f-4783-9f82-2b5e60e62c7a" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.25:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.35:42306 10.134.0.35:443 100.64.0.4:11816 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T08:20:53.663Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 284 24 - "100.64.0.4" "python-requests/2.32.5" "783425e5-2d05-4910-b5e3-35bef0c57d3a" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.132.0.56:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:57580 10.134.0.35:443 100.64.0.4:21738 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T08:20:53.807Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 283 12 - "100.64.0.2" "python-requests/2.32.5" "857b209c-b0c4-42c2-9701-d30d90c1f5bf" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.132.0.56:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:57586 10.134.0.35:443 100.64.0.2:35006 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T08:20:53.948Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 284 11 - "100.64.0.3" "python-requests/2.32.5" "ca266bb9-b61e-4850-acda-c72fee34ad9d" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.132.0.56:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:57580 10.134.0.35:443 100.64.0.3:11608 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T08:20:54.080Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 12 - "100.64.0.4" "python-requests/2.32.5" "c27b9484-2c8f-4712-8be2-95620f1352e0" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.132.0.56:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:57586 10.134.0.35:443 100.64.0.4:11818 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T08:20:54.214Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 281 11 - "100.64.0.2" "python-requests/2.32.5" "1981f741-9a86-43c0-9155-d304c928b625" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.132.0.56:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:57580 10.134.0.35:443 100.64.0.2:35016 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 2026-06-09T08:20:54.359056Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:54.359090Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:54.365993Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:54.366097Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:54.366315Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:54.366361Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:54.366754Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:20:54.366778Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T08:20:54.358Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 8 - "100.64.0.2" "python-requests/2.32.5" "80d681fc-d41e-4462-9ebb-f2ed12876597" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.132.0.56:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:57580 10.134.0.35:443 100.64.0.2:19370 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 2026-06-09T08:21:19.503855Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.503904Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.518981Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.519106Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.519290Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.523451Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.523510Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T08:21:19.523521Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.547945Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.547981Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.554727Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.554838Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.555067Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.555100Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.556232Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.556258Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.556481Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.558270Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.558422Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.558584Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.559005Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.579647Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.579678Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.585366Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.585465Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.585671Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.585700Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.586105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.586130Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.586304Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.587916Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.588030Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.588185Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.588549Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:21:19.612156Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.612189Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.618130Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.618235Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.618469Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.618498Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.618900Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.618924Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.639923Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.639963Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.645405Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.645504Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.645722Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.645751Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.646137Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.646162Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.678226Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.678264Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.684485Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.684588Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.684686Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.687619Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:19.687677Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T08:21:19.687688Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T08:21:19.502Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 335 21 - "100.64.0.3" "python-requests/2.32.5" "7fa7b3e8-1ec1-4e5c-8b28-28c9b7b938e5" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.25:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.35:42302 10.134.0.35:443 100.64.0.3:6972 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T08:21:19.546Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 12 - "100.64.0.4" "python-requests/2.32.5" "22e1ab46-ffa0-4d28-b4b2-6122664eb03b" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.132.0.56:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:57586 10.134.0.35:443 100.64.0.4:48208 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T08:21:19.578Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 9 - "100.64.0.2" "python-requests/2.32.5" "6fdaba9d-d410-45d2-a0b5-0dabfd25fa4e" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.132.0.56:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:57586 10.134.0.35:443 100.64.0.2:34156 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T08:21:19.611Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 8 - "100.64.0.3" "python-requests/2.32.5" "a09c0917-e249-46f6-b31f-db3ffdf68cff" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.132.0.56:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.3:8822 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T08:21:19.638Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 7 - "100.64.0.4" "python-requests/2.32.5" "07b6b9f8-dd67-4e76-bec0-f09c10bb6db5" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.132.0.56:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:45608 10.134.0.35:443 100.64.0.4:15138 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T08:21:19.677Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 10 - "100.64.0.2" "python-requests/2.32.5" "115b3eb1-8c7f-47d8-a091-163f42356459" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.132.0.56:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:45620 10.134.0.35:443 100.64.0.2:34168 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.3 2026-06-09T08:21:28.357118Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:28.357164Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:28.364181Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:28.364310Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:28.364576Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:28.364609Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:28.365192Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:28.365216Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:28.365455Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:28.367299Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:28.367461Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:28.367630Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:28.368006Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T08:21:28.355Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 295 12 - "100.64.0.3" "python-requests/2.32.5" "5251e2e3-25bb-4fa5-983e-9054b5726eeb" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.30:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:34042 10.134.0.35:443 100.64.0.3:8826 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T08:21:44.933545Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:44.933592Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:44.948328Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:44.948465Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:21:44.948622Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T08:21:44.931Z] "POST /maas-api/v1/api-keys HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 73 0 16 - "100.64.0.4" "python-requests/2.32.5" "e44de3f1-324e-4bb1-bbda-9aacaeaca906" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.35:443 100.64.0.4:17210 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-09T08:22:09.939721Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:09.939766Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:09.947379Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:09.947504Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:09.947743Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:09.947774Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:09.948935Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:09.948958Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:09.949132Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T08:22:09.938Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 11 - "100.64.0.2" "python-requests/2.32.5" "a772b81b-b07e-4112-a9ce-4d0723b8a679" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.2:63020 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T08:22:11.974758Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:11.974815Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:11.981921Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:11.982072Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:11.982331Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:11.982393Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:11.982927Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:11.982960Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:11.983150Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T08:22:11.973Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 10 - "100.64.0.3" "python-requests/2.32.5" "a2bd4e38-48f0-4704-93c5-f90691139255" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.3:18968 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T08:22:14.014258Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:14.014306Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:14.021937Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:14.022070Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:14.022324Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:14.022373Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:14.022928Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:14.022952Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:14.023141Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T08:22:14.012Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 10 - "100.64.0.4" "python-requests/2.32.5" "64ee84d6-3db3-408c-9802-96a0cc36977f" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.4:38828 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T08:22:16.054970Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:16.055017Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:16.061557Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:16.061689Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:16.061948Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:16.061982Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:16.062514Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:16.062540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:16.062727Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T08:22:16.053Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 9 - "100.64.0.2" "python-requests/2.32.5" "c6ed1e14-73f1-433c-b840-e98866997f97" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.2:42144 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T08:22:18.088377Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:18.088420Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:18.094853Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:18.094983Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:18.095229Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:18.095262Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:18.095791Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:18.095816Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:18.096005Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T08:22:18.086Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 9 - "100.64.0.3" "python-requests/2.32.5" "df3f0b34-da2c-4d45-8687-77d875e4f55e" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.3:18696 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T08:22:20.119850Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:20.119894Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:20.126394Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:20.126521Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:20.126767Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:20.126799Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:20.127314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:20.127359Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:20.127540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T08:22:20.118Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 9 - "100.64.0.4" "python-requests/2.32.5" "741fcd3d-3972-449d-8c6c-9c38e16d3c1e" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.4:50348 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T08:22:22.152839Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:22.152884Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:22.160010Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:22.160141Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:22.160415Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:22.160447Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:22.160979Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:22.161004Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:22.161192Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T08:22:22.151Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 9 - "100.64.0.2" "python-requests/2.32.5" "465f9f17-f66a-4181-8f46-7d11bf389cf6" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.2:42150 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T08:22:24.186220Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:24.186265Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:24.193174Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:24.193321Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:24.193614Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:24.193660Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:24.194222Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:24.194252Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:24.194471Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T08:22:24.184Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 9 - "100.64.0.3" "python-requests/2.32.5" "ff9e7f53-78ed-4126-98b2-48f0a45c443c" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.3:18710 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T08:22:26.220781Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:26.220824Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:26.227616Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:26.227745Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:26.227998Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:26.228031Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:26.228581Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:26.228607Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:26.228790Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T08:22:26.219Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 9 - "100.64.0.3" "python-requests/2.32.5" "6794a0b5-0605-4280-bf43-c390389e0423" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.3:15774 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T08:22:28.252609Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:28.252661Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:28.259367Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:28.259508Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:28.259761Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:28.259803Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:28.260371Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:28.260409Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:28.260598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T08:22:28.251Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 9 - "100.64.0.4" "python-requests/2.32.5" "c89cbdfb-22f8-4c30-ad11-773a2ebfe823" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.4:26168 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T08:22:30.284047Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:30.284093Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:30.290833Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:30.290964Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:30.291217Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:30.291252Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:30.291781Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:30.291805Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:22:30.292001Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T08:22:30.282Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 9 - "100.64.0.2" "python-requests/2.32.5" "1f2fc50d-0116-406d-ae0e-ce62ee16bc66" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.2:20638 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T08:22:32.316923Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:32.316979Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:32.323539Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:32.323688Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:32.323962Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:32.324004Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:32.324569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:32.324599Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:32.324945Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:32.339891Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:32.340037Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:32.340223Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:32.340637Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T08:22:32.315Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 292 25 - "100.64.0.3" "python-requests/2.32.5" "5b009137-2c53-4d30-ac2a-e53949afcd54" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.30:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:43210 10.134.0.35:443 100.64.0.3:3260 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T08:22:48.875441Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:48.875486Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:48.875501Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:48.882589Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:48.882721Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:48.882825Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:48.883027Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:48.883056Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:48.884455Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:48.884482Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:48.884494Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:48.884695Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:48.884716Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:48.886570Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:48.886593Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:48.886704Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:48.886724Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:48.886880Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:22:48.887293Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T08:22:48.873Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 281 13 - "100.64.0.4" "python-requests/2.32.5" "834eaf6e-72d1-45e7-860f-6b7846af8d2a" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.30:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:43210 10.134.0.35:443 100.64.0.4:63226 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T08:23:05.748026Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:05.748072Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:05.754827Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:05.754961Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:05.755198Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:05.755230Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:05.756694Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:05.756719Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:05.757059Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:05.760385Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:05.760514Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:05.760677Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:05.761093Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T08:23:05.746Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 14 - "100.64.0.4" "python-requests/2.32.5" "99dd61b7-959e-4636-838c-b13a660dc40b" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.30:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:34042 10.134.0.35:443 100.64.0.4:51878 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T08:23:10.601Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 1601 - "-" "-" "-" "-" "-" - - 10.134.0.35:443 100.64.0.2:60042 - - 2026-06-09T08:23:22.773885Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:23:22.773929Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:23:22.779282Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:23:22.779314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T08:23:22.772Z] "POST /v1/completions HTTP/1.1" 403 - - "-" 66 32 7 - "100.64.0.2" "python-requests/2.32.5" "c138ee00-d79d-42bb-806e-48777dfe94d6" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.30:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:43210 10.134.0.35:443 100.64.0.2:53110 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T08:23:31.414450Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:31.414495Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:31.415651Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:31.415682Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T08:23:31.413Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 2 - "100.64.0.3" "python-requests/2.32.5" "17bb4313-b557-4e14-9a53-75cf733b51c5" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.3:28038 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 2026-06-09T08:23:39.696839Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:39.696902Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:39.718948Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:39.719089Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:39.719296Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:39.723875Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:39.723948Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T08:23:39.723964Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:39.746301Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:39.746358Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:39.746386Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:39.751276Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:39.751314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:39.914221Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:39.914263Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:39.914285Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:39.919328Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:39.919378Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T08:23:39.695Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 69 312 28 - "100.64.0.4" "python-requests/2.32.5" "94697627-5912-4dd2-9e10-4864237f4224" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.25:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.35:37730 10.134.0.35:443 100.64.0.4:22686 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T08:23:39.745Z] "POST /v1/completions HTTP/1.1" 403 - - "-" 66 13 6 - "100.64.0.3" "python-requests/2.32.5" "54deddef-96fa-469a-98d2-383438d0ae2b" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.31:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:38062 10.134.0.35:443 100.64.0.3:15238 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.premium-simulated-simulated-premium-kserve-route.0 [2026-06-09T08:23:39.913Z] "POST /v1/completions HTTP/1.1" 403 - - "-" 66 13 6 - "100.64.0.2" "python-requests/2.32.5" "1b0e46de-b6d7-42ce-82ed-a2dc6f99b87f" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.31:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.2:31618 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.premium-simulated-simulated-premium-kserve-route.0 2026-06-09T08:23:41.946965Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:41.947011Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:41.947025Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:41.953985Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:41.954102Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:41.954385Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:41.954419Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:41.954442Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:41.956205Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:41.956242Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:41.956364Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:41.956572Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:41.956670Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:41.958569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:41.958677Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:41.958740Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:41.958914Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:41.958951Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T08:23:41.959477Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T08:23:41.945Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 278 14 - "100.64.0.3" "python-requests/2.32.5" "b2f30bee-723d-48b2-b9dc-ed65e16769dd" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "10.133.0.31:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:56974 10.134.0.35:443 100.64.0.3:8246 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com llm.premium-simulated-simulated-premium-kserve-route.0 2026-06-09T08:24:31.078105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:24:31.078156Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:24:31.093849Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:24:31.093964Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:24:31.094114Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T08:24:31.076Z] "POST /maas-api/v1/api-keys HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 79 0 17 - "100.64.0.4" "python-requests/2.32.5" "28acfa39-0099-456a-8aba-2d5a91d325dd" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.35:443 100.64.0.4:38982 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-09T08:24:40.431679Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:24:40.431729Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:24:40.441837Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:24:40.441947Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:24:40.442099Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T08:24:40.429Z] "POST /maas-api/v1/api-keys HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 71 0 12 - "100.64.0.2" "python-requests/2.32.5" "2ec2a46d-9674-424b-b4fc-94e0f125b598" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.35:443 100.64.0.2:14398 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-09T08:24:58.700711Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:24:58.700759Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:24:58.718218Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:24:58.718440Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-06-09T08:24:58.718682Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-06-09T08:24:58.698Z] "POST /maas-api/v1/api-keys HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 82 0 19 - "100.64.0.4" "python-requests/2.32.5" "0b1be94b-b901-40c6-9b5f-e29f95789e8e" "maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.35:443 100.64.0.4:59068 maas.apps.08258af1-1d51-4b71-b0a5-3050798004eb.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T08:24:59.291Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 0 - "-" "-" "-" "-" "-" - - 10.134.0.35:443 100.64.0.3:50764 - -