2026-06-08T23:09:32.885757Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:32.885782Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:32.885969Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:32.887785Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:32.887906Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:32.888063Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:32.888482Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-08T23:09:32.736Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 16 - "100.64.0.4" "python-requests/2.32.5" "4fe73b26-07c8-4adc-87e2-9e315b0ed77b" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.134.0.47:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.35:54760 10.134.0.35:443 100.64.0.4:41506 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T23:09:32.773Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 290 14 - "100.64.0.3" "python-requests/2.32.5" "82a3aa87-e16c-45ff-af43-a04f27712c04" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:35744 10.134.0.35:443 100.64.0.3:22062 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-08T23:09:32.815Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 287 12 - "100.64.0.2" "python-requests/2.32.5" "6a408f74-b5a6-4ce3-9f3c-95e9bdb10f35" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:35744 10.134.0.35:443 100.64.0.2:5106 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-08T23:09:32.845Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 284 10 - "100.64.0.4" "python-requests/2.32.5" "a02fb70e-d1f1-4ddf-9e28-fd58847260b9" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:35760 10.134.0.35:443 100.64.0.4:41522 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-08T23:09:32.877Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 10 - "100.64.0.4" "python-requests/2.32.5" "9123f4d4-478d-41d0-a29a-6f278749c7f1" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:35744 10.134.0.35:443 100.64.0.4:60670 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-08T23:09:44.772695Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.772742Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.786973Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.787122Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.787341Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.791737Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.791800Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-08T23:09:44.791810Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.815424Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.815462Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.822348Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.822533Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.822798Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.822831Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.824624Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.824648Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.824858Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.827295Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.827448Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.827617Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.828072Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.848636Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:09:44.848679Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:09:44.849587Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:09:44.849620Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:09:44.870807Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.870841Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.871669Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.871698Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.892174Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.892210Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.896660Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:44.896692Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-08T23:09:44.770Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 69 312 21 - "100.64.0.3" "python-requests/2.32.5" "12e5f6bd-b825-4715-893f-b5b9c1b04621" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.134.0.47:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.35:54754 10.134.0.35:443 100.64.0.3:11958 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T23:09:44.814Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 14 - "100.64.0.2" "python-requests/2.32.5" "e4bdd195-f682-4b3c-aa8d-04f2397bb2f5" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:35744 10.134.0.35:443 100.64.0.2:55708 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-08T23:09:44.847Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 66 0 2 - "100.64.0.4" "python-requests/2.32.5" "99350c0a-abf2-4036-9f59-be0e4883a31c" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.4:2338 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-08T23:09:44.869Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 66 0 2 - "100.64.0.3" "python-requests/2.32.5" "c7132304-39d6-448a-9e8c-a4aa874f46f1" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.3:11960 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-08T23:09:44.891Z] "POST /v1/completions HTTP/1.1" 403 - - "-" 66 98 5 - "100.64.0.2" "python-requests/2.32.5" "dd18dd74-b626-47fc-8c7a-bebdbe8a2c28" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.134.0.53:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.2:55722 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.premium-simulated-simulated-premium-kserve-route.0 2026-06-08T23:09:45.158311Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.158357Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.168975Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.169112Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.169318Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.172523Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.172586Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-08T23:09:45.172597Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.193693Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.193729Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.203752Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.203869Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.207096Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.207156Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-08T23:09:45.207166Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.229270Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.229305Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.239651Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.239778Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.239976Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.243416Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.243476Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-08T23:09:45.243486Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.264602Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:09:45.264648Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:09:45.275776Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:09:45.275922Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:09:45.279363Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:09:45.279454Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-08T23:09:45.279470Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:09:45.301300Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:09:45.301348Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:09:45.311244Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:09:45.311416Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:09:45.311635Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:09:45.313672Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:09:45.313743Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-08T23:09:45.313759Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:09:45.559206Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.559250Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.566000Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.566134Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.566416Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.566449Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.567065Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.567089Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.567308Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.569179Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.569306Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.569492Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.569905Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.591444Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.591482Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.596112Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:09:45.596163Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-08T23:09:45.156Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 37 331 15 - "100.64.0.4" "python-requests/2.32.5" "66253a44-1e5f-4a3a-a959-471525a72306" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.134.0.47:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.35:54754 10.134.0.35:443 100.64.0.4:2346 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T23:09:45.192Z] "DELETE /v1/api-keys/e5eb22ce-6ab2-4792-a617-ad92616921e1 HTTP/1.1" 200 - via_upstream - "-" 0 307 14 - "100.64.0.3" "python-requests/2.32.5" "76e23b06-d9d0-4106-a34a-42389f0be691" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.134.0.47:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.35:54754 10.134.0.35:443 100.64.0.3:49384 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T23:09:45.227Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 82 324 15 - "100.64.0.3" "python-requests/2.32.5" "70992012-838f-43a6-94e5-1f3dcad7a204" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.134.0.47:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.35:54754 10.134.0.35:443 100.64.0.3:11966 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T23:09:45.263Z] "DELETE /v1/api-keys/77985b3e-1e64-4dc5-a869-295423577851 HTTP/1.1" 200 - via_upstream - "-" 0 301 16 - "100.64.0.2" "python-requests/2.32.5" "13c89a27-44c9-4caa-bd0a-5c870a2c0ea1" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.134.0.47:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.35:54760 10.134.0.35:443 100.64.0.2:55732 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T23:09:45.299Z] "POST /v1/api-keys HTTP/1.1" 400 - via_upstream - "-" 114 91 13 - "100.64.0.4" "python-requests/2.32.5" "1a89718e-c32a-4974-9279-9c093d875d23" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.134.0.47:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.35:54760 10.134.0.35:443 100.64.0.4:60466 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T23:09:45.557Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 278 12 - "100.64.0.2" "python-requests/2.32.5" "a4376d5e-72b3-41ad-9a81-e196275a50df" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:35744 10.134.0.35:443 100.64.0.2:43846 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-08T23:09:45.589Z] "POST /v1/completions HTTP/1.1" 403 - - "-" 66 98 6 - "100.64.0.3" "python-requests/2.32.5" "2ca059ef-ca63-4a0d-b0da-14eff33c0539" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.134.0.53:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:51442 10.134.0.35:443 100.64.0.3:34562 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.premium-simulated-simulated-premium-kserve-route.0 2026-06-08T23:09:53.743703Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:09:53.743748Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:09:53.748566Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:09:53.748605Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-08T23:09:53.742Z] "POST /v1/completions HTTP/1.1" 403 - - "-" 66 98 6 - "100.64.0.2" "python-requests/2.32.5" "b705429a-1fde-4930-a287-756a79a86ad4" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.134.0.53:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.2:15684 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.premium-simulated-simulated-premium-kserve-route.0 2026-06-08T23:10:18.381456Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.381502Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.395776Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.395913Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.396124Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.412511Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.412579Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-08T23:10:18.412589Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.434178Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.434215Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.441461Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.441577Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.441851Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.441883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.443260Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.443284Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.443506Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.457108Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.457247Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.457449Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.457958Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.582767Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.582808Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.589333Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.589467Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.589724Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.589756Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.590314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.590337Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.590575Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.592432Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.592554Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.592714Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.593101Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.715163Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.715204Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.721769Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.721897Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.722150Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.722181Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.722808Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.722831Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.723059Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.726236Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.726370Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.726565Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.726982Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.852693Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.852735Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.859166Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.859283Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.859557Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.859589Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.860133Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.860159Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.860364Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.862165Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.862288Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.862472Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.862850Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:18.987205Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.987247Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.993780Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.993898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.994173Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.994206Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.994795Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.994819Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.995050Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.996925Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.997052Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.997217Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:18.997643Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-08T23:10:18.379Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 87 329 32 - "100.64.0.4" "python-requests/2.32.5" "11556884-495e-4998-94c6-b421bb27acf3" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.134.0.47:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.35:54754 10.134.0.35:443 100.64.0.4:32280 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T23:10:18.433Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 24 - "100.64.0.3" "python-requests/2.32.5" "5c26a485-2021-4a13-8bbe-391097e1fbb8" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:34376 10.134.0.35:443 100.64.0.3:43666 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-08T23:10:18.581Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 11 - "100.64.0.4" "python-requests/2.32.5" "657fdf87-f1f1-4980-a26d-9d260ce685ec" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:34376 10.134.0.35:443 100.64.0.4:22148 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-08T23:10:18.713Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 13 - "100.64.0.3" "python-requests/2.32.5" "16a49665-7302-4cb1-a93a-cc4828f1dab3" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:34382 10.134.0.35:443 100.64.0.3:27980 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-08T23:10:18.851Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 11 - "100.64.0.2" "python-requests/2.32.5" "81959738-f7ed-4933-ae26-a8ca88ffe524" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:34382 10.134.0.35:443 100.64.0.2:30378 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-08T23:10:18.985Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 283 11 - "100.64.0.2" "python-requests/2.32.5" "7be99595-2608-4994-958a-e2bc570b57fd" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:34376 10.134.0.35:443 100.64.0.2:60432 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 2026-06-08T23:10:19.117967Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:19.118012Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:19.124487Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:19.124609Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:19.124871Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:19.124906Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:19.125487Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:19.125510Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-08T23:10:19.116Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 8 - "100.64.0.4" "python-requests/2.32.5" "8b6c9f87-9d26-438b-9737-f5e4d8775606" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:34382 10.134.0.35:443 100.64.0.4:22150 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 2026-06-08T23:10:44.203244Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.203289Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.219588Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.219734Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.219968Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.224319Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.224383Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-08T23:10:44.224418Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.247446Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:44.247483Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:44.258535Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:44.258672Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:44.258967Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:44.259033Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:44.260351Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:44.260382Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:44.260632Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:44.264899Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:44.265053Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:44.265242Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:44.265910Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:44.286733Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.286774Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.293547Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.293659Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.293911Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.293942Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.294460Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.294483Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.294690Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.296329Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.296480Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.296641Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.297034Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.315792Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.315830Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.322204Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.322314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.322575Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.322605Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.323094Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.323117Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.345633Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:44.345674Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:44.351593Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:44.351702Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:44.351936Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:44.351967Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:44.352496Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:44.352521Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:44.375421Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.375455Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.381377Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.381529Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.381641Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.384718Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:10:44.384782Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-08T23:10:44.384794Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-08T23:10:44.201Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 93 335 22 - "100.64.0.3" "python-requests/2.32.5" "9d938984-ffe8-4f27-aca8-ee7c6068155b" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.134.0.47:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.35:54760 10.134.0.35:443 100.64.0.3:54982 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T23:10:44.246Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 19 - "100.64.0.2" "python-requests/2.32.5" "bd2ab251-7def-48d8-9be3-5e50118b4fab" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:43020 10.134.0.35:443 100.64.0.2:30156 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-08T23:10:44.285Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 278 11 - "100.64.0.4" "python-requests/2.32.5" "24558387-dcdc-4ad4-bab2-657e4a06378f" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:34376 10.134.0.35:443 100.64.0.4:38144 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-08T23:10:44.314Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 8 - "100.64.0.4" "python-requests/2.32.5" "e752a110-c159-4bc1-ade6-c1ef25560747" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:34376 10.134.0.35:443 100.64.0.4:38374 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-08T23:10:44.344Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 8 - "100.64.0.3" "python-requests/2.32.5" "c85f4650-3ff6-4dda-9ce0-87b90dd9c7a4" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:43020 10.134.0.35:443 100.64.0.3:54986 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-08T23:10:44.374Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 10 - "100.64.0.3" "python-requests/2.32.5" "22391825-1549-49a6-aa42-a93b534dddca" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.40:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:43032 10.134.0.35:443 100.64.0.3:13734 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.3 2026-06-08T23:10:52.976005Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:52.976050Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:52.982520Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:52.982651Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:52.982905Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:52.982938Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:52.983541Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:52.983566Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:52.983781Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:52.985994Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:52.986132Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:52.986300Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:10:52.986747Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-08T23:10:52.974Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 12 - "100.64.0.2" "python-requests/2.32.5" "555e3968-35d1-4585-bd4b-f4f5aeafc9fa" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:35744 10.134.0.35:443 100.64.0.2:39832 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-08T23:11:09.493379Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:09.493447Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:09.508855Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:09.508999Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:09.509195Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-08T23:11:09.491Z] "POST /maas-api/v1/api-keys HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 73 0 17 - "100.64.0.2" "python-requests/2.32.5" "625deea6-6e61-4bba-a2c0-534de325b176" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "-" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local - 10.134.0.35:443 100.64.0.2:11392 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 2026-06-08T23:11:34.373244Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:34.373290Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:34.381469Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:34.381608Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:34.381885Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:34.381920Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:34.383486Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:34.383518Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:34.383713Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-08T23:11:34.371Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 11 - "100.64.0.4" "python-requests/2.32.5" "2a9787fe-bdd6-46e6-9549-dc41f4187f1c" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.4:30744 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-08T23:11:36.406353Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:36.406425Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:36.413388Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:36.413545Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:36.413845Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:36.413880Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:36.414505Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:36.414531Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:36.414736Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-08T23:11:36.404Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 9 - "100.64.0.3" "python-requests/2.32.5" "7bf60e5b-055f-4e77-ae7a-121ef596dae9" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.3:5220 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-08T23:11:38.442461Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:38.442505Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:38.451140Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:38.451282Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:38.451604Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:38.451638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:38.452297Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:38.452322Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:38.452550Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-08T23:11:38.441Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 11 - "100.64.0.4" "python-requests/2.32.5" "18fe88e5-948d-4762-a253-2554be55f33c" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.4:50988 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-08T23:11:40.483226Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:40.483280Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:40.489891Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:40.490043Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:40.490314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:40.490359Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:40.490997Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:40.491040Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:40.491251Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-08T23:11:40.481Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 9 - "100.64.0.3" "python-requests/2.32.5" "76eeb148-418c-4473-9eae-bfcb345c7d2b" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.3:34998 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-08T23:11:42.517553Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:42.517594Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:42.524098Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:42.524235Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:42.524524Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:42.524557Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:42.525103Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:42.525126Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:42.525317Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-08T23:11:42.516Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 9 - "100.64.0.2" "python-requests/2.32.5" "22fff470-c978-4d6a-ab08-9d805290286d" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.2:11882 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-08T23:11:44.556355Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:44.556431Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:44.563157Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:44.563292Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:44.563594Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:44.563631Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:44.564212Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:44.564237Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:44.564456Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-08T23:11:44.554Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 9 - "100.64.0.4" "python-requests/2.32.5" "4b43fd45-315e-4a33-a37a-e99152589123" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.4:50992 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-08T23:11:46.589335Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:46.589379Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:46.595625Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:46.595763Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:46.596042Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:46.596077Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:46.596687Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:46.596713Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:46.596912Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-08T23:11:46.587Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 9 - "100.64.0.3" "python-requests/2.32.5" "c8d0562a-f848-436c-97d0-4d2f09d635e3" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.3:35004 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-08T23:11:48.624884Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:48.624931Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:48.631431Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:48.631571Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:48.631859Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:48.631894Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:48.632434Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:48.632458Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:48.632656Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-08T23:11:48.623Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 9 - "100.64.0.2" "python-requests/2.32.5" "f731a12d-3daf-48b4-bc5d-fc232594c719" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.2:50306 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-08T23:11:50.658960Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:50.659014Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:50.665869Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:50.666013Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:50.666310Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:50.666345Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:50.666919Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:50.666945Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:50.667159Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-08T23:11:50.657Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 9 - "100.64.0.4" "python-requests/2.32.5" "7d66f95b-1c79-4c4f-b35d-c3102fabfa0c" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.4:14444 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-08T23:11:52.693453Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:52.693501Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:52.700083Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:52.700237Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:52.700533Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:52.700574Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:52.701104Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:52.701135Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:11:52.701335Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-08T23:11:52.692Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 9 - "100.64.0.2" "python-requests/2.32.5" "ba907a71-d5fe-41a9-a84e-978e6d203676" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.2:29568 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-08T23:11:54.729204Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:54.729249Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:54.735727Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:54.735865Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:54.736153Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:54.736185Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:54.736763Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:54.736791Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:54.737155Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:54.752347Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:54.752524Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:54.752708Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:11:54.753257Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-08T23:11:54.727Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 290 25 - "100.64.0.4" "python-requests/2.32.5" "bb44d162-82d8-4932-ade4-4ee65d302bc2" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:35760 10.134.0.35:443 100.64.0.4:57344 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-08T23:12:11.263687Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:11.263730Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:11.263745Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:11.270849Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:11.270995Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:11.271115Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:11.271344Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:11.271374Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:11.273121Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:11.273146Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:11.273158Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:11.273362Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:11.273381Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:11.275291Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:11.275312Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:11.275444Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:11.275465Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:11.275624Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:11.276041Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-08T23:12:11.262Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 287 13 - "100.64.0.3" "python-requests/2.32.5" "785f7bb2-e873-48a9-9461-73e5607b5154" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:35760 10.134.0.35:443 100.64.0.3:34876 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-08T23:12:28.114640Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:28.114683Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:28.121030Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:28.121170Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:28.121461Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:28.121494Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:28.122612Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:28.122635Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:28.122858Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:28.125044Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:28.125178Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:28.125347Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:12:28.125767Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-08T23:12:28.113Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 12 - "100.64.0.2" "python-requests/2.32.5" "2f94b5f8-4131-4026-96d1-5a92da5823c9" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:35760 10.134.0.35:443 100.64.0.2:63062 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-08T23:12:45.068143Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:12:45.068191Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:12:45.072892Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:12:45.072933Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-08T23:12:45.066Z] "POST /v1/completions HTTP/1.1" 403 - - "-" 66 32 6 - "100.64.0.3" "python-requests/2.32.5" "e145b2f2-b994-424d-aa1e-1af3560af55a" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.132.0.41:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:35744 10.134.0.35:443 100.64.0.3:1424 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-08T23:12:53.667205Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:12:53.667251Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:12:53.668313Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:12:53.668344Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-08T23:12:53.665Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 2 - "100.64.0.2" "python-requests/2.32.5" "af859411-5efd-465d-9056-b7d99ea43d2e" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.2:33536 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 2026-06-08T23:13:01.950978Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:13:01.951033Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:13:01.990072Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:13:01.990222Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:13:01.990666Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:13:01.999141Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:13:01.999234Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-08T23:13:01.999253Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:13:02.045973Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:13:02.046017Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:13:02.046030Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:13:02.051229Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-08T23:13:02.051269Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-08T23:13:01.949Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 69 311 49 - "100.64.0.4" "python-requests/2.32.5" "3b14cb6d-7fa0-47c8-958f-f8b349259094" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.134.0.47:8443" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local 10.134.0.35:58492 10.134.0.35:443 100.64.0.4:46388 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 [2026-06-08T23:13:02.044Z] "POST /v1/completions HTTP/1.1" 403 - - "-" 66 13 6 - "100.64.0.4" "python-requests/2.32.5" "f614f25a-7cca-4243-8b68-3ff53b013527" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.134.0.53:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.35:443 100.64.0.4:51288 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.premium-simulated-simulated-premium-kserve-route.0 2026-06-08T23:13:02.217461Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:02.217501Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:02.217513Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:02.222480Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:02.222525Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-08T23:13:02.216Z] "POST /v1/completions HTTP/1.1" 403 - - "-" 66 13 6 - "100.64.0.3" "python-requests/2.32.5" "bbb4a377-3176-480d-947e-a6d9d38a0d6c" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.134.0.53:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:40972 10.134.0.35:443 100.64.0.3:26114 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.premium-simulated-simulated-premium-kserve-route.0 2026-06-08T23:13:04.260935Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:04.260980Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:04.260993Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:04.267835Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:04.267958Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:04.268065Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:04.268273Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:04.268300Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:04.269979Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:04.270003Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:04.270017Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:04.270223Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:04.270242Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:04.273287Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:04.273311Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:04.273456Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:04.273478Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:04.273638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:04.274094Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-08T23:13:04.259Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 14 - "100.64.0.3" "python-requests/2.32.5" "8aa91db1-78d7-4c50-9471-4d9f1042ca70" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "10.134.0.53:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.134.0.35:59004 10.134.0.35:443 100.64.0.3:52616 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com llm.premium-simulated-simulated-premium-kserve-route.0 2026-06-08T23:13:52.537975Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:52.538023Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:52.554087Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:52.554224Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:13:52.554442Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-08T23:13:52.536Z] "POST /maas-api/v1/api-keys HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 79 0 18 - "100.64.0.2" "python-requests/2.32.5" "97cb96ba-3a16-45c7-a24e-9b9ae0ecb2dc" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "-" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local - 10.134.0.35:443 100.64.0.2:3512 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1 2026-06-08T23:14:01.759241Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:14:01.759288Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:14:01.770120Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:14:01.770266Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-08T23:14:01.770481Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-08T23:14:01.757Z] "POST /maas-api/v1/api-keys HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 71 0 12 - "100.64.0.2" "python-requests/2.32.5" "ffb96d5f-f8c1-4b83-8ce9-f94e2f469521" "maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com" "-" outbound|8443||maas-api.redhat-ai-gateway-infra.svc.cluster.local - 10.134.0.35:443 100.64.0.2:42470 maas.apps.6186d923-f800-4c32-897a-c9bd8d980991.prod.konfluxeaas.com redhat-ai-gateway-infra.maas-api-route.1