[2026-07-14T15:22:55.403Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_connection_timeout{upstream_reset_before_response_started{connection_timeout}}} - "-" 73 0 20016 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "18cc1260-e9f7-4844-816a-82e346d6049a" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.26:443 10.132.0.9:34000 - llm.facebook-opt-125m-simulated-kserve-route.0 2026-07-14T15:23:25.512890Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:23:25.512938Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:23:25.518333Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:23:25.518366Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-07-14T15:23:15.511Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 65 0 10006 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "8b0c7a5e-ee52-4837-8a09-1f5cc7b02e16" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.26:443 10.132.0.9:59764 - llm.facebook-opt-125m-simulated-kserve-route.0 2026-07-14T15:23:35.548541Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:23:35.548611Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:23:35.550142Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:23:35.550176Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:23:35.580902Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:23:35.580943Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:23:35.590163Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:23:35.590283Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:23:35.590450Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:23:35.593381Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:23:35.593430Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:23:35.593463Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-07-14T15:23:35.593473Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-07-14T15:23:25.546Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 65 0 10003 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "ba2c571f-9553-465f-a445-6df563916f7e" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.26:443 10.132.0.9:43204 - llm.facebook-opt-125m-simulated-kserve-route.0 [2026-07-14T15:23:35.579Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 317 13 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "b90db0ba-858d-4e96-97b1-a778d3a04e51" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:43246 10.134.0.26:443 10.132.0.9:38518 - odh-ai-gateway-infra.maas-api-route.3 2026-07-14T15:23:45.616286Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:23:45.616335Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:23:45.625912Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:23:45.626053Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:23:45.626479Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:23:45.626524Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:23:45.629450Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:23:45.629484Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:23:45.629895Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:23:55.628863Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:23:55.658387Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:23:55.658448Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:23:55.667071Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:23:55.667206Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:23:55.672589Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:23:55.672656Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:23:55.672706Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:23:55.672721Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-07-14T15:23:35.614Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_connection_timeout{upstream_reset_before_response_started{connection_timeout}}} - "-" 65 0 20014 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "13fdecef-42bb-47e0-98dc-648be8bcba88" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.26:443 10.132.0.9:38526 - llm.facebook-opt-125m-simulated-kserve-route.0 [2026-07-14T15:23:55.656Z] "DELETE /v1/api-keys/2d3d78ac-32d1-45f2-be49-5682a1356336 HTTP/1.1" 200 - via_upstream - "-" 0 361 15 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "c8a8c182-dc6d-4d7c-aa33-4043ad36ed4d" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:37500 - odh-ai-gateway-infra.maas-api-route.3 2026-07-14T15:24:07.712593Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:07.712638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:07.719625Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:07.719659Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-07-14T15:23:57.710Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 65 0 10008 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "bf4a74b5-001a-4e2f-b50c-4654f77dbae1" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.26:443 10.132.0.9:37508 - llm.facebook-opt-125m-simulated-kserve-route.0 2026-07-14T15:24:17.753325Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:17.753375Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:17.763266Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:17.763409Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:17.763684Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:17.763717Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:17.765882Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:17.765906Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:17.766267Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:27.766289Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:27.797613Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.797655Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.806653Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.806775Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.806939Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.810067Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.810117Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.810151Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:24:27.810160Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.837682Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.837724Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.845460Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.845589Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.848683Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.848729Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.848763Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:24:27.848772Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.876389Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:27.876425Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:27.884668Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:27.884782Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:27.886515Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:27.886589Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:27.886623Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-07-14T15:24:27.886632Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:27.913789Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.913824Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.921714Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.921819Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.922960Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.923004Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.923034Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:24:27.923042Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.949766Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.949800Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.956624Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.956718Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.956842Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.959141Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.959180Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.959210Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:24:27.959218Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:27.982857Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:27.982893Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:27.990785Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:27.990906Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:27.993826Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:27.993873Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:27.993908Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-07-14T15:24:27.993917Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:28.021490Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:28.021541Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:28.028909Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:28.029065Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:28.029253Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:28.031632Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:28.031694Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:28.031741Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:24:28.031754Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-07-14T15:24:07.750Z] "POST /llm/facebook-opt-125m-simulated/v1/chat/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_connection_timeout{upstream_reset_before_response_started{connection_timeout}}} - "-" 100 0 20015 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "49c17d97-84e6-42a7-a312-d91f7905316f" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.26:443 10.132.0.9:38976 - llm.facebook-opt-125m-simulated-kserve-route.2 [2026-07-14T15:24:27.796Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 30 314 13 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "1e07607b-14bc-49a4-855e-eb24a8544389" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:45636 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:24:27.836Z] "DELETE /v1/api-keys/223c35f4-c5ac-47fb-9a2d-d21826b49b38 HTTP/1.1" 200 - via_upstream - "-" 0 322 12 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "2a9817cd-a845-4f34-a9a8-be589873e8e8" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:45642 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:24:27.875Z] "DELETE /v1/api-keys/223c35f4-c5ac-47fb-9a2d-d21826b49b38 HTTP/1.1" 404 - via_upstream - "-" 0 29 11 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "2f9e5385-10b7-4c97-9633-2b705201651b" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:43246 10.134.0.26:443 10.132.0.9:45650 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:24:27.912Z] "DELETE /v1/api-keys/nonexistent-uuid-12345 HTTP/1.1" 404 - via_upstream - "-" 0 29 10 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "6037b0a2-3e91-43ef-82fc-11f53526266b" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:45664 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:24:27.948Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 74 316 10 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "c92ecfe6-6cd2-4180-9bd2-547c37116e7b" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:45670 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:24:27.981Z] "DELETE /v1/api-keys/e12e562c-c4db-41d3-ae39-c9ec3802a699 HTTP/1.1" 200 - via_upstream - "-" 0 324 12 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "3fc5a2d4-730e-471b-8ff1-c7d911f24eab" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:43246 10.134.0.26:443 10.132.0.9:45686 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:24:28.020Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 74 316 11 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "ada8b4e5-a24e-4698-bf86-b5428ba21f01" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:45694 - odh-ai-gateway-infra.maas-api-route.3 2026-07-14T15:24:38.059038Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.059088Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.064346Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.064379Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.247918Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.247964Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.256666Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.256791Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.256953Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.259650Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.259697Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.259734Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:24:38.259743Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.286680Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:38.286718Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:38.294473Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:38.294599Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:38.294745Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:38.296992Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:38.297033Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:38.297066Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-07-14T15:24:38.297074Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:38.325011Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:38.325042Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:38.332465Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:38.332578Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:38.332702Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:38.334906Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:38.334944Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:38.334974Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-07-14T15:24:38.334982Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:38.359431Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.359467Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.367577Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.367691Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.370610Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.370652Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.370683Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:24:38.370692Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.396120Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.396161Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.403986Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.404122Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.407094Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.407145Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.407182Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:24:38.407192Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.430153Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.430191Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.437428Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.437569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.440567Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.440616Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.440651Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:24:38.440661Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.466960Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.466996Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.474113Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.474224Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.474372Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.476633Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.476677Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.476711Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:24:38.476720Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.499817Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.499852Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.507512Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.507628Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.507771Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.509890Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.509934Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.509967Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:24:38.509976Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.534369Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.534419Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.541673Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.541781Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.541922Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.544203Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.544244Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:38.544277Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:24:38.544286Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-07-14T15:24:28.058Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 65 0 10006 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "2e9343e7-3254-4c94-8e50-b370bb388aaa" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.26:443 10.132.0.9:45702 - llm.facebook-opt-125m-simulated-kserve-route.0 [2026-07-14T15:24:38.246Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 69 312 13 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "b7006109-2c91-42b4-9921-966a5b461f07" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:59478 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:24:38.285Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 69 310 11 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "48036825-7c54-40ad-a4f4-478e04a0bd12" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:43246 10.134.0.26:443 10.132.0.9:59484 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:24:38.323Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 69 311 11 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "23a1ef8f-9908-4da0-86c1-4644ae2b28d5" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:43246 10.134.0.26:443 10.132.0.9:59498 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:24:38.358Z] "DELETE /v1/api-keys/38d0b781-71a1-431c-9c14-16c1265b342a HTTP/1.1" 200 - via_upstream - "-" 0 319 12 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "f385ff89-0282-499d-994a-d46f2957f3be" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:59504 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:24:38.394Z] "DELETE /v1/api-keys/7b01a092-e5e6-48d6-b68d-a4967ea2f06b HTTP/1.1" 200 - via_upstream - "-" 0 319 12 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "d03fe943-995c-45b6-8849-8f3d6242acf3" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:59516 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:24:38.428Z] "DELETE /v1/api-keys/b0b648bb-0332-464c-90dc-31fb7491f984 HTTP/1.1" 200 - via_upstream - "-" 0 319 11 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "14cb2ce5-1ebd-472d-8a0f-c1fc87704729" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:59526 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:24:38.465Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 11 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "657df7e4-9b17-4584-9851-ccbcdb5b2b30" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:59534 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:24:38.498Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 313 11 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "f79d69a7-4c5b-4a81-832f-12ca6eaea17b" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:59546 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:24:38.533Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 11 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "443eba29-61ed-434d-99e3-c7b4e5afb975" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:59548 - odh-ai-gateway-infra.maas-api-route.3 2026-07-14T15:24:48.567867Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:48.567926Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:48.577961Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:48.578117Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:48.578387Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:48.578439Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:48.580533Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:48.580588Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:48.580994Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:58.579924Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-07-14T15:24:38.567Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_connection_timeout{upstream_reset_before_response_started{connection_timeout}}} - "-" 65 0 20012 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "97f63860-ea30-4a96-8f53-5faa0e41bde4" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.26:443 10.132.0.9:59552 - llm.facebook-opt-125m-simulated-kserve-route.0 2026-07-14T15:24:58.975500Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:58.975575Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:58.983960Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:58.984101Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:58.984291Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:58.987363Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:58.987423Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:58.987471Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:24:58.987484Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:59.012839Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:59.012883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:59.020636Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:59.020757Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:59.020916Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:59.026580Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:59.026638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:59.026718Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:24:59.026739Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:59.058224Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:59.058265Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:59.065601Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:59.065716Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:59.065866Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:59.067245Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:59.067300Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:59.067367Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:24:59.067385Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:24:59.103870Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:59.103906Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:59.111078Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:59.111197Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:59.111341Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:59.113745Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:59.113794Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:59.113831Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-07-14T15:24:59.113841Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:59.672712Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:59.672757Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:59.681008Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:59.681127Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:59.682510Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:59.682582Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:24:59.682618Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-07-14T15:24:59.682627Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-07-14T15:24:58.974Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 77 321 13 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "b43e7565-66f7-4f13-be58-967caf85f716" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:44808 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:24:59.011Z] "POST /v1/api-keys/search HTTP/1.1" 200 - via_upstream - "-" 103 3535 15 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "c3e19887-0072-44d3-91af-86d6d8b9de1c" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:44822 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:24:59.057Z] "POST /v1/api-keys/search HTTP/1.1" 200 - via_upstream - "-" 77 3206 10 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "ae118d05-3c15-4533-8b30-40f6dd1c3d74" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:44834 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:24:59.102Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 75 320 11 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "6db0db30-5bc7-4896-a79b-123e5fe28e4c" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:43246 10.134.0.26:443 10.132.0.9:44848 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:24:59.671Z] "GET /v1/api-keys/1a2ef014-c909-4dd9-81c6-3c7581ae96fb HTTP/1.1" 200 - via_upstream - "-" 0 327 11 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "cf782fdf-f0be-4f04-9b92-8a68ec0c29a6" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:43246 10.134.0.26:443 10.132.0.9:44854 - odh-ai-gateway-infra.maas-api-route.3 2026-07-14T15:25:00.619133Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:25:00.619181Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:25:00.627701Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:25:00.627822Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:25:00.627993Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:25:00.630716Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:25:00.630763Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:25:00.630826Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-07-14T15:25:00.630836Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-07-14T15:25:00.617Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 68 310 13 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "a0faa192-4832-485b-a869-3a56cec93f26" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:43246 10.134.0.26:443 10.132.0.9:59600 - odh-ai-gateway-infra.maas-api-route.3 2026-07-14T15:25:16.056120Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:16.056177Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:16.066292Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:16.066442Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:16.066647Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:16.069475Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:16.069535Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:16.069596Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:25:16.069610Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-07-14T15:25:16.054Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 72 314 14 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "31ec1463-a5da-4dd2-a642-c07a1ae0cf40" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:35924 - odh-ai-gateway-infra.maas-api-route.3 2026-07-14T15:25:31.491048Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:31.491099Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:31.501080Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:31.501211Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:31.501386Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:31.502661Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:31.502711Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:31.502744Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:25:31.502752Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-07-14T15:25:31.489Z] "POST /v1/api-keys HTTP/1.1" 403 - via_upstream - "-" 68 100 13 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "c4deeaab-eaa1-4f16-8a66-3e5396d39fac" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:42148 - odh-ai-gateway-infra.maas-api-route.3 2026-07-14T15:25:46.954909Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:46.954959Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:46.971190Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:46.971324Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:46.971502Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:46.976981Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:46.977034Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:46.977070Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:25:46.977079Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-07-14T15:25:46.953Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 70 312 23 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "35fd666a-efe4-435e-b80a-e611e1f64133" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:47912 - odh-ai-gateway-infra.maas-api-route.3 2026-07-14T15:25:53.577167Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:53.577219Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:53.586336Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:53.586490Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:53.586704Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:53.587637Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:53.587686Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:25:53.587721Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:25:53.587730Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-07-14T15:25:53.575Z] "POST /v1/api-keys HTTP/1.1" 400 - via_upstream - "-" 80 94 12 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "5d36c211-39f0-449e-bc13-62237f92e7c3" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:45216 - odh-ai-gateway-infra.maas-api-route.3 2026-07-14T15:26:16.167728Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:16.168039Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:16.168072Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:16.170479Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:16.170563Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:16.170617Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-07-14T15:26:16.170640Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-07-14T15:26:16.166Z] "POST /v1/api-keys HTTP/1.1" 500 - via_upstream - "-" 71 96 4 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "88120972-a686-4caf-8b94-f17840139e05" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:39420 10.134.0.26:443 10.132.0.9:52172 - odh-ai-gateway-infra.maas-api-route.3 2026-07-14T15:26:21.203147Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.203198Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.214600Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.214725Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.214896Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.218351Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.218410Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.218449Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:26:21.218458Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.244938Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.244974Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.253040Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.253151Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.253297Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.255511Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.255575Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.255612Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:26:21.255621Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.278654Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.278691Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.288060Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.288177Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.288329Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.290678Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.290722Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.290757Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:26:21.290766Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.318232Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:21.318278Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:21.334852Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:21.334992Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:21.335169Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:21.340746Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:21.340801Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:21.340841Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-07-14T15:26:21.340851Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:21.366304Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:21.366348Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:21.373918Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:21.374040Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:21.377385Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:21.377442Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:21.377478Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-07-14T15:26:21.377487Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:21.405302Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.405345Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.413443Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.413591Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.416944Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.416996Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.417033Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:26:21.417042Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:21.439505Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:21.439557Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:21.447179Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:21.447328Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:21.450289Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:21.450339Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:21.450374Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-07-14T15:26:21.450383Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-07-14T15:26:21.201Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 16 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "6d71fe61-5bd8-4121-ba13-6a7c6c79e059" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:51836 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:26:21.243Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 313 11 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "37f5a62e-b196-42fe-b10e-9094c2db2696" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:51840 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:26:21.277Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 13 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "c5bc0d9e-0100-4eba-8e54-6b03e035972d" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:51844 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:26:21.316Z] "POST /v1/api-keys/search HTTP/1.1" 200 - via_upstream - "-" 122 363 24 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "adfd35a2-4974-435a-90ff-d1369c8113b1" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:39420 10.134.0.26:443 10.132.0.9:51850 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:26:21.364Z] "DELETE /v1/api-keys/01df2ca3-84eb-47f2-9611-d03bd105daa1 HTTP/1.1" 200 - via_upstream - "-" 0 320 12 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "54599ac6-6093-4c34-8c00-75d8b26852ee" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:39420 10.134.0.26:443 10.132.0.9:51854 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:26:21.403Z] "DELETE /v1/api-keys/d5a8da30-ffcd-4d9b-98dc-f3d7fcaa2cfe HTTP/1.1" 200 - via_upstream - "-" 0 320 13 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "282b7838-fe0a-4a2a-8097-83ce78c53727" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:51858 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:26:21.438Z] "DELETE /v1/api-keys/8efa145a-372e-487d-990b-cb3012aafb63 HTTP/1.1" 200 - via_upstream - "-" 0 320 12 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "7e9109b2-cce3-489d-bc0a-fde6c0186621" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:39420 10.134.0.26:443 10.132.0.9:51870 - odh-ai-gateway-infra.maas-api-route.3 2026-07-14T15:26:26.714795Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.714845Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.725086Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.725208Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.725373Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.728219Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.728268Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.728309Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:26:26.728318Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.752827Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:26.752873Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-07-14T15:26:26.713Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 68 310 15 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "091b2eae-21b1-4535-ba5a-1edf5730b41b" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:51884 - odh-ai-gateway-infra.maas-api-route.3 2026-07-14T15:26:26.761101Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:26.761236Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:26.761397Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:26.763931Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:26.763986Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:26.764031Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-07-14T15:26:26.764044Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:26.789147Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.789180Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.797315Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.797418Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.797563Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.802676Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.802723Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.802781Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:26:26.802794Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.824221Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:26.824263Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:26.832219Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:26.832336Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:26.835159Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:26.835220Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:26.835264Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-07-14T15:26:26.835277Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:26.857826Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.857863Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.865099Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.865204Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.867880Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.867924Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.867958Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:26:26.867967Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.902365Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.902420Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.910136Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.910264Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.910440Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.912599Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.912647Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:26.912683Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=26 2026-07-14T15:26:26.912692Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-07-14T15:26:26.751Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 68 311 12 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "381d87d1-d560-47fd-bfc1-dc55fd1ed097" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:39420 10.134.0.26:443 10.132.0.9:51888 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:26:26.787Z] "POST /v1/api-keys/search HTTP/1.1" 200 - via_upstream - "-" 77 3842 14 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "13604292-fd4f-420a-9ff6-2f2f5e5334fc" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:51894 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:26:26.822Z] "DELETE /v1/api-keys/de35fba3-9a2c-43fc-94e5-54d917946aed HTTP/1.1" 200 - via_upstream - "-" 0 318 12 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "ae62e6cc-fd87-4fe3-9aa1-bea3fd5444ec" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:39420 10.134.0.26:443 10.132.0.9:51898 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:26:26.856Z] "DELETE /v1/api-keys/dba369f1-9631-4b09-a9fa-4dc78a0f6520 HTTP/1.1" 200 - via_upstream - "-" 0 318 11 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "2e4362aa-8667-4694-ba6f-688ba7aa008d" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:51908 - odh-ai-gateway-infra.maas-api-route.3 [2026-07-14T15:26:26.900Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 11 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "16b0a0f1-8e10-4ca8-8a31-094d8110004a" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:36006 10.134.0.26:443 10.132.0.9:51918 - odh-ai-gateway-infra.maas-api-route.3 2026-07-14T15:26:36.934936Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:36.934986Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:36.934998Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:36.935010Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:36.954374Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:36.954540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:36.954877Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:36.954915Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:36.954930Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:36.954944Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:36.957033Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:36.957057Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:36.957140Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:36.957154Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:36.957496Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:36.957614Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:36.957631Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:46.956651Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:46.956783Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:46.956799Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:46.993346Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:46.993400Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:47.004282Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:47.004429Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:47.004641Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:47.007204Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:47.007264Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-07-14T15:26:47.007311Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-07-14T15:26:47.007325Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-07-14T15:26:26.934Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_connection_timeout{upstream_reset_before_response_started{connection_timeout}}} - "-" 66 0 20022 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "abd96440-9f6d-4c3b-b186-8ae04c20421e" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.26:443 10.132.0.9:51920 - llm.facebook-opt-125m-simulated-kserve-route.0 [2026-07-14T15:26:46.991Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 71 314 15 - "52.71.36.72,10.132.0.9" "python-requests/2.32.5" "a957373a-53bf-401a-83e7-2fbc91cea8eb" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "10.133.0.37:8443" outbound|8443||maas-api.odh-ai-gateway-infra.svc.cluster.local 10.134.0.26:39420 10.134.0.26:443 10.132.0.9:53712 - odh-ai-gateway-infra.maas-api-route.3 2026-07-14T15:26:57.028481Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:57.028532Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:57.037882Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:57.038035Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:57.038321Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:57.038356Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:57.040390Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:57.040415Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:26:57.040817Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 2026-07-14T15:27:07.039916Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=26 [2026-07-14T15:26:47.028Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_connection_timeout{upstream_reset_before_response_started{connection_timeout}}} - "-" 66 0 20011 - "52.71.36.72,10.132.0.9" "-" "1c61b84c-283e-407e-9a08-a8c82519003b" "maas.apps.f2870b7c-a9d8-4ac1-a7a6-e5c9692c1c9c.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.26:443 10.132.0.9:53716 - llm.facebook-opt-125m-simulated-kserve-route.0