2026-06-09T11:14:46.264348Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.264390Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.289700Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.289751Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.295458Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.295505Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.584484Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:14:46.584534Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:14:46.596640Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:14:46.596794Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:14:46.597029Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:14:46.600951Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:14:46.601020Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-09T11:14:46.601031Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:14:46.620739Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.620792Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.632406Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.632561Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.636791Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.636868Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T11:14:46.636884Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.660556Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.660633Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.673125Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.673278Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.673506Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.677142Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.677222Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T11:14:46.677238Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.694943Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.694993Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.706054Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.706203Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.710443Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.710521Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T11:14:46.710537Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.733470Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:14:46.733517Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:14:46.744540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:14:46.744718Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:14:46.744939Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:14:46.746921Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:14:46.746978Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-09T11:14:46.746988Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-09T11:14:46.136Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 69 311 27 - "100.64.0.2" "python-requests/2.32.5" "e17c01c6-38a9-447d-b121-e700d93f1bd6" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.133.0.36:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.43:39530 10.134.0.43:443 100.64.0.2:13974 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T11:14:46.183Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 15 - "100.64.0.4" "python-requests/2.32.5" "1ec29d3a-9937-4ec5-b6cb-dfa674bf69b4" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.133.0.40:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:40802 10.134.0.43:443 100.64.0.4:17108 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T11:14:46.226Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 66 0 2 - "100.64.0.3" "python-requests/2.32.5" "bd0c05c0-0e94-4f5c-8eb0-71092d4994ec" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 100.64.0.3:26230 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T11:14:46.261Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 401 - - "-" 66 0 2 - "100.64.0.2" "python-requests/2.32.5" "0d41465c-971d-4a15-bff6-02d13ea3f308" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 100.64.0.2:13980 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T11:14:46.288Z] "POST /v1/completions HTTP/1.1" 403 - - "-" 66 98 7 - "100.64.0.4" "python-requests/2.32.5" "e2460bd3-7c02-463b-8c6a-30f51b773ae4" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 100.64.0.4:26744 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.premium-simulated-simulated-premium-kserve-route.0 [2026-06-09T11:14:46.582Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 37 330 18 - "100.64.0.4" "python-requests/2.32.5" "0608d793-947d-444e-9761-c473e205b41c" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.133.0.36:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.43:39530 10.134.0.43:443 100.64.0.4:17116 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T11:14:46.619Z] "DELETE /v1/api-keys/32af0ffd-c03d-4a4f-90cc-18399c13fcc6 HTTP/1.1" 200 - via_upstream - "-" 0 307 17 - "100.64.0.3" "python-requests/2.32.5" "ce3ff0be-8322-49be-bb7e-1b970e8075c5" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.133.0.36:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.43:39520 10.134.0.43:443 100.64.0.3:26236 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T11:14:46.659Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 82 324 18 - "100.64.0.2" "python-requests/2.32.5" "f7654090-39b5-4366-9b5b-5ec100f21fdb" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.133.0.36:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.43:39520 10.134.0.43:443 100.64.0.2:13990 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T11:14:46.693Z] "DELETE /v1/api-keys/88c73e52-2ad5-4269-9ed7-7cb6c98d58b3 HTTP/1.1" 200 - via_upstream - "-" 0 301 17 - "100.64.0.4" "python-requests/2.32.5" "094bb0c5-11cf-47f0-9ea1-0565f45dabb5" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.133.0.36:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.43:39520 10.134.0.43:443 100.64.0.4:17132 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T11:14:46.731Z] "POST /v1/api-keys HTTP/1.1" 400 - via_upstream - "-" 114 91 15 - "100.64.0.2" "python-requests/2.32.5" "cf157a5c-f426-42de-869d-b3cbd889d013" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.133.0.36:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.43:39530 10.134.0.43:443 100.64.0.2:56486 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-09T11:14:46.997807Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:46.997859Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:47.005147Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:47.005309Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:47.005672Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:47.005716Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:47.006327Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:47.006362Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:47.006633Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:47.008522Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:47.008700Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:47.008900Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:47.009340Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:47.039063Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:47.039113Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:47.044435Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:14:47.044474Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T11:14:46.996Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 13 - "100.64.0.3" "python-requests/2.32.5" "2fe768e7-b576-46bd-a8f1-37d3e1b82634" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.133.0.40:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:48226 10.134.0.43:443 100.64.0.3:54002 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T11:14:47.037Z] "POST /v1/completions HTTP/1.1" 403 - - "-" 66 98 7 - "100.64.0.4" "python-requests/2.32.5" "a7fcfee5-18c0-42d8-8d8c-1d3b95af3ead" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:52694 10.134.0.43:443 100.64.0.4:26746 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.premium-simulated-simulated-premium-kserve-route.0 [2026-06-09T11:14:51.300Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 0 - "-" "-" "-" "-" "-" - - 10.134.0.43:443 100.64.0.3:21060 - - [2026-06-09T11:14:51.334Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 0 - "-" "-" "-" "-" "-" - - 10.134.0.43:443 100.64.0.2:51956 - - 2026-06-09T11:14:55.205374Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:14:55.205423Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:14:55.221241Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:14:55.221284Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-09T11:14:55.203Z] "POST /v1/completions HTTP/1.1" 403 - - "-" 66 98 17 - "100.64.0.4" "python-requests/2.32.5" "84df91e1-e417-4afb-937f-532f7344dc98" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 100.64.0.4:41668 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.premium-simulated-simulated-premium-kserve-route.0 2026-06-09T11:15:19.867421Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:19.867472Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:19.883210Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:19.883362Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:19.883664Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:19.888268Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:19.888360Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T11:15:19.888377Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:19.913886Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:19.913939Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:19.921392Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:19.921517Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:19.921950Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:19.921998Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:19.923354Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:19.923385Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:19.923666Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:19.937674Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:19.937887Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:19.938140Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:19.938744Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.059110Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.059156Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.066663Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.066788Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.067120Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.067156Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.067815Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.067843Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.068079Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.071832Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.071982Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.072182Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.072721Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.192795Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.192841Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.200057Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.200185Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.200490Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.200526Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.201213Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.201239Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.201466Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.203557Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.203727Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.203906Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.204350Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.326313Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.326360Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.333689Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.333814Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.334205Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.334243Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.334791Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.334817Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.335116Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.337211Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.337351Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.337529Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.338006Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.460822Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.460869Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.468078Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.468202Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.468503Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.468537Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.469138Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.469161Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.469383Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.471432Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.471621Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.471821Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.472301Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:20.592573Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.592637Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.599950Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.600074Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.600370Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.600404Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.600949Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:20.600975Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T11:15:19.865Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 87 329 22 - "100.64.0.3" "python-requests/2.32.5" "158184f1-d9ac-43d2-a56d-0d595ef823ce" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.133.0.36:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.43:39520 10.134.0.43:443 100.64.0.3:17240 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T11:15:19.912Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 26 - "100.64.0.2" "python-requests/2.32.5" "5c379dac-b708-40ce-993a-a5e129419487" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.132.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:33854 10.134.0.43:443 100.64.0.2:7528 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T11:15:20.057Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 15 - "100.64.0.3" "python-requests/2.32.5" "199797f0-85f3-49ee-b80c-5a3121abdd83" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.132.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:33866 10.134.0.43:443 100.64.0.3:44874 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T11:15:20.191Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 283 13 - "100.64.0.4" "python-requests/2.32.5" "bb18b303-fb07-4014-9d91-98c2a4fb277c" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.132.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:33854 10.134.0.43:443 100.64.0.4:40484 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T11:15:20.324Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 13 - "100.64.0.2" "python-requests/2.32.5" "93a7e362-4ea7-4c88-8060-e04b6ef96536" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.132.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:33854 10.134.0.43:443 100.64.0.2:29212 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T11:15:20.459Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 284 13 - "100.64.0.4" "python-requests/2.32.5" "0ee18f0a-20cb-44ab-b1e4-1161e61cb967" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.132.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:33866 10.134.0.43:443 100.64.0.4:50236 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T11:15:20.591Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 9 - "100.64.0.2" "python-requests/2.32.5" "47daa0e0-1322-47a2-b0e3-70e2aa158ff6" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.132.0.31:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:33854 10.134.0.43:443 100.64.0.2:7540 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-09T11:15:24.828Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 114 - "-" "-" "-" "-" "-" - - 10.134.0.43:443 100.64.0.4:50252 - - [2026-06-09T11:15:24.828Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 115 - "-" "-" "-" "-" "-" - - 10.134.0.43:443 100.64.0.2:29240 - - [2026-06-09T11:15:24.828Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 115 - "-" "-" "-" "-" "-" - - 10.134.0.43:443 100.64.0.4:50246 - - [2026-06-09T11:15:24.827Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 116 - "-" "-" "-" "-" "-" - - 10.134.0.43:443 100.64.0.2:29224 - - [2026-06-09T11:15:24.829Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 114 - "-" "-" "-" "-" "-" - - 10.134.0.43:443 100.64.0.3:17272 - - [2026-06-09T11:15:24.826Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 117 - "-" "-" "-" "-" "-" - - 10.134.0.43:443 100.64.0.3:17244 - - [2026-06-09T11:15:24.828Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 115 - "-" "-" "-" "-" "-" - - 10.134.0.43:443 100.64.0.3:17260 - - 2026-06-09T11:15:45.693870Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:45.693924Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:45.711591Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:45.711757Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:45.711958Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-09T11:15:45.692Z] "POST /maas-api/v1/api-keys HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 93 0 19 - "100.64.0.2" "python-requests/2.32.5" "8d9b6d60-c00c-49c6-90ee-ac5424fd98fe" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.43:443 100.64.0.2:11896 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-09T11:15:54.388314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:54.388361Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:54.396665Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:54.396830Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:54.397139Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:54.397174Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:54.398665Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:54.398691Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:15:54.398894Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-09T11:15:54.386Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 12 - "100.64.0.3" "python-requests/2.32.5" "e3ef575a-76e4-4049-a523-e711dde48075" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 100.64.0.3:33658 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T11:15:56.426873Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:56.426917Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:56.426928Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:56.434425Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:56.434565Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:56.434866Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:56.434900Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:56.434916Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:56.435570Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:56.435595Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:56.435694Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:56.435841Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:56.435911Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T11:15:56.425Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 10 - "100.64.0.4" "python-requests/2.32.5" "03613e4e-4ee0-424f-9b1e-a76e6817373f" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 100.64.0.4:37448 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T11:15:58.465180Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:58.465227Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:58.465240Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:58.472652Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:58.472810Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:58.472927Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:58.473153Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:58.473182Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:58.473842Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:58.473867Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:58.473878Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:58.474065Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:15:58.474081Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T11:15:58.463Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 10 - "100.64.0.3" "python-requests/2.32.5" "da6607e1-e51c-4884-b51c-ae6ab85d0acb" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 100.64.0.3:46518 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T11:16:00.498691Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:00.498737Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:00.498750Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:00.506352Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:00.506489Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:00.506812Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:00.506847Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:00.506861Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:00.507447Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:00.507472Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:00.507553Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:00.507716Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:00.507786Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-09T11:16:00.497Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 10 - "100.64.0.2" "python-requests/2.32.5" "c56e969c-49d9-451c-8796-27462103493c" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 100.64.0.2:56544 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T11:16:02.534344Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:02.534391Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:02.534403Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:02.541889Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:02.542030Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:02.542147Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:02.542376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:02.542411Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:02.543019Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:02.543045Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:02.543057Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:02.543266Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:02.543283Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T11:16:02.532Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 10 - "100.64.0.4" "python-requests/2.32.5" "940ffbf7-188c-48a1-82d2-4cca17a5c5fb" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 100.64.0.4:4246 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T11:16:04.567841Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:04.567889Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:04.567902Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:04.575340Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:04.575482Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:04.575627Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:04.575869Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:04.575899Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:04.576498Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:04.576521Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:04.576536Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:04.576761Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:04.576779Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-09T11:16:04.566Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 10 - "100.64.0.2" "python-requests/2.32.5" "a2621cc9-a8c4-471c-9379-d9f7a3351609" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 100.64.0.2:41932 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T11:16:06.613511Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:06.613559Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:06.613570Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:06.621089Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:06.621231Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:06.621351Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:06.621634Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:06.621670Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:06.622277Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:06.622301Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:06.622313Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:06.622510Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:06.622528Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-09T11:16:06.611Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 10 - "100.64.0.4" "python-requests/2.32.5" "c4d896ff-8ede-4aa2-b59b-8658455cd130" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 100.64.0.4:49686 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T11:16:08.649671Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:08.649714Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:08.649726Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:08.656850Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:08.656990Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:08.657267Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:08.657309Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:08.657331Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:08.657958Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:08.657983Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:08.658064Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:08.658200Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:08.658270Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-09T11:16:08.648Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 10 - "100.64.0.3" "python-requests/2.32.5" "3cc149f1-80e1-4fd2-b6e5-cf9bd97d3c7c" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 100.64.0.3:62642 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T11:16:10.685756Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:10.685805Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:10.685818Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:10.693022Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:10.693170Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:10.693464Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:10.693498Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:10.693513Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:10.694102Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:10.694126Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:10.694207Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:10.694348Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:10.694416Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T11:16:10.684Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 10 - "100.64.0.3" "python-requests/2.32.5" "42224341-bc4a-404e-9ab1-16d0e9b2c7e5" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 100.64.0.3:50566 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T11:16:12.723803Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:12.723848Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:12.723861Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:12.730952Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:12.731090Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:12.731380Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:12.731415Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:12.731429Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:12.732041Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:12.732065Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:12.732145Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:12.732288Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:12.732359Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T11:16:12.721Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 10 - "100.64.0.4" "python-requests/2.32.5" "1bb6db16-1136-4f72-9950-e72920d5c5f6" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 100.64.0.4:39184 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T11:16:14.757596Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:14.757668Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:14.757680Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:14.764479Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:14.764645Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:14.764941Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:14.764975Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:14.764991Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:14.765591Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:14.765643Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:14.765728Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:14.765860Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:14.765942Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-09T11:16:14.756Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 9 - "100.64.0.2" "python-requests/2.32.5" "fbe04dc1-2b80-4897-ae47-01389b8276a9" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 100.64.0.2:60782 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T11:16:16.790799Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:16.790847Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:16.790861Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:16.798093Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:16.798231Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:16.798517Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:16.798552Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:16.798566Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:16.799207Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:16.799231Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:16.799310Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:16.799444Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:16.799540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T11:16:16.789Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 10 - "100.64.0.4" "python-requests/2.32.5" "43d840c8-c394-4bb5-b80d-a9464b2d02dc" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 100.64.0.4:41660 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T11:16:18.821847Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:18.821895Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:18.821908Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:18.829154Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:18.829295Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:18.829591Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:18.829645Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:18.829662Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:18.830284Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:18.830307Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:18.830387Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:18.830523Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:18.830596Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-09T11:16:18.820Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 10 - "100.64.0.2" "python-requests/2.32.5" "0dfeac9a-97aa-4936-889e-f643a9f93476" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 100.64.0.2:23638 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T11:16:20.854544Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:20.854590Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:20.854620Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:20.861769Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:20.861909Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:20.862190Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:20.862223Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:20.862238Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:20.862862Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:20.862887Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:20.862968Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:20.863102Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:20.863171Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-09T11:16:20.853Z] "POST /llm/facebook-opt-125m-simulated/v1/completions HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 66 0 10 - "100.64.0.3" "python-requests/2.32.5" "372ba3e6-4acd-4b49-a8e1-cb90c6de71fd" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 100.64.0.3:55778 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T11:16:22.887667Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:22.887719Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:22.887737Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:22.895182Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:22.895341Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:22.895662Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:22.895706Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:22.895727Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:22.896354Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:22.896384Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:22.896479Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:22.896848Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:22.896943Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:22.913538Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:22.913710Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:22.913792Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:22.914011Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:22.914060Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:22.914631Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T11:16:22.885Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 278 28 - "100.64.0.3" "python-requests/2.32.5" "cd4974c0-af43-456f-8d80-772d79192e51" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.133.0.40:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:48226 10.134.0.43:443 100.64.0.3:58574 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T11:16:39.447483Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:39.447533Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:39.463325Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:39.463491Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:39.463779Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:39.469845Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:39.469925Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-09T11:16:39.469941Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:16:39.492973Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:39.493015Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:39.493028Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:39.500223Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:39.500355Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:39.500674Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:39.500719Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:39.500734Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:39.501977Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:39.502000Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:39.502083Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:39.502397Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:39.502473Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:39.506380Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:39.506481Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:39.506543Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:39.506747Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:39.506784Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:16:39.507286Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-09T11:16:39.445Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 73 316 24 - "100.64.0.4" "python-requests/2.32.5" "5abf081a-0684-4621-b336-bcc72e025f91" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.133.0.36:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.43:44868 10.134.0.43:443 100.64.0.4:50914 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T11:16:39.491Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 278 15 - "100.64.0.2" "python-requests/2.32.5" "3446edb4-77ea-447c-90db-c990cfddc469" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:36242 10.134.0.43:443 100.64.0.2:17322 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.premium-simulated-simulated-premium-kserve-route.0 2026-06-09T11:17:04.376378Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:04.376427Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:04.383883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:04.384021Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:04.384306Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:04.384338Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:04.385660Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:04.385686Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:04.385937Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:04.387976Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:04.388110Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:04.388283Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:04.388757Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-09T11:17:04.374Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 14 - "100.64.0.2" "python-requests/2.32.5" "52618de0-e501-4794-9e4f-b60b87e6aab5" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.133.0.40:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:40802 10.134.0.43:443 100.64.0.2:6748 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T11:17:20.893497Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:20.893544Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:20.893556Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:20.900907Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:20.901056Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:20.901352Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:20.901386Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:20.901400Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:20.903012Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:20.903036Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:20.903116Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:20.903271Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:20.903344Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:20.905143Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:20.905238Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:20.905298Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:20.905466Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:20.905501Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:20.905951Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-09T11:17:20.891Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 284 14 - "100.64.0.3" "python-requests/2.32.5" "49637121-32c1-4bec-8475-b5512dea2991" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.133.0.40:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:40802 10.134.0.43:443 100.64.0.3:46610 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T11:17:37.749569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:17:37.749644Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:17:37.757352Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:17:37.757495Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:17:37.757830Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:17:37.757866Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:17:37.759249Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:17:37.759273Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:17:37.759502Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:17:37.761386Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:17:37.761517Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:17:37.761716Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:17:37.762199Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T11:17:37.747Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 283 14 - "100.64.0.4" "python-requests/2.32.5" "f712a56e-63c3-4e06-823e-58e2c2c0665f" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.133.0.40:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:48226 10.134.0.43:443 100.64.0.4:2112 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T11:17:54.721139Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:54.721187Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:54.727253Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:17:54.727292Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-09T11:17:54.719Z] "POST /v1/completions HTTP/1.1" 403 - - "-" 66 32 8 - "100.64.0.4" "python-requests/2.32.5" "ec3af543-c156-4cfe-9ec0-b95a2a18136c" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "10.133.0.40:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.43:40802 10.134.0.43:443 100.64.0.4:17108 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.0 2026-06-09T11:18:03.328980Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:18:03.329029Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:18:03.330168Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:18:03.330198Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-09T11:18:03.327Z] "POST /llm/e2e-unconfigured-facebook-opt-125m-simulated/v1/completions HTTP/1.1" 403 - - "-" 66 0 2 - "100.64.0.2" "python-requests/2.32.5" "f6bdd60d-f095-4720-bbaa-fd6fb61ee7e1" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "-" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.43:443 100.64.0.2:14254 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 2026-06-09T11:18:11.602473Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:18:11.602541Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:18:11.628396Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:18:11.628560Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:18:11.629053Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-09T11:18:11.629799Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-09T11:18:11.600Z] "POST /maas-api/v1/api-keys HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{upstream_connect_error_or_disconnect/reset_before_headers._reset_reason:_remote_connection_failure,_transport_failure_reason:_delayed_connect_error:_Connection_refused{upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}}} - "-" 69 0 29 - "100.64.0.3" "python-requests/2.32.5" "b36f9615-ee45-46a7-bf94-c54076d35f2a" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.43:443 100.64.0.3:26464 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-09T11:18:40.430Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 34 - "-" "-" "-" "-" "-" - - 10.134.0.43:443 100.64.0.3:64792 data.phishtank.com - 2026-06-09T11:18:59.677562Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:18:59.677636Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:18:59.694667Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:18:59.694816Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:18:59.695004Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-09T11:18:59.675Z] "POST /maas-api/v1/api-keys HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 79 0 19 - "100.64.0.2" "python-requests/2.32.5" "dd20bb32-afa1-4ec2-93b8-aeec29847245" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.43:443 100.64.0.2:33584 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-09T11:19:08.871549Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:19:08.871628Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:19:08.882796Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:19:08.882948Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-09T11:19:08.883138Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-09T11:19:08.869Z] "POST /maas-api/v1/api-keys HTTP/1.1" 500 - ext_proc_error_gRPC_error_14{no_healthy_upstream{no_healthy_upstream}} - "-" 71 0 13 - "100.64.0.4" "python-requests/2.32.5" "bee38448-a91a-43ab-be77-699eee7a4548" "maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.43:443 100.64.0.4:35928 maas.apps.5d3ea805-67f8-4911-9d19-96bc9a325e00.prod.konfluxeaas.com opendatahub.maas-api-route.1