2026-06-03T20:08:31.233018Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T20:08:31.206Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 26 - "100.64.0.3" "python-requests/2.32.5" "cf5f678e-453a-4a77-aa9c-212b4fbace32" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:48782 10.132.0.52:443 100.64.0.3:29514 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:08:47.336542Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:08:47.336586Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:08:47.351038Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:08:47.351170Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:08:47.351409Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:08:47.355234Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:08:47.355328Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:08:47.355340Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T20:08:47.335Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 45 329 19 - "100.64.0.2" "python-requests/2.32.5" "acc83460-48a2-47b3-bf34-508d6e4976fd" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:51568 10.132.0.52:443 100.64.0.2:2912 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:08:55.383162Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:08:55.383214Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:08:55.397098Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:08:55.397227Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:08:55.407908Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:08:55.407944Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:08:55.414167Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:08:55.414315Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:08:55.414412Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:08:55.416841Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:08:55.416898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:08:55.416909Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:08:55.418014Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:08:55.418063Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:08:55.418074Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T20:08:55.407Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 9 - "10.133.0.32" "Go-http-client/1.1" "838804c5-7d4d-40a5-bc42-0034accedf02" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.34:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.52:44004 10.132.0.52:443 10.133.0.32:36866 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-03T20:08:55.382Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 471 36 - "100.64.0.2" "python-requests/2.32.5" "ba52228a-fb6a-4138-a6c6-54957d701c02" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:51568 10.132.0.52:443 100.64.0.2:29350 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:09:11.578461Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:11.578516Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:11.592138Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:11.592298Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:11.592514Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:11.595774Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:11.595843Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:09:11.595858Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:11.612012Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:11.612052Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:11.621759Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:11.621859Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:11.622024Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:11.624678Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:11.624738Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:09:11.624749Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T20:09:11.577Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 297 18 - "100.64.0.3" "python-requests/2.32.5" "5285e2c8-0499-4859-a5eb-480864766740" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:51568 10.132.0.52:443 100.64.0.3:3398 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:09:11.611Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 298 13 - "100.64.0.4" "python-requests/2.32.5" "5f7d9015-d6f2-4279-a06d-e8b0fb125324" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:51568 10.132.0.52:443 100.64.0.4:39298 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:09:19.655503Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:09:19.655551Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:09:19.659632Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:09:19.659754Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:09:19.662111Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:19.662151Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:19.668349Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:19.668463Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:19.668560Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:19.670326Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:19.670383Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:09:19.670394Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:19.671626Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:09:19.671684Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:09:19.671695Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:09:19.688424Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:09:19.688457Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:09:19.691533Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:09:19.691625Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:09:19.693607Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:19.693638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:19.699623Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:19.699726Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:19.699812Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:19.703382Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:19.703430Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:09:19.703440Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:19.705383Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:09:19.705440Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:09:19.705451Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T20:09:19.661Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 9 - "10.133.0.32" "Go-http-client/1.1" "1b36103a-5356-4da1-a861-eec90a607ba0" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.34:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.52:44004 10.132.0.52:443 10.133.0.32:36866 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-03T20:09:19.654Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 468 17 - "100.64.0.4" "python-requests/2.32.5" "2606c307-09d1-4dab-910d-47f718e1b54b" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:48782 10.132.0.52:443 100.64.0.4:28302 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:09:19.692Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 10 - "10.133.0.32" "Go-http-client/1.1" "4986e4f2-97c5-4eac-908b-cabe5f96effc" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.134.0.26:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.52:43056 10.132.0.52:443 10.133.0.32:36866 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-03T20:09:19.687Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 345 17 - "100.64.0.3" "python-requests/2.32.5" "011f4c84-dbef-4705-bb67-e83df1872803" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:48782 10.132.0.52:443 100.64.0.3:26746 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:09:38.208139Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.208197Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.223110Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.223225Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.225686Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.225718Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.235012Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.235118Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.236664Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.236723Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:09:38.236733Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.238012Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.238044Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.246955Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.247075Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.248574Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.248634Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:09:38.248649Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.249858Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.249895Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.257616Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.257715Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.259083Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.259130Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:09:38.259140Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.260212Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:38.260286Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:09:38.260299Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T20:09:38.224Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 11 - "10.133.0.32" "Go-http-client/1.1" "2edd457d-d876-49f8-859d-04f7ab72bdb3" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.34:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.52:44004 10.132.0.52:443 10.133.0.32:36866 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-03T20:09:38.237Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 11 - "10.133.0.32" "Go-http-client/1.1" "e76f2c5f-b6a3-4580-afaf-3ddd8ce619a7" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.134.0.26:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.52:43056 10.132.0.52:443 10.133.0.32:36866 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-03T20:09:38.249Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.133.0.32" "Go-http-client/1.1" "5e4728d8-3b51-4630-820b-7240f0b766ea" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.52:48860 10.132.0.52:443 10.133.0.32:36866 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T20:09:38.207Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 1361 53 - "100.64.0.2" "python-requests/2.32.5" "12b43bdc-239b-47bb-bf7c-1c596a92d7e2" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:51568 10.132.0.52:443 100.64.0.2:14046 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:09:56.757548Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:56.757597Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:56.771514Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:56.771636Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:56.773907Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:56.773940Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:56.783477Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:56.783606Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:56.785256Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:56.785327Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:09:56.785343Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:56.786497Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:56.786554Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:09:56.786569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:56.806075Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:09:56.806114Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:09:56.815323Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:09:56.815422Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:09:56.817492Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:56.817530Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:56.826043Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:56.826160Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:56.827752Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:56.827813Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:09:56.827828Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:09:56.828902Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:09:56.828956Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:09:56.828967Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T20:09:56.773Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 12 - "10.133.0.32" "Go-http-client/1.1" "f8a045ad-92a2-471a-a153-aa5cdefeaff6" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.34:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.52:44004 10.132.0.52:443 10.133.0.32:36866 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-03T20:09:56.756Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 470 30 - "100.64.0.3" "python-requests/2.32.5" "574e9fc8-8ec8-447b-ab0e-92d3038fb4a6" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:51568 10.132.0.52:443 100.64.0.3:19522 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:09:56.816Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 11 - "10.133.0.32" "Go-http-client/1.1" "1644e321-21d8-4fa9-a111-50f197002276" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.134.0.26:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.52:43056 10.132.0.52:443 10.133.0.32:36866 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-03T20:09:56.805Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 347 23 - "100.64.0.4" "python-requests/2.32.5" "0fba0070-bc6a-4763-a0bb-3628ccf434a0" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:48782 10.132.0.52:443 100.64.0.4:54164 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:10:05.830082Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:10:05.830133Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:10:05.831710Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:10:05.831753Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T20:10:05.828Z] "GET /maas-api/v1/models HTTP/1.1" 401 - - "-" 0 0 2 - "100.64.0.2" "python-requests/2.32.5" "1a870d97-ce5a-416a-9686-8b537ed032ce" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.132.0.52:443 100.64.0.2:11038 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:10:22.437520Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.437567Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.453256Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.453386Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.453599Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.457682Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.457741Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:10:22.457752Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.477837Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.477880Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.486977Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.487093Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.487447Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.487490Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.489364Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.489404Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.489696Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.492142Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.492338Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.492569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.493332Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.510666Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.510698Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.517199Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.517328Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.517543Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.517577Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.518124Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.518148Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.536806Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:10:22.536850Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:10:22.543271Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:10:22.543381Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:10:22.543603Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:10:22.543635Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:10:22.544228Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:10:22.544277Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T20:10:22.436Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 330 21 - "100.64.0.3" "python-requests/2.32.5" "64463f42-673e-4a1a-823e-5013fd2c9744" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:51568 10.132.0.52:443 100.64.0.3:12430 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:10:22.477Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 16 - "100.64.0.2" "python-requests/2.32.5" "4b788d6f-5efc-4e4c-ae89-11fcc7c025fc" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.52:59766 10.132.0.52:443 100.64.0.2:49168 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-03T20:10:22.510Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 8 - "100.64.0.4" "python-requests/2.32.5" "2235fa2e-8b48-43dc-b467-d9dc2714264e" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.52:59766 10.132.0.52:443 100.64.0.4:10386 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-03T20:10:22.536Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 8 - "100.64.0.4" "python-requests/2.32.5" "3a107427-d090-4353-b03a-9bd2e3667647" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.52:443 100.64.0.4:3048 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 2026-06-03T20:10:22.563785Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:10:22.563820Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:10:22.566619Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:10:22.566707Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:10:22.568979Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.569011Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.575041Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.575158Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.575274Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.577900Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.577966Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:10:22.577981Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:22.579403Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:10:22.579456Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:10:22.579466Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T20:10:22.568Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.133.0.32" "Go-http-client/1.1" "a9adc5fe-f1e3-4278-9bca-cc8360472794" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.134.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.52:46304 10.132.0.52:443 10.133.0.32:36866 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T20:10:22.562Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 547 16 - "100.64.0.3" "python-requests/2.32.5" "b76bc8c0-1240-4689-b8e3-5bb1c009f78e" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:48782 10.132.0.52:443 100.64.0.3:46758 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:10:31.398804Z info ads ADS: new connection for node:6 2026-06-03T20:10:31.401064Z info cache read certificate from file resource=file-root:system 2026-06-03T20:10:40.262403Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.262450Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.276139Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.276308Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.276540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.279681Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.279750Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:10:40.279765Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T20:10:40.261Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 322 18 - "100.64.0.2" "python-requests/2.32.5" "a653c701-64ca-49da-b79e-da755f0ee2d4" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:51568 10.132.0.52:443 100.64.0.2:57576 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:10:40.622315Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.622367Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.623295Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.623337Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.641114Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.641154Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.642002Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.642035Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.662196Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.662231Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.668897Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.669007Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.669284Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.669321Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.670871Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.670895Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.671102Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.672048Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:10:40.672152Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:10:40.672163Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T20:10:40.621Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 2 - "100.64.0.2" "python-requests/2.32.5" "ce6d63f9-926b-4f1a-9d22-50f6d1893b4a" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.132.0.52:443 100.64.0.2:36044 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-06-03T20:10:40.640Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 1 - "100.64.0.3" "python-requests/2.32.5" "dc65c35a-523f-4cd8-b877-c109d2bfbaf7" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.132.0.52:443 100.64.0.3:50150 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-06-03T20:10:40.661Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 404 - - "-" 83 98 10 - "100.64.0.4" "python-requests/2.32.5" "aa5d1b74-b35e-4e31-91fe-a0ea2aed2078" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.132.0.52:443 100.64.0.4:23098 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com llm.e2e-external-model.0 2026-06-03T20:11:18.184292Z info ads ADS: "" 6 terminated 2026-06-03T20:11:20.088590Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.088640Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.091387Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.091498Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.091684Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.104932Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.104993Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:11:20.105004Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.185470Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.185513Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.206479Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.206513Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.228508Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.228542Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.230206Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.230259Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.248292Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.248327Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.249554Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.249584Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.591560Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.591608Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.594142Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.594281Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.594486Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.597875Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.597932Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:11:20.597943Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T20:11:20.087Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 17 - "100.64.0.4" "python-requests/2.32.5" "dc20a814-a69b-49ed-a3f4-5caf7325cd1b" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:51568 10.132.0.52:443 100.64.0.4:58620 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:11:20.184Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 37 0 22 - "100.64.0.2" "python-requests/2.32.5" "edf0ddf3-d46b-422d-a9a9-999dcbcb62df" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.132.0.52:443 100.64.0.2:27190 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:11:20.227Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 35 0 2 - "100.64.0.3" "python-requests/2.32.5" "10f29da9-8039-4d7b-a5af-02b3a05711db" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.132.0.52:443 100.64.0.3:47430 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:11:20.247Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 36 0 2 - "100.64.0.2" "python-requests/2.32.5" "519cc3f8-8558-43e8-a09e-2e6a9624e974" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.132.0.52:443 100.64.0.2:17480 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:11:20.590Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 28 312 7 - "100.64.0.3" "python-requests/2.32.5" "b91872d4-9fee-443e-a43f-db241bc2186d" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:51568 10.132.0.52:443 100.64.0.3:38278 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:11:20.792957Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.793003Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.795632Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.795751Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.795941Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.798860Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.798922Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:11:20.798933Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.816191Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.816225Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.820265Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.820367Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.822812Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.822844Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.829081Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.829209Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.829318Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.830852Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.830905Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:11:20.830915Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.832323Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.832373Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:11:20.832383Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:20.851572Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.851604Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.857782Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.857902Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.858117Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.858145Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.859799Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.859824Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.860003Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.861703Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.861819Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.861974Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.862457Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.944328Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.944373Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.946804Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.946908Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.947086Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.949827Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.949881Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:11:20.949891Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.968430Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.968463Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.970820Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.970943Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.975216Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:20.975285Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:11:20.975297Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T20:11:20.791Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 312 7 - "100.64.0.4" "python-requests/2.32.5" "68a7424d-43e6-4391-bb3b-125409d9dacb" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:48782 10.132.0.52:443 100.64.0.4:14344 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:11:20.822Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.133.0.32" "Go-http-client/1.1" "e3ed54b6-a537-4efa-b5b9-66579045a19a" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.52:48860 10.132.0.52:443 10.133.0.32:36866 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T20:11:20.814Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 17 - "100.64.0.4" "python-requests/2.32.5" "0285f5fd-a122-455c-8382-d03529b86544" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:51568 10.132.0.52:443 100.64.0.4:58622 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:11:20.851Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 123 337 11 - "100.64.0.3" "python-requests/2.32.5" "06a15e9a-e633-42fc-9193-c329edc0d6f6" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.52:40132 10.132.0.52:443 100.64.0.3:47438 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-06-03T20:11:20.943Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 6 - "100.64.0.2" "python-requests/2.32.5" "6e85a853-e993-4069-9ae3-053b8f5a63cc" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:48782 10.132.0.52:443 100.64.0.2:17484 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:11:20.967Z] "DELETE /v1/api-keys/14ac8b29-dd7c-47f5-98b9-7798c65efdd7 HTTP/1.1" 200 - via_upstream - "-" 0 251 7 - "100.64.0.4" "python-requests/2.32.5" "5ce504a8-0a50-47f8-a2b2-ad14091a13df" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:48782 10.132.0.52:443 100.64.0.4:14356 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:11:23.996695Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:23.996747Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.000611Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.000687Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.002432Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.002489Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:11:24.002499Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.123667Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.123707Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.137856Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.137899Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.276927Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.276980Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.279522Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.279649Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.279851Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.283286Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.283352Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:11:24.283367Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.303406Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.303447Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.305690Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.305797Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.305964Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.308998Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.309059Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:11:24.309073Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.390681Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.390731Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.393329Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.393447Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.393634Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.396898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.396961Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:11:24.396971Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.413812Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.413848Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.416212Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.416327Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.419963Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.420016Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:11:24.420026Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.435466Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.435501Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.438036Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.438132Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.440466Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.440522Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:11:24.440532Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.526949Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.526995Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.529522Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.529633Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.529820Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.533009Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.533066Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:11:24.533076Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.548752Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.548789Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.552039Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.552158Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.554005Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.554038Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.560263Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.560382Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.560471Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.561968Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.562024Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:11:24.562034Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.563320Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.563385Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:11:24.563400Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.638447Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.638482Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.640956Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.641054Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.641214Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.644198Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.644270Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:11:24.644281Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.660299Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.660341Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T20:11:23.995Z] "GET /v1/models HTTP/1.1" 500 - via_upstream - "-" 0 96 6 - "100.64.0.3" "python-requests/2.32.5" "1fa25257-ef98-4a62-ab94-3d5fa5967ef1" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:51568 10.132.0.52:443 100.64.0.3:46142 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:11:24.122Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 38 0 15 - "100.64.0.2" "python-requests/2.32.5" "140b9e0d-3410-41fb-b372-ac7c094aa570" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.132.0.52:443 100.64.0.2:51554 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:11:24.275Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 34 318 7 - "100.64.0.4" "python-requests/2.32.5" "0f508da6-7bdb-4a99-9ed0-03e618b4912b" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:48782 10.132.0.52:443 100.64.0.4:15528 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:11:24.302Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 32 317 6 - "100.64.0.3" "python-requests/2.32.5" "0e8478b6-c4fa-4b15-ba30-fec81b114bbb" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:48782 10.132.0.52:443 100.64.0.3:46146 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:11:24.389Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 31 315 7 - "100.64.0.2" "python-requests/2.32.5" "13669f99-2b63-4659-b85a-3e87cc81fa18" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:51568 10.132.0.52:443 100.64.0.2:27198 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:11:24.412Z] "DELETE /v1/api-keys/66019ab8-79d0-4417-bd62-93e5f9d880cb HTTP/1.1" 200 - via_upstream - "-" 0 253 7 - "100.64.0.3" "python-requests/2.32.5" "ad541db4-3fc6-4764-a64e-2fb2d5bf81f3" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:51568 10.132.0.52:443 100.64.0.3:38290 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:11:24.434Z] "DELETE /v1/api-keys/66019ab8-79d0-4417-bd62-93e5f9d880cb HTTP/1.1" 404 - via_upstream - "-" 0 29 6 - "100.64.0.2" "python-requests/2.32.5" "d12619c3-1554-4f46-8b44-7548a6c14d0e" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:51568 10.132.0.52:443 100.64.0.2:51570 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:11:24.525Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 7 - "100.64.0.4" "python-requests/2.32.5" "b2fdcc80-0331-4585-8fcb-034c8a98166d" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:51568 10.132.0.52:443 100.64.0.4:15544 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:11:24.553Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.133.0.32" "Go-http-client/1.1" "c8554bf6-eda0-4b77-b7e1-a74c0c25e491" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.52:48860 10.132.0.52:443 10.133.0.32:36866 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T20:11:24.547Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 15 - "100.64.0.3" "python-requests/2.32.5" "39d1edd0-c4ab-4393-a8a8-9ac369f4e1ab" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:48782 10.132.0.52:443 100.64.0.3:46158 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:11:24.637Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 6 - "100.64.0.2" "python-requests/2.32.5" "808fe805-d22c-49dd-8b39-9818585a6994" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:51568 10.132.0.52:443 100.64.0.2:51578 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:11:24.663363Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.663463Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.664983Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.665031Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:11:24.665041Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.681482Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.681514Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.684322Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.684427Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.686220Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.686284Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.692505Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.692629Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.692723Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.694369Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.694428Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:11:24.694441Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.695550Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.695611Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:11:24.695626Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.775286Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.775327Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.777709Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.777805Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.777964Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.781297Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.781346Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:11:24.781356Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.797325Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.797356Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.800128Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.800228Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.802005Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.802036Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.807656Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.807778Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.807864Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.809465Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.809511Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:11:24.809522Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.810548Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.810594Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:11:24.810604Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.826687Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.826729Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.829778Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.829880Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.831536Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.831566Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.837400Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.837517Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.837603Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.839052Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.839097Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:11:24.839109Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:11:24.840200Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.840279Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:11:24.840291Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.917406Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.917443Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.919845Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.919945Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.920106Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.923183Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:11:24.923232Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:11:24.923260Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T20:11:24.659Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 5 - "100.64.0.4" "python-requests/2.32.5" "802564c8-b9a4-4eeb-8bd7-91115c099435" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:48782 10.132.0.52:443 100.64.0.4:15546 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:11:24.685Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.133.0.32" "Go-http-client/1.1" "c8823925-7c3b-4f3d-80ff-2f57867e4a13" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.52:48860 10.132.0.52:443 10.133.0.32:36866 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T20:11:24.680Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 15 - "100.64.0.4" "python-requests/2.32.5" "bdb94b85-94cd-4521-ab1b-8279da014dfb" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:48782 10.132.0.52:443 100.64.0.4:58632 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:11:24.774Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 7 - "100.64.0.3" "python-requests/2.32.5" "2da5f3f2-2c4c-474c-8587-c7434f021913" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:51568 10.132.0.52:443 100.64.0.3:46160 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:11:24.801Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.133.0.32" "Go-http-client/1.1" "8abf4bfe-7604-4851-aad1-ae001689e8b2" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.52:48860 10.132.0.52:443 10.133.0.32:36866 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T20:11:24.796Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 14 - "100.64.0.2" "python-requests/2.32.5" "340dce16-fff5-4f36-96e3-ead219d64b8c" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:51568 10.132.0.52:443 100.64.0.2:51584 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:11:24.830Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.133.0.32" "Go-http-client/1.1" "8400c7ba-14dd-4c38-8060-0e9a1efa9edb" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.52:48860 10.132.0.52:443 10.133.0.32:36866 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T20:11:24.825Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 14 - "100.64.0.4" "python-requests/2.32.5" "f93d7d3d-4790-4d27-8d0f-cf183021db04" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:48782 10.132.0.52:443 100.64.0.4:15550 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:11:24.916Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 36 320 6 - "100.64.0.3" "python-requests/2.32.5" "c5b02444-4985-497d-9490-68c0f1a8e53e" "maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com" "10.133.0.32:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.52:48782 10.132.0.52:443 100.64.0.3:46164 maas.apps.1fd4c4aa-417e-4700-817c-4a8707020974.prod.konfluxeaas.com opendatahub.maas-api-route.1