[2026-06-02T05:15:40.483Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 19 - "100.64.0.2" "python-requests/2.32.5" "15bb27a2-953f-43ef-8464-60b75eeb4b6a" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44134 10.133.0.33:443 100.64.0.2:58260 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:15:54.505780Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:15:54.505841Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:15:54.519947Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:15:54.520082Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:15:54.520303Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:15:54.524024Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:15:54.524088Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:15:54.524104Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-06-02T05:15:54.504Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 45 330 19 - "100.64.0.2" "python-requests/2.32.5" "e0260d5e-c00d-4b44-8b75-7bffeaa5cbf5" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44754 10.133.0.33:443 100.64.0.2:52162 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:16:02.554595Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:02.554645Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:02.568434Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:02.568567Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:02.578306Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:02.578341Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:02.584767Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:02.584903Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:02.585013Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:02.588430Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:02.588488Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:16:02.588498Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:02.590702Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:02.590758Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:16:02.590770Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-06-02T05:16:02.577Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 11 - "10.133.0.35" "Go-http-client/1.1" "62c75927-b5c7-4762-94d3-ffb04d0d928c" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.134.0.53:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:55348 10.133.0.33:443 10.133.0.35:59370 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-02T05:16:02.553Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 471 37 - "100.64.0.4" "python-requests/2.32.5" "ca6bfe2c-9b0b-4c4a-9550-5f03296b5c3f" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44754 10.133.0.33:443 100.64.0.4:11518 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:16:16.715157Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:16.715209Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:16.731497Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:16.731621Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:16.731855Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:16.735272Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:16.735330Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:16:16.735340Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:16.758919Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:16:16.758956Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:16:16.770905Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:16:16.771046Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:16:16.771249Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:16:16.774237Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:16:16.774285Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:16:16.774294Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-06-02T05:16:16.713Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 298 21 - "100.64.0.3" "python-requests/2.32.5" "5b1ebffc-ea0b-4e9f-99c6-60e80b55705a" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44754 10.133.0.33:443 100.64.0.3:28462 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:16:16.757Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 297 16 - "100.64.0.4" "python-requests/2.32.5" "49736b0d-1618-4b79-87f5-a8714b5829b8" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44134 10.133.0.33:443 100.64.0.4:49234 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:16:24.810518Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.810567Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.814541Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.814679Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.816733Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.816763Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.823317Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.823435Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.823529Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.824798Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.824891Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:16:24.824904Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.825696Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.825744Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:16:24.825754Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.847428Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.847461Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.851028Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.851122Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.852795Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.852825Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.859511Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.859625Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.859740Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.862591Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.862638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:16:24.862646Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.864250Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:24.864296Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:16:24.864306Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-06-02T05:16:24.815Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 9 - "10.133.0.35" "Go-http-client/1.1" "74b1cd2a-0fd4-4a5b-81fa-b42639a70c1d" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.134.0.53:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:55348 10.133.0.33:443 10.133.0.35:59370 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-02T05:16:24.809Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 468 16 - "100.64.0.2" "python-requests/2.32.5" "801ce5a2-0b09-42ee-a2d5-7b6894795274" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44754 10.133.0.33:443 100.64.0.2:44572 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:16:24.851Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 10 - "10.133.0.35" "Go-http-client/1.1" "4780028d-d177-40e4-b394-49e71d27ac1e" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.134.0.52:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:60598 10.133.0.33:443 10.133.0.35:59370 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-02T05:16:24.846Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 345 17 - "100.64.0.3" "python-requests/2.32.5" "ed1f4ff1-e987-4371-afa3-00110176ad46" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44754 10.133.0.33:443 100.64.0.3:32742 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:16:43.429267Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:16:43.429317Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:16:43.443835Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:16:43.443962Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:16:43.446325Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:43.446361Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:43.456937Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:43.457063Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:43.458379Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:43.458434Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:16:43.458444Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:43.459803Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:43.459837Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:43.469448Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:43.469555Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:43.470781Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:43.470829Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:16:43.470839Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:43.472070Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:43.472101Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:43.481001Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:43.481102Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:43.482412Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:43.482465Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:16:43.482477Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:43.483358Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:16:43.483432Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:16:43.483447Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-06-02T05:16:43.445Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 13 - "10.133.0.35" "Go-http-client/1.1" "09166a42-9a37-47a8-b08a-597ee601d313" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.134.0.53:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:55348 10.133.0.33:443 10.133.0.35:59370 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-02T05:16:43.458Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 11 - "10.133.0.35" "Go-http-client/1.1" "3e472de3-413e-4069-8adb-7ec7e3e32764" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.134.0.52:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:60598 10.133.0.33:443 10.133.0.35:59370 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-02T05:16:43.471Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 11 - "10.133.0.35" "Go-http-client/1.1" "0fca772e-9fc9-48b8-aea3-b073e9d0bfb9" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.132.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:47000 10.133.0.33:443 10.133.0.35:59370 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-02T05:16:43.427Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 1361 55 - "100.64.0.4" "python-requests/2.32.5" "69a91f6c-e841-4e49-9cfe-dad379ff0e7d" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44134 10.133.0.33:443 100.64.0.4:49056 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:16:59.900988Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.901038Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.915824Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.915941Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.918220Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.918253Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.928241Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.928355Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.929614Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.929697Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:16:59.929710Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.930560Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.930610Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:16:59.930620Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.950649Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.950710Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.961291Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.961399Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.963464Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.963497Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.973426Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.973543Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.974884Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.974936Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:16:59.974947Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.975739Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:16:59.975786Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:16:59.975796Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-06-02T05:16:59.917Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 12 - "10.133.0.35" "Go-http-client/1.1" "e1cd033a-331f-43d8-a368-0a803a3f562e" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.134.0.53:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:55348 10.133.0.33:443 10.133.0.35:59370 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-02T05:16:59.899Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 470 30 - "100.64.0.3" "python-requests/2.32.5" "62c8aa3f-ee05-4e73-9394-a077c3b2f536" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44754 10.133.0.33:443 100.64.0.3:64576 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:16:59.962Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 12 - "10.133.0.35" "Go-http-client/1.1" "f6a06303-059f-43ce-9ac3-89ab3cd57e3a" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.134.0.52:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:60598 10.133.0.33:443 10.133.0.35:59370 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-02T05:16:59.949Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 347 26 - "100.64.0.2" "python-requests/2.32.5" "19583043-7216-4c79-b488-6f0166b75e8c" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44754 10.133.0.33:443 100.64.0.2:7512 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:17:08.980006Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:08.980064Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:08.981542Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:08.981584Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-06-02T05:17:08.978Z] "GET /maas-api/v1/models HTTP/1.1" 401 - - "-" 0 0 3 - "100.64.0.4" "python-requests/2.32.5" "9eab49c4-d9bf-4623-a9be-20a3e29d86c4" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.33:443 100.64.0.4:19854 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:17:25.613081Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.613133Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.628272Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.628387Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.628590Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.633530Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.633607Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:17:25.633624Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.656482Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:25.656518Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:25.664471Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:25.664586Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:25.664860Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:25.664890Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:25.665875Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:25.665898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:25.666116Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:25.669611Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:25.669768Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:25.669942Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:25.670390Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:25.691961Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.692003Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.699169Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.699310Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.699567Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.699606Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.699935Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.699969Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.722409Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.722445Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.729295Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.729404Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.729627Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.729674Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.729979Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.730000Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.751724Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.751756Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.755074Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.755167Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.757220Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:25.757253Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:25.763684Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:25.763793Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:25.763889Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:25.764960Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:25.765009Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:17:25.765019Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:25.766040Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:25.766091Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:17:25.766102Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-06-02T05:17:25.611Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 331 21 - "100.64.0.2" "python-requests/2.32.5" "f267da69-691a-417b-8ec6-f54a3dc8f3af" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44134 10.133.0.33:443 100.64.0.2:54802 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:17:25.655Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 282 14 - "100.64.0.3" "python-requests/2.32.5" "cd26143d-f859-48ce-9b4e-f1b46c747389" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.134.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:42854 10.133.0.33:443 100.64.0.3:5376 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-02T05:17:25.691Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 8 - "100.64.0.2" "python-requests/2.32.5" "225f54aa-326c-45fa-a2a4-b30c905ad8d9" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.134.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:59568 10.133.0.33:443 100.64.0.2:46506 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-02T05:17:25.722Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 8 - "100.64.0.4" "python-requests/2.32.5" "7bc6bb63-0abf-43e0-84f2-b9e5a4300225" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.134.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.33:443 100.64.0.4:62714 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-02T05:17:25.756Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.133.0.35" "Go-http-client/1.1" "33198cca-69c1-499c-8d4a-91a2ebb25725" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.134.0.54:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:42854 10.133.0.33:443 10.133.0.35:59370 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.3 [2026-06-02T05:17:25.750Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 547 15 - "100.64.0.3" "python-requests/2.32.5" "1a494f86-1ba4-4a22-9dd6-65bf668015ea" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44134 10.133.0.33:443 100.64.0.3:5382 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:17:34.621671Z info ads ADS: new connection for node:6 2026-06-02T05:17:34.624575Z info cache read certificate from file resource=file-root:system 2026-06-02T05:17:43.464375Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:43.464432Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:43.478745Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:43.478883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:43.479111Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:43.482485Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:43.482556Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:17:43.482572Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-06-02T05:17:43.462Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 322 19 - "100.64.0.2" "python-requests/2.32.5" "ae19744b-01bc-4131-88d4-fe5eec8b9e8a" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44754 10.133.0.33:443 100.64.0.2:31848 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:17:43.842714Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:43.842760Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:43.843898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:43.843932Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:17:43.864391Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:43.864436Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:43.865424Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:43.865464Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:43.886415Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:43.886456Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:43.893153Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:43.893296Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:43.893569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:43.893609Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:43.894725Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:43.894760Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:43.895013Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:43.895821Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:17:43.895965Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:17:43.895981Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-06-02T05:17:43.841Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 2 - "100.64.0.4" "python-requests/2.32.5" "6b87dd05-652a-4c95-b121-5bf444d6404f" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.133.0.33:443 100.64.0.4:39938 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-06-02T05:17:43.863Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 2 - "100.64.0.3" "python-requests/2.32.5" "f35e9f0b-8b4d-4dfd-851e-7f5a96bab44f" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.133.0.33:443 100.64.0.3:2470 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-06-02T05:17:43.885Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 404 - - "-" 83 98 10 - "100.64.0.2" "python-requests/2.32.5" "82468c86-d153-4885-963d-ef1ea0a72ad2" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.133.0.33:443 100.64.0.2:31864 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com llm.e2e-external-model.0 2026-06-02T05:18:21.554325Z info ads ADS: "" 6 terminated 2026-06-02T05:18:23.391468Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:23.391521Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:23.394363Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:23.394480Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:23.394700Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:23.407377Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:23.407436Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:18:23.407447Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:23.511222Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:23.511263Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:23.537059Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:23.537111Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:23.577820Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:23.577863Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:23.579517Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:23.579548Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:23.603201Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:23.603246Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:23.604802Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:23.604839Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-06-02T05:18:23.390Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 17 - "100.64.0.4" "python-requests/2.32.5" "c749902f-fd03-4d24-8ba4-1198d1cd3ee7" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44134 10.133.0.33:443 100.64.0.4:54394 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:23.510Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 37 0 27 - "100.64.0.4" "python-requests/2.32.5" "96c6b99e-8ce2-4119-8aa1-2d894595162e" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.33:443 100.64.0.4:14618 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:23.576Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 35 0 3 - "100.64.0.3" "python-requests/2.32.5" "a802a9c6-706b-421f-a0ce-1f29efed7f7b" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.33:443 100.64.0.3:1960 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:23.601Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 36 0 3 - "100.64.0.2" "python-requests/2.32.5" "8d3d3ec2-93a1-4560-8af3-32f531449ddf" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.33:443 100.64.0.2:2070 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:18:24.018991Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.019041Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.021760Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.021880Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.022083Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.025506Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.025569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:18:24.025580Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.254814Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.254859Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.257806Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.257918Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.258116Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.261142Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.261196Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:18:24.261207Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.283405Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.283438Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.287904Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.287998Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.290930Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.290964Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.297408Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.297545Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.297647Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.299005Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.299057Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:18:24.299067Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.300031Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.300079Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:18:24.300090Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.320971Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.321003Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.327256Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.327389Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.327616Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.327644Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.328697Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.328721Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.328914Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.330464Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.330586Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.330787Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.331133Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:24.424409Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.424446Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.427107Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.427208Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.427368Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.430318Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.430366Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:18:24.430376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.449469Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.449502Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.452082Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.452181Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.455970Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:24.456014Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:18:24.456024Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-06-02T05:18:24.017Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 28 312 7 - "100.64.0.4" "python-requests/2.32.5" "22fbb3d2-fc22-4df9-8ee8-5c03d5872e84" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44754 10.133.0.33:443 100.64.0.4:14626 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:24.253Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 7 - "100.64.0.3" "python-requests/2.32.5" "54c6ebeb-781b-4081-827f-dc52acf6b764" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44134 10.133.0.33:443 100.64.0.3:1976 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:24.290Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.133.0.35" "Go-http-client/1.1" "59cf0339-8063-45a6-a312-534923244897" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.132.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:47000 10.133.0.33:443 10.133.0.35:59370 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-02T05:18:24.282Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 17 - "100.64.0.2" "python-requests/2.32.5" "c52a6e3c-d6a6-4ca8-9c4b-696554cee709" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44134 10.133.0.33:443 100.64.0.2:2086 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:24.320Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 123 382 10 - "100.64.0.4" "python-requests/2.32.5" "f673f03c-f446-47e5-b084-7e845f7f11a3" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.132.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:51812 10.133.0.33:443 100.64.0.4:14630 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-06-02T05:18:24.423Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 7 - "100.64.0.3" "python-requests/2.32.5" "57885a0e-567c-4fc5-9072-be01b1d19cc3" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44754 10.133.0.33:443 100.64.0.3:1982 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:24.448Z] "DELETE /v1/api-keys/f1f16b3e-a4eb-45e2-9172-23228ea81306 HTTP/1.1" 200 - via_upstream - "-" 0 251 7 - "100.64.0.2" "python-requests/2.32.5" "ce9bee64-e74d-496a-abd6-818b98cff8cf" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44754 10.133.0.33:443 100.64.0.2:2100 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:18:27.484889Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.484938Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.488764Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.488845Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.490293Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.490348Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:18:27.490358Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-06-02T05:18:27.483Z] "GET /v1/models HTTP/1.1" 500 - via_upstream - "-" 0 96 6 - "100.64.0.4" "python-requests/2.32.5" "1cecc5b5-9918-4d2e-a3fd-b98c28b687fa" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44134 10.133.0.33:443 100.64.0.4:14638 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:18:27.619833Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:27.619876Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:27.635679Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:27.635712Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:27.804879Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.804922Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.807969Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.808078Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.808276Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.811259Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.811313Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:18:27.811324Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.831899Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.831932Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.834622Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.834737Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.834896Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.837685Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.837745Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:18:27.837758Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.932874Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.932920Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.935931Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.936047Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.936248Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.939403Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.939458Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:18:27.939469Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.977303Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.977345Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.980339Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.980444Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.983858Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.983913Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:18:27.983924Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.005440Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.005478Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.008298Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.008395Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.010245Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.010292Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:18:28.010303Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.103061Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.103097Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.105791Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.105892Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.106054Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.108943Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.108995Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:18:28.109005Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.129022Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.129066Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.132702Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.132813Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.134947Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.134983Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.141266Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.141417Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.141516Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.142894Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.142945Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:18:28.142955Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.143835Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.143884Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:18:28.143896Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.239384Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.239425Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.242402Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.242499Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.242672Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.245577Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.245624Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:18:28.245634Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.268650Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.268709Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.272230Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.272338Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.273501Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.273548Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:18:28.273558Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.292088Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.292122Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.295192Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.295289Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.297042Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.297073Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.302866Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.303011Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.303105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.304323Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.304371Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:18:28.304382Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.305147Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.305196Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:18:28.305207Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.394539Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.394581Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.397300Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.397420Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.397603Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.400236Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.400295Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:18:28.400310Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.419973Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.420013Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.423314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.423419Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.425205Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.425238Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.431292Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.431437Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.431534Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.432849Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.432898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:18:28.432908Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.433644Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.433720Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:18:28.433737Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:28.450611Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.450644Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.453786Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.453876Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.455464Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.455495Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.461494Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.461624Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.461745Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.462857Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.462901Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:18:28.462911Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.463680Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.463727Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:18:28.463738Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.554961Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.554994Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.558316Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.558406Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.558549Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.561301Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-06-02T05:18:28.561346Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-06-02T05:18:28.561356Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-06-02T05:18:27.618Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 38 0 17 - "100.64.0.3" "python-requests/2.32.5" "47c839ec-1346-40c0-affd-b98470319674" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.33:443 100.64.0.3:1986 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:27.803Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 34 319 7 - "100.64.0.3" "python-requests/2.32.5" "2faa7ea4-ed17-4d7c-bf3e-a39e1b702fb7" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44134 10.133.0.33:443 100.64.0.3:27982 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:27.830Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 32 317 6 - "100.64.0.2" "python-requests/2.32.5" "1bc20c09-38ac-48d3-bad7-7189df8aa738" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44134 10.133.0.33:443 100.64.0.2:2116 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:27.931Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 31 316 8 - "100.64.0.4" "python-requests/2.32.5" "622be653-97fc-481c-be03-399d652281a4" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44134 10.133.0.33:443 100.64.0.4:14654 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:27.975Z] "DELETE /v1/api-keys/3d9f54c1-ecf1-4114-8c9e-288f1c4886c1 HTTP/1.1" 200 - via_upstream - "-" 0 253 7 - "100.64.0.3" "python-requests/2.32.5" "2904cbb3-9d0b-4fd0-9d3f-5f69fa994ca1" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44134 10.133.0.33:443 100.64.0.3:1992 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:28.004Z] "DELETE /v1/api-keys/3d9f54c1-ecf1-4114-8c9e-288f1c4886c1 HTTP/1.1" 404 - via_upstream - "-" 0 29 6 - "100.64.0.2" "python-requests/2.32.5" "a59c81d8-c8dd-4dc0-8a4a-9205e912410c" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44754 10.133.0.33:443 100.64.0.2:2128 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:28.101Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 7 - "100.64.0.4" "python-requests/2.32.5" "13dc56da-c3d9-4136-b58d-15530a026757" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44754 10.133.0.33:443 100.64.0.4:14656 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:28.133Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.133.0.35" "Go-http-client/1.1" "dc6aac0f-bcc9-46f7-9249-3c268c38c85a" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.132.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:47000 10.133.0.33:443 10.133.0.35:59370 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-02T05:18:28.127Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 16 - "100.64.0.3" "python-requests/2.32.5" "9bcc8143-bbc8-476a-b800-8509c48ce282" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44754 10.133.0.33:443 100.64.0.3:1998 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:28.238Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 7 - "100.64.0.2" "python-requests/2.32.5" "60b2034f-9e74-46b8-b714-3ab85d07e17b" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44754 10.133.0.33:443 100.64.0.2:2136 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:28.267Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 6 - "100.64.0.4" "python-requests/2.32.5" "b549a924-3e4d-42dd-85b5-5b9b3f3e918a" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44134 10.133.0.33:443 100.64.0.4:14662 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:28.296Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.133.0.35" "Go-http-client/1.1" "870b3f67-1d89-428b-bebd-2493badd170f" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.132.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:47000 10.133.0.33:443 10.133.0.35:59370 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-02T05:18:28.290Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 14 - "100.64.0.3" "python-requests/2.32.5" "3664c652-59b8-41e3-81c4-76b5601c8cb6" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44134 10.133.0.33:443 100.64.0.3:27300 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:28.393Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 7 - "100.64.0.2" "python-requests/2.32.5" "bd523c70-2f44-4818-9e9c-b4e35ba64633" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44134 10.133.0.33:443 100.64.0.2:47806 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:28.424Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.133.0.35" "Go-http-client/1.1" "0e8824e5-418b-46dd-9544-252a819b1550" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.132.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:47000 10.133.0.33:443 10.133.0.35:59370 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-02T05:18:28.418Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 14 - "100.64.0.4" "python-requests/2.32.5" "c48ab979-adcf-4a64-9290-96ea62c7d94b" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44134 10.133.0.33:443 100.64.0.4:2812 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:28.454Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.133.0.35" "Go-http-client/1.1" "cf40fd20-ae80-4ada-8f0c-1569c90789db" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.132.0.35:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.33:47000 10.133.0.33:443 10.133.0.35:59370 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-02T05:18:28.449Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 14 - "100.64.0.3" "python-requests/2.32.5" "a33d6077-9dc2-4d38-91a1-7870d70bb5cc" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44754 10.133.0.33:443 100.64.0.3:27308 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:28.553Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 36 320 7 - "100.64.0.2" "python-requests/2.32.5" "e3cab73c-01c2-4540-81c9-6a84a2184d6b" "maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com" "10.133.0.35:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.33:44754 10.133.0.33:443 100.64.0.2:47812 maas.apps.e5b2ed39-7874-4693-b0d0-d5916a6abfdd.prod.konfluxeaas.com opendatahub.maas-api-route.1