2026-05-25T01:33:43.664282Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:33:43.664350Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:33:43.664362Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:33:43.665766Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:33:43.665841Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:33:43.665856Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-25T01:33:43.654Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.134.0.59" "Go-http-client/1.1" "cdd070a5-9a98-41c4-b6bd-f5ec4512766f" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.62:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:58782 10.134.0.50:443 10.134.0.59:51930 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-25T01:33:43.631Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 34 - "100.64.0.4" "python-requests/2.32.5" "9c87ae70-2231-4001-a163-12949a020f93" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:56684 10.134.0.50:443 100.64.0.4:19904 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-25T01:33:44.006079Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:33:44.006135Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:33:44.011385Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:33:44.011532Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:33:44.011771Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:33:44.014994Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:33:44.015063Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-25T01:33:44.015075Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-25T01:33:44.004Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 33 318 10 - "100.64.0.3" "python-requests/2.32.5" "32380f7d-5ba9-4d9f-8679-34c1538bb160" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:55160 10.134.0.50:443 100.64.0.3:8680 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-25T01:33:52.056185Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:33:52.056267Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:33:52.059974Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:33:52.060139Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:33:52.062526Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:33:52.062571Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:33:52.068617Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:33:52.068775Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:33:52.068904Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:33:52.070074Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:33:52.070143Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:33:52.070159Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:33:52.071046Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:33:52.071100Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-25T01:33:52.071112Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-25T01:33:52.061Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.134.0.59" "Go-http-client/1.1" "83d8d712-28ff-4b1d-9e4e-6102b6e583f0" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.62:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:58782 10.134.0.50:443 10.134.0.59:51930 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-25T01:33:52.054Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 16 - "100.64.0.2" "python-requests/2.32.5" "28604b8c-9926-4d9b-be49-48af2eea3330" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:55160 10.134.0.50:443 100.64.0.2:64960 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-25T01:33:52.734659Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:33:52.734704Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:33:52.740171Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:33:52.740324Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:33:52.740525Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:33:52.743807Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:33:52.743896Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:33:52.743911Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-25T01:33:52.733Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 321 10 - "100.64.0.3" "python-requests/2.32.5" "91227588-c7a8-4cbf-8b68-95ec666f1124" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:56684 10.134.0.50:443 100.64.0.3:10618 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-25T01:34:00.772485Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:00.772538Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:00.776117Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:00.776291Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:00.778288Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:00.778330Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:00.778349Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:00.784369Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:00.784507Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:00.784621Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:00.784720Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:00.786034Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:00.786060Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:00.786112Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:34:00.786122Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:00.786135Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:34:00.786140Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:00.787131Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:00.787187Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:34:00.787198Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-25T01:34:00.777Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.134.0.59" "Go-http-client/1.1" "6e30aabc-5437-425e-99f4-760b48a22ad8" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.62:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:58782 10.134.0.50:443 10.134.0.59:51930 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-25T01:34:00.771Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 474 15 - "100.64.0.2" "python-requests/2.32.5" "4d09ef38-8a91-43d8-a7f6-9916f7f159a3" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:56684 10.134.0.50:443 100.64.0.2:5236 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-25T01:34:09.905390Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:09.905441Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:09.911092Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:09.911254Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:09.911475Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:09.914688Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:09.914748Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:34:09.914759Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-25T01:34:09.904Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 85 328 10 - "100.64.0.4" "python-requests/2.32.5" "4c33da1f-1d19-4f5d-a028-1e6ae44e9579" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:56684 10.134.0.50:443 100.64.0.4:42720 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-25T01:34:26.162676Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:26.162724Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:26.166102Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:26.166236Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:26.167666Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:26.167725Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:34:26.167735Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-25T01:34:26.161Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 82 6 - "100.64.0.4" "python-requests/2.32.5" "3ad99782-fca1-4be1-89ba-9bced5e2816b" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:56684 10.134.0.50:443 100.64.0.4:51994 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-25T01:34:43.166975Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:43.167026Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:43.172064Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:43.172209Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:43.173871Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:34:43.173943Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:34:43.173960Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-25T01:34:43.165Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 8 - "100.64.0.2" "python-requests/2.32.5" "387d55b9-28f5-4c56-828d-93258b3ebcba" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:56684 10.134.0.50:443 100.64.0.2:1924 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-25T01:35:00.306140Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:00.306190Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:00.310971Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:00.311093Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:00.312648Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:00.312704Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:35:00.312714Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-25T01:35:00.304Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 82 7 - "100.64.0.3" "python-requests/2.32.5" "ca2ab98f-084e-45d3-b434-9086734d4a14" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:56684 10.134.0.50:443 100.64.0.3:54438 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-25T01:35:17.628535Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:17.628586Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:17.634049Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:17.634179Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:17.637261Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:17.637326Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-25T01:35:17.637337Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-25T01:35:17.627Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 10 - "100.64.0.3" "python-requests/2.32.5" "3af99944-8f51-4e21-87f7-aec09647048f" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:34660 10.134.0.50:443 100.64.0.3:54732 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-25T01:35:29.545596Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:29.545654Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:29.552124Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:29.552294Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:29.552584Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:29.557453Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:29.557516Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:35:29.557527Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-25T01:35:29.543Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 45 328 13 - "100.64.0.4" "python-requests/2.32.5" "a1814a8e-6686-4589-a815-727ad4af9c4f" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:56684 10.134.0.50:443 100.64.0.4:31008 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-25T01:35:37.583874Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:37.583933Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:37.596254Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:37.596393Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:37.605254Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:37.605292Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:37.611249Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:37.611380Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:37.611489Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:37.612824Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:37.612884Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-25T01:35:37.612895Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:37.613752Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:37.613807Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-25T01:35:37.613818Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-25T01:35:37.604Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 8 - "10.134.0.59" "Go-http-client/1.1" "53378dc4-fc42-44ab-89e8-c2bad0abfc93" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.61:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:54526 10.134.0.50:443 10.134.0.59:60248 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.2 [2026-05-25T01:35:37.582Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 471 31 - "100.64.0.2" "python-requests/2.32.5" "c9261200-e66d-4080-9552-65fb0c72d7ea" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:34660 10.134.0.50:443 100.64.0.2:5702 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-25T01:35:49.585602Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:49.585652Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:49.590794Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:49.590932Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:49.591147Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:49.594583Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:49.594647Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:35:49.594659Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:49.617406Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:49.617453Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:49.622082Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:49.622239Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:49.622454Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:49.625602Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:49.625660Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-25T01:35:49.625671Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-25T01:35:49.584Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 297 10 - "100.64.0.2" "python-requests/2.32.5" "598ceea8-1221-4b69-8d5f-aa092d749e26" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:56684 10.134.0.50:443 100.64.0.2:53512 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-25T01:35:49.616Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 297 9 - "100.64.0.3" "python-requests/2.32.5" "666fd057-7660-4724-8381-25c6ef230a90" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:34660 10.134.0.50:443 100.64.0.3:1092 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-25T01:35:57.657428Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:57.657478Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:57.661304Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:57.661438Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:57.663848Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:57.663885Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:57.673095Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:57.673255Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:57.673403Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:57.674832Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:57.674905Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-25T01:35:57.674922Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:57.676013Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:57.676081Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:35:57.676097Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:57.695696Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:57.695750Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:57.699434Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:57.699573Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:57.701670Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:57.701715Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:57.709023Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:57.709151Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:57.709308Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:57.714015Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:57.714079Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-25T01:35:57.714090Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:35:57.715995Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:35:57.716064Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:35:57.716081Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-25T01:35:57.662Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 12 - "10.134.0.59" "Go-http-client/1.1" "73f6c13e-059e-4f87-8cb5-4c3a9385f24a" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.61:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:54526 10.134.0.50:443 10.134.0.59:60248 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.2 [2026-05-25T01:35:57.656Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 468 19 - "100.64.0.4" "python-requests/2.32.5" "04910484-20c2-44e0-b849-9af1b011dc1f" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:56684 10.134.0.50:443 100.64.0.4:33804 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-25T01:35:57.700Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 13 - "10.134.0.59" "Go-http-client/1.1" "e2cb5a90-377d-49d9-b297-66fa8fd8dd9a" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.133.0.29:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:56972 10.134.0.50:443 10.134.0.59:60248 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.2 [2026-05-25T01:35:57.694Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 345 21 - "100.64.0.2" "python-requests/2.32.5" "54f8dc3e-0860-4129-908a-1ad75753fe26" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:56684 10.134.0.50:443 100.64.0.2:53518 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-25T01:36:14.123318Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:14.123366Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:14.128962Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:14.129096Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:14.131287Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:14.131324Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:14.139930Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:14.140078Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:14.141399Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:14.141460Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-25T01:36:14.141472Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:14.142777Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:14.142811Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:14.151315Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:14.151455Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:14.152968Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:14.153026Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-25T01:36:14.153038Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:14.154289Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:14.154318Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:14.162350Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:14.162479Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:14.163642Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:14.163698Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-25T01:36:14.163710Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:14.164702Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:14.164783Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:36:14.164797Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-25T01:36:14.130Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 11 - "10.134.0.59" "Go-http-client/1.1" "aec5267a-8c87-40b7-a20b-9be332b3300c" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.61:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:54526 10.134.0.50:443 10.134.0.59:60248 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.2 [2026-05-25T01:36:14.141Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 11 - "10.134.0.59" "Go-http-client/1.1" "c1e80e95-0b00-405c-85ca-d355c2c5001a" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.133.0.29:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:56972 10.134.0.50:443 10.134.0.59:60248 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.2 [2026-05-25T01:36:14.153Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 10 - "10.134.0.59" "Go-http-client/1.1" "69506e59-0dd0-436d-a49c-32aa6c6476e9" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.62:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:39138 10.134.0.50:443 10.134.0.59:60248 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-25T01:36:14.122Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 1361 42 - "100.64.0.3" "python-requests/2.32.5" "b590079a-5fe8-4210-af98-85e3428ee94c" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:56684 10.134.0.50:443 100.64.0.3:34486 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-25T01:36:21.305Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 2 - "-" "-" "-" "-" "-" - - 10.134.0.50:443 100.64.0.4:16812 - - 2026-05-25T01:36:30.562207Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:30.562292Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:30.567735Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:30.567871Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:30.570119Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:30.570160Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:30.579578Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:30.579736Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:30.581295Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:30.581369Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-25T01:36:30.581383Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:30.582539Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:30.582616Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:36:30.582634Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:30.601990Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:30.602045Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:30.607551Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:30.607708Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:30.609815Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:30.609858Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:30.618673Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:30.618818Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:30.620393Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:30.620453Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-25T01:36:30.620465Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:30.621441Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:30.621510Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-25T01:36:30.621526Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-25T01:36:30.569Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 12 - "10.134.0.59" "Go-http-client/1.1" "c4e09c1a-2bda-4a48-ae99-4e874b0c5cb5" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.61:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:54526 10.134.0.50:443 10.134.0.59:60248 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.2 [2026-05-25T01:36:30.560Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 470 22 - "100.64.0.2" "python-requests/2.32.5" "a256bd90-ed0a-4e8d-8c3b-24460a3f5225" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:56684 10.134.0.50:443 100.64.0.2:24644 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-25T01:36:30.608Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 11 - "10.134.0.59" "Go-http-client/1.1" "37546e20-d872-96b3-b39d-250f4b0c3ac1" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.133.0.29:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:56972 10.134.0.50:443 10.134.0.59:60248 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.2 [2026-05-25T01:36:30.600Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 347 20 - "100.64.0.3" "python-requests/2.32.5" "e9fcf683-a92b-44c4-a409-fdbcf280b707" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:34660 10.134.0.50:443 100.64.0.3:23302 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-25T01:36:39.644271Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:39.644328Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:39.645629Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:39.645664Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-25T01:36:39.643Z] "GET /maas-api/v1/models HTTP/1.1" 401 - - "-" 0 0 2 - "100.64.0.4" "python-requests/2.32.5" "689b0c28-b056-44c6-a9b1-f4aae3d7f7e3" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.50:443 100.64.0.4:64778 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-25T01:36:56.233832Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.233883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.239944Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.240070Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.240297Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.244649Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.244747Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:36:56.244764Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.261789Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.261832Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.269442Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.269581Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.269884Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.269920Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.271215Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.271263Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.271509Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.273945Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.274146Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.274459Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.275051Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.292636Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:56.292681Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:56.298304Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:56.298433Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:56.298692Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:56.298735Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:56.299233Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:56.299265Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:56.315343Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.315381Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.320997Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.321118Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.321413Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.321453Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.321866Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.321890Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.337445Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.337480Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.340250Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.340358Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.343690Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:56.343727Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:56.349932Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:56.350050Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:56.350158Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:56.351649Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:56.351708Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-25T01:36:56.351730Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:36:56.352986Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:36:56.353041Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:36:56.353054Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-25T01:36:56.232Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 330 12 - "100.64.0.2" "python-requests/2.32.5" "c7524558-7ebb-4e4f-876f-ae3def998155" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:56684 10.134.0.50:443 100.64.0.2:50340 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-25T01:36:56.261Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 13 - "100.64.0.3" "python-requests/2.32.5" "d7d65186-7321-49dc-a47b-4a7df0111038" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.132.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:38878 10.134.0.50:443 100.64.0.3:24316 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-05-25T01:36:56.292Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 7 - "100.64.0.4" "python-requests/2.32.5" "4f034406-4b34-49a6-9763-6777dc62a6c6" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.132.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.50:443 100.64.0.4:18932 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-05-25T01:36:56.314Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 7 - "100.64.0.2" "python-requests/2.32.5" "68693522-649d-4abf-b675-b018d698228f" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.132.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:38878 10.134.0.50:443 100.64.0.2:50342 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-05-25T01:36:56.343Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.134.0.59" "Go-http-client/1.1" "e158dee6-e7bc-4a78-91d5-1b3287664ee1" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.132.0.27:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:53636 10.134.0.50:443 10.134.0.59:60248 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.2 [2026-05-25T01:36:56.336Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 547 16 - "100.64.0.3" "python-requests/2.32.5" "65f01fd0-2eb4-42e3-b1f3-5f325a9aaa17" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:56684 10.134.0.50:443 100.64.0.3:24326 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-25T01:37:05.191879Z info ads ADS: new connection for node:6 2026-05-25T01:37:05.195812Z info cache read certificate from file resource=file-root:system 2026-05-25T01:37:11.939271Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:11.939320Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:11.944763Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:11.944899Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:11.945107Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:11.948036Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:11.948095Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:37:11.948105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:12.289704Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:12.289751Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:12.290816Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:12.290851Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:12.312983Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:12.313021Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:12.313871Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:12.313901Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:12.352299Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:12.352339Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:12.358619Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:12.358739Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:12.359025Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:12.359058Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:12.360216Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:12.360274Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:12.360510Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:12.361396Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:12.361530Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:37:12.361542Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-25T01:37:11.938Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 322 10 - "100.64.0.4" "python-requests/2.32.5" "8f97de90-0999-4731-a4c2-7a4754540b5d" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:56684 10.134.0.50:443 100.64.0.4:60556 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-25T01:37:12.288Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 2 - "100.64.0.4" "python-requests/2.32.5" "f0d628ab-a1e7-4321-96d8-7582f88cff06" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.134.0.50:443 100.64.0.4:2706 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-05-25T01:37:12.312Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 1 - "100.64.0.2" "python-requests/2.32.5" "33831780-1757-42a5-a7bb-e52796c2b1c3" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.134.0.50:443 100.64.0.2:4800 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-05-25T01:37:12.351Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 404 - - "-" 83 98 10 - "100.64.0.3" "python-requests/2.32.5" "136bb554-edcd-4f89-8b78-61a136f08f1a" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.134.0.50:443 100.64.0.3:36726 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.e2e-external-model.0 2026-05-25T01:37:49.980468Z info ads ADS: "" 6 terminated 2026-05-25T01:37:53.833105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:53.833165Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:53.839015Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:53.839149Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:53.839392Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:53.852430Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:53.852519Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:37:53.852537Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:53.891304Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:53.891347Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:53.895709Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:53.895820Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:53.898091Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:53.898133Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-25T01:37:53.831Z] "POST /v1/api-keys HTTP/2" 201 - via_upstream - "-" 55 319 20 - "100.64.0.3" "curl/7.76.1" "7e52274e-3e65-4b68-a6cf-8a64448827c9" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:56684 10.134.0.50:443 100.64.0.3:31526 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-25T01:37:53.903633Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:53.903771Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:53.903874Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:53.905069Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:53.905122Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-25T01:37:53.905133Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:53.906181Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:53.906263Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-25T01:37:53.906276Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:53.953424Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:53.953465Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:53.959303Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:53.959435Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:53.959678Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:53.959708Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:53.961306Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:53.961340Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:53.961613Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:53.963016Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:53.963215Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:53.963493Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:53.963928Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:53.997998Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:53.998031Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.003729Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.003861Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.004086Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.004115Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.004551Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.004574Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.004760Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.006062Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.006194Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.006393Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.006742Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.030048Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.030080Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.035379Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.035505Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.035724Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.035754Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.036208Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.036255Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.036444Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.037707Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.037844Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.038013Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.038357Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.063779Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.063813Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.069108Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.069253Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.069497Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.069527Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.070032Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.070055Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.094063Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.094098Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.099619Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.099747Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.099989Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.100019Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.100590Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.100613Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.123978Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.124009Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.129456Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.129584Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.129806Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.129834Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.130318Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.130340Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.152943Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.152977Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.158443Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.158581Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.158813Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.158843Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.159365Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.159388Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.183066Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.183099Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.188587Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.188733Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.188985Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.189026Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.189467Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.189498Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.214387Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.214419Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.219518Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.219645Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.219871Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.219900Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.220366Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.220389Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.241108Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.241140Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.247922Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.248047Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.248311Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.248343Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.248789Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.248811Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-25T01:37:54.275393Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.275432Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.280756Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.280891Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.281119Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.281155Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.281635Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.281657Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.304732Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.304764Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.305533Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.305561Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.326833Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.326865Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.331551Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.331645Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.336368Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-25T01:37:54.336427Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-25T01:37:54.336439Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-25T01:37:53.897Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 7 - "10.134.0.59" "Go-http-client/1.1" "0bad2164-6f46-4477-9715-b434fa4c4c85" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.62:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:39138 10.134.0.50:443 10.134.0.59:60248 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-25T01:37:53.890Z] "GET /v1/models HTTP/2" 200 - via_upstream - "-" 0 590 15 - "100.64.0.4" "curl/7.76.1" "e8276a62-78aa-4764-b7e4-61cf6b3b3289" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:56684 10.134.0.50:443 100.64.0.4:9734 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-25T01:37:53.952Z] "POST /v1/chat/completions HTTP/2" 200 - via_upstream - "-" 100 357 11 - "100.64.0.2" "curl/7.76.1" "792081ca-6498-46be-aeea-d5896ecffb51" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.62:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:39138 10.134.0.50:443 100.64.0.2:46138 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-25T01:37:53.997Z] "POST /v1/chat/completions HTTP/2" 200 - via_upstream - "-" 100 511 9 - "100.64.0.3" "curl/7.76.1" "952908d4-deb8-4b31-8d5e-6864a459df0c" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.62:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:39138 10.134.0.50:443 100.64.0.3:33466 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-25T01:37:54.029Z] "POST /v1/chat/completions HTTP/2" 200 - via_upstream - "-" 100 432 8 - "100.64.0.2" "curl/7.76.1" "cf8b7e5d-e84c-4d57-b942-ae2fb5f1802d" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.62:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:39138 10.134.0.50:443 100.64.0.2:15846 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-25T01:37:54.063Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 6 - "100.64.0.4" "curl/7.76.1" "5feac4a2-82c0-4bc9-8387-2d9a12b1dbdc" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.62:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:39138 10.134.0.50:443 100.64.0.4:14518 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-25T01:37:54.093Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 7 - "100.64.0.3" "curl/7.76.1" "c5c09d1d-fc22-4dd3-b723-9d6e2a3bb131" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.62:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:58782 10.134.0.50:443 100.64.0.3:31536 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-25T01:37:54.123Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 7 - "100.64.0.2" "curl/7.76.1" "8ccecda2-7344-4930-a1f6-78aeee7ecac6" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.62:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.50:443 100.64.0.2:15856 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-25T01:37:54.152Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 7 - "100.64.0.4" "curl/7.76.1" "19aab071-0618-4449-b39d-1b177ef494a0" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.62:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.50:443 100.64.0.4:9748 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-25T01:37:54.182Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 7 - "100.64.0.2" "curl/7.76.1" "40f2cc45-b27a-4a90-a402-eca20c7bfad0" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.62:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:59350 10.134.0.50:443 100.64.0.2:46148 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-25T01:37:54.213Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 6 - "100.64.0.3" "curl/7.76.1" "2b3bd99e-7086-48ce-bd7b-61f7b9c70a41" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.62:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.50:59360 10.134.0.50:443 100.64.0.3:33480 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-25T01:37:54.240Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 8 - "100.64.0.4" "curl/7.76.1" "aac5cf01-3055-4289-bed4-920edda654b7" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.62:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.50:443 100.64.0.4:9750 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-25T01:37:54.274Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 7 - "100.64.0.2" "curl/7.76.1" "b7ac92bc-09b2-4696-a020-92ee1e7a595a" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.62:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.50:443 100.64.0.2:46154 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-25T01:37:54.304Z] "POST /llm/facebook-opt-125m-simulated/v1/chat/completions HTTP/2" 401 - - "-" 100 0 1 - "100.64.0.3" "curl/7.76.1" "e4de9bf9-d917-4cf9-907a-ce64459e707a" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.134.0.50:443 100.64.0.3:33484 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-25T01:37:54.325Z] "DELETE /v1/api-keys/361a9c8e-38c0-413a-a4d3-d5fe1e67a151 HTTP/2" 200 - via_upstream - "-" 0 334 10 - "100.64.0.4" "curl/7.76.1" "ef3cbc20-be4e-4711-9b72-cafe7c9bc2ca" "maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com" "10.134.0.59:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.50:50360 10.134.0.50:443 100.64.0.4:9758 maas.apps.d312857a-e4f3-4d86-a56e-2339ee879a7e.prod.konfluxeaas.com opendatahub.maas-api-route.1