[2026-06-02T05:17:48.768Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 16 - "100.64.0.2" "python-requests/2.32.5" "20b14f59-e856-4753-a881-53798e928d1f" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:41388 10.134.0.33:443 100.64.0.2:60192 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:18:04.995336Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:04.995389Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:05.010785Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:05.010950Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:05.011175Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:05.015151Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:05.015212Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-02T05:18:05.015223Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-06-02T05:18:04.994Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 45 329 21 - "100.64.0.4" "python-requests/2.32.5" "606178a6-b88e-4ea0-a7d2-0acf3630a586" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:37878 10.134.0.33:443 100.64.0.4:39598 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:18:13.067011Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:13.067071Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:13.080238Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:13.080425Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:13.092576Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:13.092614Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:13.099164Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:13.099300Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:13.099416Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:13.100992Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:13.101049Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:18:13.101059Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:13.102302Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:13.102357Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-02T05:18:13.102368Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-06-02T05:18:13.091Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 9 - "10.133.0.27" "Go-http-client/1.1" "7dcaac56-d388-4164-abf5-bbb2bac7a515" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.134.0.43:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:47282 10.134.0.33:443 10.133.0.27:58618 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-02T05:18:13.064Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 471 37 - "100.64.0.3" "python-requests/2.32.5" "7085824a-5924-41d5-a799-8b740e947476" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:37878 10.134.0.33:443 100.64.0.3:43728 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:18:27.242311Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.242362Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.256964Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.257096Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.257320Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.261587Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.261646Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:18:27.261656Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:27.287287Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:27.287325Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:27.296630Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:27.296755Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:27.296955Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:27.300242Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:27.300295Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-02T05:18:27.300307Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-06-02T05:18:27.241Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 297 20 - "100.64.0.3" "python-requests/2.32.5" "f40b308a-4062-4714-ba96-49c19710ce5c" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:41388 10.134.0.33:443 100.64.0.3:64990 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:27.286Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 297 14 - "100.64.0.2" "python-requests/2.32.5" "a913c498-69b6-4568-9b50-4d6a626b4878" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:37878 10.134.0.33:443 100.64.0.2:20988 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:18:35.335045Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:35.335092Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:35.338899Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:35.339020Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:35.341438Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:35.341470Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:35.347431Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:35.347552Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:35.347646Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:35.349110Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:35.349165Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:18:35.349175Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:35.350229Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:35.350278Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:18:35.350288Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:35.375476Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:35.375516Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:35.378550Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:35.378658Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:35.380929Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:35.380961Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:35.386683Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:35.386796Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:35.386922Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:35.388460Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:35.388509Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:18:35.388519Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:35.389655Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:35.389708Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-02T05:18:35.389719Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-06-02T05:18:35.340Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 8 - "10.133.0.27" "Go-http-client/1.1" "822a5640-7f90-45ae-a8b2-d002e8a9d2e8" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.134.0.43:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:47282 10.134.0.33:443 10.133.0.27:58618 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-02T05:18:35.333Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 468 16 - "100.64.0.2" "python-requests/2.32.5" "b3f38c64-362d-46a5-b6f7-ea07d713404d" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:41388 10.134.0.33:443 100.64.0.2:7366 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:18:35.380Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 8 - "10.133.0.27" "Go-http-client/1.1" "3d3f09d6-4836-48b5-bebb-1b8db6f6d48e" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.132.0.49:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:49536 10.134.0.33:443 10.133.0.27:58618 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-02T05:18:35.374Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 345 15 - "100.64.0.4" "python-requests/2.32.5" "7b7ee5ec-75b0-44ef-a31e-c601f8e6bce7" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:37878 10.134.0.33:443 100.64.0.4:17036 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:18:54.030600Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:54.030646Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:54.045920Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:54.046038Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:54.048752Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:54.048787Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:54.058059Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:54.058185Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:54.059637Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:54.059692Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:18:54.059703Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:54.061215Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:54.061246Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:54.071042Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:54.071148Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:54.072545Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:54.072591Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:18:54.072602Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:54.073959Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:54.073989Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:54.082919Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:54.083035Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:54.084335Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:54.084386Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:18:54.084396Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:18:54.085460Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:18:54.085527Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-02T05:18:54.085540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-06-02T05:18:54.047Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 12 - "10.133.0.27" "Go-http-client/1.1" "566971c9-79cf-40a1-8cbd-62137a66ebdd" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.134.0.43:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:47282 10.134.0.33:443 10.133.0.27:58618 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-02T05:18:54.060Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 12 - "10.133.0.27" "Go-http-client/1.1" "a826d6a4-2c22-4dcd-b088-028e01dc0a7b" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.132.0.49:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:49536 10.134.0.33:443 10.133.0.27:58618 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-02T05:18:54.073Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 11 - "10.133.0.27" "Go-http-client/1.1" "ecc13db6-5580-4392-b918-898c309aec86" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.134.0.44:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:40876 10.134.0.33:443 10.133.0.27:58618 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-02T05:18:54.029Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 1361 56 - "100.64.0.3" "python-requests/2.32.5" "8909bf6c-1c13-439e-bc31-82531885eb27" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:37878 10.134.0.33:443 100.64.0.3:34506 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:19:12.670710Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.670761Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.685729Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.685879Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.688369Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.688401Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.699013Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.699140Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.700553Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.700623Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:19:12.700640Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.701797Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.701871Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:19:12.701883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.723714Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.723751Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.733424Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.733523Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.735683Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.735715Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.744273Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.744380Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.745961Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.746008Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:19:12.746018Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.746974Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:12.747019Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:19:12.747029Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-06-02T05:19:12.687Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 13 - "10.133.0.27" "Go-http-client/1.1" "a336aa2e-f505-402a-bf58-b30593570ab2" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.134.0.43:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:47282 10.134.0.33:443 10.133.0.27:58618 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-02T05:19:12.669Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 470 32 - "100.64.0.2" "python-requests/2.32.5" "ef5f3775-4e99-4898-a2f1-05e67133039b" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:41388 10.134.0.33:443 100.64.0.2:19156 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:19:12.734Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 11 - "10.133.0.27" "Go-http-client/1.1" "be3dd4ab-73a2-4bd1-8ae5-90271c1e0c33" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.132.0.49:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:49536 10.134.0.33:443 10.133.0.27:58618 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-02T05:19:12.722Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 347 24 - "100.64.0.4" "python-requests/2.32.5" "b6e1036b-744b-4db1-8c5e-4020d3c4bd96" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:41388 10.134.0.33:443 100.64.0.4:52078 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:19:21.807881Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:21.807927Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:21.809276Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:21.809309Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-06-02T05:19:21.806Z] "GET /maas-api/v1/models HTTP/1.1" 401 - - "-" 0 0 2 - "100.64.0.3" "python-requests/2.32.5" "b7413e16-071b-47e1-887d-094dabddd9e8" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.33:443 100.64.0.3:34910 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:19:38.442955Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.443012Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.458164Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.458305Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.458526Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.464152Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.464223Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:19:38.464239Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.484883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.484916Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.492045Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.492162Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.492405Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.492437Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.493660Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.493686Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.493929Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.495659Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.495784Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.495998Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.496550Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.521863Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.521899Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.528521Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.528627Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.528877Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.528907Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.529362Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.529398Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.552800Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.552859Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.559263Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.559391Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.559639Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.559679Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.560209Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.560240Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.581266Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.581299Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.584175Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.584269Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.586771Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.586824Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.592670Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.592796Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.592917Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.595892Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.595955Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:19:38.595970Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:19:38.597176Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:38.597225Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-02T05:19:38.597236Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-06-02T05:19:38.441Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 330 22 - "100.64.0.2" "python-requests/2.32.5" "fe0be4c3-1dca-4a17-98eb-1a32080b8659" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:41388 10.134.0.33:443 100.64.0.2:18702 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:19:38.484Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 12 - "100.64.0.4" "python-requests/2.32.5" "905cf903-f817-46ad-8dbd-27e7c6f58bb1" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.132.0.51:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:59526 10.134.0.33:443 100.64.0.4:26978 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-02T05:19:38.521Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 8 - "100.64.0.4" "python-requests/2.32.5" "ae6e1c39-e73a-468e-b87e-5b791e99033f" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.132.0.51:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:59526 10.134.0.33:443 100.64.0.4:2368 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-02T05:19:38.552Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 7 - "100.64.0.3" "python-requests/2.32.5" "7544cab0-bf0c-4d31-b3fe-4bd188a19f3a" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.132.0.51:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:33148 10.134.0.33:443 100.64.0.3:3732 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-02T05:19:38.586Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.133.0.27" "Go-http-client/1.1" "528a680a-a125-4a05-9e2a-361d02e9b592" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.132.0.51:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:49034 10.134.0.33:443 10.133.0.27:58618 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.3 [2026-06-02T05:19:38.580Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 547 17 - "100.64.0.2" "python-requests/2.32.5" "a2d9f739-c6bd-4108-964f-e52ea7e3e881" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:37878 10.134.0.33:443 100.64.0.2:18708 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:19:47.458558Z info ads ADS: new connection for node:6 2026-06-02T05:19:47.462098Z info cache read certificate from file resource=file-root:system 2026-06-02T05:19:56.353012Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.353058Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.367471Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.367596Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.367790Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.371143Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.371198Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-02T05:19:56.371208Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-06-02T05:19:56.351Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 322 19 - "100.64.0.3" "python-requests/2.32.5" "93ae1695-201e-41ec-ad68-d5b561293db1" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:37878 10.134.0.33:443 100.64.0.3:38188 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:19:56.737978Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.738014Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.738849Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.738883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.760177Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.760208Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.761063Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.761092Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.782226Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.782258Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.789156Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.789273Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.789508Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.789536Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.791124Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.791148Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.791359Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.792464Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:19:56.792576Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-02T05:19:56.792587Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-06-02T05:19:56.736Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 2 - "100.64.0.4" "python-requests/2.32.5" "a7a36946-7abd-44ee-b5ef-ff930a7a7d27" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.134.0.33:443 100.64.0.4:64386 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-06-02T05:19:56.759Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 1 - "100.64.0.3" "python-requests/2.32.5" "63932a6d-3363-4c1c-aff6-15c5075dbdf6" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.134.0.33:443 100.64.0.3:54520 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-06-02T05:19:56.781Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 404 - - "-" 83 98 11 - "100.64.0.2" "python-requests/2.32.5" "e5800e1a-b3f9-4a72-965d-cb13f0d852ee" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.134.0.33:443 100.64.0.2:61994 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com llm.e2e-external-model.0 2026-06-02T05:20:34.500949Z info ads ADS: "" 6 terminated 2026-06-02T05:20:36.378774Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:36.378845Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:36.381483Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:36.381607Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:36.381805Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:36.396022Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:36.396081Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-02T05:20:36.396091Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:36.495933Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:36.495974Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:36.522754Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:36.522794Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:36.547757Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:36.547795Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:36.549063Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:36.549096Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:36.575582Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:36.575619Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:36.576746Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:36.576775Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-06-02T05:20:36.377Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 18 - "100.64.0.2" "python-requests/2.32.5" "e7f23c58-3902-43da-86a0-6e5a8bc637b0" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:37878 10.134.0.33:443 100.64.0.2:3440 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:20:36.494Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 37 0 28 - "100.64.0.4" "python-requests/2.32.5" "d11fa356-e7ed-472e-ae6c-1b0240cd6405" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.33:443 100.64.0.4:18772 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:20:36.546Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 35 0 2 - "100.64.0.4" "python-requests/2.32.5" "4e795a78-d4f2-4ff7-acab-6d374653d678" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.33:443 100.64.0.4:49778 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:20:36.574Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 36 0 2 - "100.64.0.3" "python-requests/2.32.5" "f22f9a8c-c9b1-41c1-b6d6-e5b72485e903" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.33:443 100.64.0.3:47858 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:20:36.970054Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:36.970103Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:36.972778Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:36.972913Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:36.973113Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:36.976932Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:36.976988Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:20:36.976999Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:37.200661Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.200710Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.203397Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.203527Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.203742Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.207379Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.207435Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-02T05:20:37.207445Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.230302Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.230338Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.234235Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.234337Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.237233Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:37.237268Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:37.243376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:37.243512Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:37.243614Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:37.245683Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:37.245734Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:20:37.245744Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:37.247119Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.247168Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-02T05:20:37.247179Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.270376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:37.270408Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:37.276388Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:37.276517Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:37.276762Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:37.276791Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:37.278165Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:37.278193Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:37.278388Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:37.280033Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:37.280166Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:37.280346Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:37.280804Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:37.367725Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.367761Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.370036Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.370140Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.370301Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.373581Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.373628Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-02T05:20:37.373638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.394781Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.394842Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.396965Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.397055Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.401261Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:37.401306Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-02T05:20:37.401316Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-06-02T05:20:36.968Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 28 312 8 - "100.64.0.2" "python-requests/2.32.5" "8e088e10-b310-4b53-bc3e-f947457bb289" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:41388 10.134.0.33:443 100.64.0.2:16842 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:20:37.199Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 8 - "100.64.0.4" "python-requests/2.32.5" "19711d3a-5da8-4d0c-8725-819f69e0865e" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:37878 10.134.0.33:443 100.64.0.4:18776 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:20:37.236Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.133.0.27" "Go-http-client/1.1" "d3f6d092-2b45-473f-b6e0-213fae282a17" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.134.0.44:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:40876 10.134.0.33:443 10.133.0.27:58618 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-02T05:20:37.229Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 18 - "100.64.0.3" "python-requests/2.32.5" "4ca8a6da-1e62-465c-943a-20fcf0d92378" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:37878 10.134.0.33:443 100.64.0.3:54156 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:20:37.269Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 123 369 11 - "100.64.0.3" "python-requests/2.32.5" "531313e0-cc6c-4bcf-be88-d5c9b2e6512d" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.134.0.44:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:40876 10.134.0.33:443 100.64.0.3:47866 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-06-02T05:20:37.366Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 7 - "100.64.0.2" "python-requests/2.32.5" "299b9fad-3a0a-497a-8810-b01ee5f17422" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:37878 10.134.0.33:443 100.64.0.2:3444 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:20:37.393Z] "DELETE /v1/api-keys/0fe92ad4-6f24-4d3b-894a-9eef2fba34f1 HTTP/1.1" 200 - via_upstream - "-" 0 251 7 - "100.64.0.4" "python-requests/2.32.5" "90ee0691-b794-40cd-af7c-0e2471b474df" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:37878 10.134.0.33:443 100.64.0.4:49788 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:20:40.429412Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.429460Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.433323Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.433406Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.435384Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.435441Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-02T05:20:40.435451Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.567660Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.567695Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.583837Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.583868Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-06-02T05:20:40.428Z] "GET /v1/models HTTP/1.1" 500 - via_upstream - "-" 0 96 7 - "100.64.0.2" "python-requests/2.32.5" "7ad5af78-2bd6-4a77-9543-45f0f9656c24" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:37878 10.134.0.33:443 100.64.0.2:22590 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:20:40.566Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 38 0 17 - "100.64.0.4" "python-requests/2.32.5" "81f75ccb-8bdc-4c05-81d3-249b0a405c45" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.134.0.33:443 100.64.0.4:21434 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-02T05:20:40.747640Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.747684Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.750268Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.750380Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.750572Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.754266Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.754320Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-02T05:20:40.754330Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.773921Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:40.773964Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:40.776391Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:40.776509Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:40.776711Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:40.779789Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:40.779874Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:20:40.779887Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:40.861673Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.861706Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.863795Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.863902Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.864052Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.867317Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.867362Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-02T05:20:40.867372Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:40.887692Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:40.887731Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:40.890054Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:40.890169Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:40.893805Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:40.893874Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:20:40.893890Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:40.916141Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:40.916178Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:40.918343Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:40.918454Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:40.920559Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:40.920611Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:20:40.920625Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.006912Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.006949Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.009083Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.009185Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.009346Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.013572Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.013618Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:20:41.013628Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.033978Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.034009Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.037139Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.037234Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.039218Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.039249Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.045391Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.045522Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.045614Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.046981Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.047026Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:20:41.047036Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.048090Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.048137Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:20:41.048147Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.137894Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.137927Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.140142Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.140248Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.140403Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.143792Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.143851Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-02T05:20:41.143862Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.165128Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.165161Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.168178Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.168273Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.169861Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.169905Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:20:41.169915Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.190127Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.190161Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.193106Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.193202Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.195410Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.195439Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.201207Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.201334Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.201424Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.202770Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.202834Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:20:41.202845Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.203930Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.203977Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-02T05:20:41.203987Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.292582Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.292615Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.294711Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.294805Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.294971Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.297979Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.298023Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:20:41.298033Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.319952Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.319985Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.322866Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.322963Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.324954Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.324983Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.330346Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.330478Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.330566Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.331911Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.331954Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:20:41.331965Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.333012Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.333062Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-02T05:20:41.333073Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.352095Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.352127Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.354806Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.354916Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.356788Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.356839Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.362284Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.362409Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.362496Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.363648Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.363692Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-02T05:20:41.363703Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-02T05:20:41.364753Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.364802Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-02T05:20:41.364837Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.450895Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.450925Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.453237Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.453330Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.453477Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.456555Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-02T05:20:41.456600Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-02T05:20:41.456610Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-06-02T05:20:40.746Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 34 318 7 - "100.64.0.3" "python-requests/2.32.5" "740afade-393d-4c9a-8486-646afa551517" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:37878 10.134.0.33:443 100.64.0.3:54162 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:20:40.772Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 32 317 7 - "100.64.0.2" "python-requests/2.32.5" "4c94fa13-9a96-472c-acfd-902b7f37152e" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:41388 10.134.0.33:443 100.64.0.2:3450 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:20:40.860Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 31 315 6 - "100.64.0.3" "python-requests/2.32.5" "af2fcddd-827c-490a-88ca-23ccbfff8b28" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:37878 10.134.0.33:443 100.64.0.3:3268 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:20:40.886Z] "DELETE /v1/api-keys/a4ec6ac2-7d17-4314-9002-f899b5248ab8 HTTP/1.1" 200 - via_upstream - "-" 0 253 7 - "100.64.0.2" "python-requests/2.32.5" "3d21802b-4468-4f37-b895-a7892eadaf79" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:41388 10.134.0.33:443 100.64.0.2:22604 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:20:40.915Z] "DELETE /v1/api-keys/a4ec6ac2-7d17-4314-9002-f899b5248ab8 HTTP/1.1" 404 - via_upstream - "-" 0 29 5 - "100.64.0.4" "python-requests/2.32.5" "75ddc071-cc09-4606-ae81-8cc3a4752ffd" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:41388 10.134.0.33:443 100.64.0.4:49802 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:20:41.005Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 7 - "100.64.0.3" "python-requests/2.32.5" "6059b2e3-f93c-40de-9b01-e718a4e9f822" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:41388 10.134.0.33:443 100.64.0.3:54176 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:20:41.038Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.133.0.27" "Go-http-client/1.1" "29b80106-8959-426f-8091-892d9c008e64" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.134.0.44:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:40876 10.134.0.33:443 10.133.0.27:58618 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-02T05:20:41.032Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 15 - "100.64.0.2" "python-requests/2.32.5" "29782d3a-0808-46b5-8459-f2b5452bcab6" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:41388 10.134.0.33:443 100.64.0.2:3456 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:20:41.136Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 7 - "100.64.0.4" "python-requests/2.32.5" "be0e0248-ac82-4232-9902-ce69e37e4e56" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:37878 10.134.0.33:443 100.64.0.4:49814 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:20:41.164Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 5 - "100.64.0.3" "python-requests/2.32.5" "731ccf99-6cd7-496a-a7cf-9492d8008ac8" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:41388 10.134.0.33:443 100.64.0.3:54190 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:20:41.194Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.133.0.27" "Go-http-client/1.1" "69d37eca-285e-49b4-9473-84aab312b794" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.134.0.44:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:40876 10.134.0.33:443 10.133.0.27:58618 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-02T05:20:41.189Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 14 - "100.64.0.2" "python-requests/2.32.5" "a1c1c928-68d5-45c4-8e1c-88353ec9a917" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:37878 10.134.0.33:443 100.64.0.2:3472 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:20:41.291Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 6 - "100.64.0.4" "python-requests/2.32.5" "1b88dd88-1bfd-416a-915b-a7ebb5620302" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:41388 10.134.0.33:443 100.64.0.4:21444 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:20:41.324Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 7 - "10.133.0.27" "Go-http-client/1.1" "cbf07273-9c62-4915-8034-b794c6362505" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.134.0.44:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:40876 10.134.0.33:443 10.133.0.27:58618 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-02T05:20:41.318Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 14 - "100.64.0.3" "python-requests/2.32.5" "12369499-34d1-4165-bda8-1cb835126d0c" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:37878 10.134.0.33:443 100.64.0.3:3272 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:20:41.355Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 7 - "10.133.0.27" "Go-http-client/1.1" "0a281a6a-f939-436e-8825-ea069c2c64e4" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.134.0.44:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.134.0.33:40876 10.134.0.33:443 10.133.0.27:58618 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-02T05:20:41.351Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 13 - "100.64.0.2" "python-requests/2.32.5" "3ed24a8d-15bb-4c40-a4ad-5780b1d89027" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:37878 10.134.0.33:443 100.64.0.2:22616 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-02T05:20:41.449Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 36 321 6 - "100.64.0.4" "python-requests/2.32.5" "72d39c8a-c57f-43b8-b6b9-3fc00507e819" "maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com" "10.133.0.27:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.134.0.33:37878 10.134.0.33:443 100.64.0.4:49820 maas.apps.f90e8664-c6aa-4e3e-99fd-ee3d84f4b884.prod.konfluxeaas.com opendatahub.maas-api-route.1