2026-06-06T22:38:45.195980Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:38:45.196104Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:38:45.196203Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:38:45.197700Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:38:45.197758Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:38:45.197768Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:38:45.198767Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:38:45.198848Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:38:45.198864Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-06T22:38:45.187Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 9 - "10.133.0.40" "Go-http-client/1.1" "5f5f6c66-ca54-4170-ac7a-f244d87bf9ab" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.134.0.51:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:53482 10.133.0.35:443 10.133.0.40:42966 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-06T22:38:45.163Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 471 35 - "100.64.0.4" "python-requests/2.32.5" "390359f3-bd57-406a-af16-2876210e4eb8" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:39966 10.133.0.35:443 100.64.0.4:24464 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-06T22:38:57.189148Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:38:57.189197Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:38:57.204498Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:38:57.204636Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:38:57.204889Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:38:57.210088Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:38:57.210148Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-06T22:38:57.210158Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:38:57.226154Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:38:57.226199Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:38:57.235793Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:38:57.235952Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:38:57.236154Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:38:57.239118Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:38:57.239174Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:38:57.239184Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-06T22:38:57.188Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 298 21 - "100.64.0.3" "python-requests/2.32.5" "6ec68f98-431d-4a67-8494-c171fb8c08b9" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:57840 10.133.0.35:443 100.64.0.3:36370 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:38:57.225Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 297 13 - "100.64.0.3" "python-requests/2.32.5" "a65dd9f7-f32a-4dcb-8804-ac1409a14373" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:39966 10.133.0.35:443 100.64.0.3:39106 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-06T22:39:05.287581Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:05.287643Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:05.291962Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:05.292084Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:05.294340Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:05.294379Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:05.300751Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:05.300897Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:05.301005Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:05.302366Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:05.302435Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:39:05.302450Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:05.303340Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:05.303394Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:39:05.303405Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:05.319486Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:05.319521Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:05.322865Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:05.322962Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:05.324782Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:05.324837Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:05.330673Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:05.330774Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:05.330876Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:05.331945Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:05.332005Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:39:05.332020Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:05.332915Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:05.332968Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-06T22:39:05.332979Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-06T22:39:05.293Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 9 - "10.133.0.40" "Go-http-client/1.1" "aad8524a-41da-4c60-b4e6-90f983a94e1c" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.134.0.51:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:53482 10.133.0.35:443 10.133.0.40:42966 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-06T22:39:05.286Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 468 16 - "100.64.0.4" "python-requests/2.32.5" "c3e45d7b-0ecb-4761-8b4e-aa10bcf7403c" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:39966 10.133.0.35:443 100.64.0.4:18442 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:39:05.323Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 8 - "10.133.0.40" "Go-http-client/1.1" "0d2820f9-1de1-4672-a69a-7fc6d8baadf1" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.44:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:45114 10.133.0.35:443 10.133.0.40:42966 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-06T22:39:05.318Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 345 14 - "100.64.0.2" "python-requests/2.32.5" "3df037d9-f294-4260-8877-8cc397eb1c54" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:57840 10.133.0.35:443 100.64.0.2:59792 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-06T22:39:21.743180Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:21.743233Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:21.756970Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:21.757105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:21.759554Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:21.759598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:21.769854Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:21.770012Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:21.771343Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:21.771416Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:39:21.771431Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:21.772941Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:21.772971Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:21.782545Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:21.782652Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:21.783698Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:21.783748Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:39:21.783758Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:21.785164Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:21.785194Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:21.794435Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:21.794537Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:21.795546Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:21.795596Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:39:21.795606Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:21.796515Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:21.796582Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-06T22:39:21.796594Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-06T22:39:21.758Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 13 - "10.133.0.40" "Go-http-client/1.1" "9e812707-5469-4c19-a1c1-b870a19ce2f5" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.134.0.51:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:53482 10.133.0.35:443 10.133.0.40:42966 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-06T22:39:21.771Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 11 - "10.133.0.40" "Go-http-client/1.1" "89883564-47ae-44c7-8fc5-a35fa756be33" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.44:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:45114 10.133.0.35:443 10.133.0.40:42966 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-06T22:39:21.784Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 11 - "10.133.0.40" "Go-http-client/1.1" "7fab057d-dd6c-41e0-b6d9-11cc5819b723" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.47:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:34702 10.133.0.35:443 10.133.0.40:42966 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-06T22:39:21.742Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 1361 54 - "100.64.0.3" "python-requests/2.32.5" "99bde4a4-46a4-417a-9992-a0998069f442" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:57840 10.133.0.35:443 100.64.0.3:34702 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:39:26.586Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 1967 - "-" "-" "-" "-" "-" - - 10.133.0.35:443 100.64.0.2:24042 - - 2026-06-06T22:39:36.142543Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:36.142588Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:36.156914Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:36.157039Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:36.160174Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:36.160208Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:36.170746Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:36.170895Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:36.172247Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:36.172306Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:39:36.172316Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:36.173409Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:36.173479Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-06T22:39:36.173495Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:36.191170Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:36.191210Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:36.200432Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:36.200555Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:36.202547Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:36.202580Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:36.212435Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:36.212548Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:36.214374Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:36.214427Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:39:36.214438Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:39:36.215296Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:36.215343Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-06T22:39:36.215353Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-06T22:39:36.159Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 13 - "10.133.0.40" "Go-http-client/1.1" "e28fefc4-0c11-458f-a481-9ac5a06b5475" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.134.0.51:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:53482 10.133.0.35:443 10.133.0.40:42966 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-06T22:39:36.141Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 470 31 - "100.64.0.2" "python-requests/2.32.5" "14a19222-85fa-4f04-90a7-acc2f4371aa4" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:57840 10.133.0.35:443 100.64.0.2:35084 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:39:36.201Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 12 - "10.133.0.40" "Go-http-client/1.1" "bb651bb6-b0f7-46ca-be36-63627306f323" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.44:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:45114 10.133.0.35:443 10.133.0.40:42966 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-06T22:39:36.190Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 347 24 - "100.64.0.4" "python-requests/2.32.5" "ef676932-9e09-4332-8bb8-22b3ab8f423f" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:57840 10.133.0.35:443 100.64.0.4:23702 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:39:41.957Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 0 - "-" "-" "-" "-" "-" - - 10.133.0.35:443 100.64.0.3:37094 - - 2026-06-06T22:39:45.227251Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:45.227305Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:45.228820Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:39:45.228860Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-06T22:39:45.226Z] "GET /maas-api/v1/models HTTP/1.1" 401 - - "-" 0 0 2 - "100.64.0.4" "python-requests/2.32.5" "9e6e58cb-406a-4ab4-b70c-c9d12ec160d9" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.35:443 100.64.0.4:2348 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-06T22:40:01.825902Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.825948Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.841543Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.841678Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.841916Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.846470Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.846530Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-06T22:40:01.846541Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.864095Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:01.864140Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:01.872378Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:01.872489Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:01.872741Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:01.872775Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:01.874205Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:01.874230Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:01.874432Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:01.876947Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:01.877080Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:01.877261Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:01.877924Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:01.899851Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.899882Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.907113Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.907220Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.907438Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.907466Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.908034Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.908056Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.908232Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.909366Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.909484Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.909639Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.910060Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-06T22:40:01.825Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 331 21 - "100.64.0.2" "python-requests/2.32.5" "c3ba0ed5-c168-4f9e-a7ca-f6f9ee879f69" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:57840 10.133.0.35:443 100.64.0.2:5664 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:40:01.862Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 278 15 - "100.64.0.3" "python-requests/2.32.5" "3882a7f5-e421-4b09-95d2-2dba79bf8655" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.46:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:45530 10.133.0.35:443 100.64.0.3:3740 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-06T22:40:01.898Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 11 - "100.64.0.4" "python-requests/2.32.5" "16788fa5-72af-4d44-9749-8ea0e2dc448a" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.46:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:44290 10.133.0.35:443 100.64.0.4:36698 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 2026-06-06T22:40:01.930186Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.930219Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.937478Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.937582Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.937793Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.937844Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.938378Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.938406Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.955034Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.955063Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.962270Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.962368Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.962569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.962598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.963105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.963126Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.992854Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.992887Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.996551Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.996645Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:01.999158Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:01.999191Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:02.005958Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:02.006072Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:02.006161Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:02.007246Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:02.007299Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:40:02.007309Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:02.008361Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:02.008410Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-06T22:40:02.008421Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-06T22:40:01.929Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 9 - "100.64.0.3" "python-requests/2.32.5" "a3a642c3-542a-469b-95ee-17842621c024" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.46:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:44290 10.133.0.35:443 100.64.0.3:6784 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-06T22:40:01.954Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 9 - "100.64.0.2" "python-requests/2.32.5" "2b97bfbb-fb2b-45e2-b2f5-7849d9e7c58a" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.46:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.35:443 100.64.0.2:50406 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-06T22:40:01.998Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.133.0.40" "Go-http-client/1.1" "0bf86bb0-c224-44fa-b3af-1215479bb36c" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.46:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:45530 10.133.0.35:443 10.133.0.40:42966 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.3 [2026-06-06T22:40:01.992Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 547 16 - "100.64.0.4" "python-requests/2.32.5" "0676fae5-73c7-4ee2-ad97-7740343cfdb8" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:57840 10.133.0.35:443 100.64.0.4:10644 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-06T22:40:10.890300Z info ads ADS: new connection for node:6 2026-06-06T22:40:10.892739Z info cache read certificate from file resource=file-root:system 2026-06-06T22:40:15.584089Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:15.584137Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:15.597924Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:15.598065Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:15.598285Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:15.603542Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:15.603605Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:40:15.603616Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-06T22:40:15.583Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 321 20 - "100.64.0.2" "python-requests/2.32.5" "1600472c-5b9e-45c2-8997-dd1841800998" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:42380 10.133.0.35:443 100.64.0.2:10974 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-06T22:40:15.949871Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:15.949912Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:15.951022Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:15.951053Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:15.970084Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:15.970120Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:15.971023Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:15.971051Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:40:15.998125Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:15.998160Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:16.005663Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:16.005842Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:16.006151Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:16.006185Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:16.007331Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:16.007355Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:16.007554Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:16.008411Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:40:16.008515Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:40:16.008526Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-06T22:40:15.947Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 3 - "100.64.0.3" "python-requests/2.32.5" "540ee085-35e1-4e20-abd1-f9b8ffb9f404" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.133.0.35:443 100.64.0.3:37178 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-06-06T22:40:15.968Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 2 - "100.64.0.3" "python-requests/2.32.5" "d4a6a8c2-832a-4cca-b2cf-da2705ec873d" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.133.0.35:443 100.64.0.3:18716 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-06-06T22:40:15.996Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 404 - - "-" 83 98 11 - "100.64.0.4" "python-requests/2.32.5" "5f724807-e44e-4157-b3a6-5356ee578a24" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.133.0.35:443 100.64.0.4:22110 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.e2e-external-model.0 2026-06-06T22:40:53.531497Z info ads ADS: "" 6 terminated 2026-06-06T22:41:21.244271Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:21.244321Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:21.247271Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:21.247382Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:21.247565Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:21.262195Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:21.262267Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:41:21.262278Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:21.378203Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:21.378248Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:21.412143Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:21.412185Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:21.433387Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:21.433432Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:21.435055Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:21.435087Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:21.455500Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:21.455540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:21.456918Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:21.456948Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:21.803304Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:21.803350Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:21.806250Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:21.806356Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:21.806537Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:21.810030Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:21.810088Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:41:21.810099Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-06T22:41:21.243Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 18 - "100.64.0.2" "python-requests/2.32.5" "6c2e333a-bf41-4ffe-a0db-42867850b792" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:41800 10.133.0.35:443 100.64.0.2:17068 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:41:21.377Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 37 0 34 - "100.64.0.3" "python-requests/2.32.5" "e65c2205-71cb-4fd0-a23e-411b86e90fcf" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.35:443 100.64.0.3:32420 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:41:21.432Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 35 0 2 - "100.64.0.2" "python-requests/2.32.5" "51781ab8-7631-4d78-a3d6-45ab261c8898" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.35:443 100.64.0.2:14586 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:41:21.454Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 36 0 2 - "100.64.0.4" "python-requests/2.32.5" "08289554-7192-41d4-b179-209ea7fcbc0c" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.35:443 100.64.0.4:43282 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:41:21.802Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 28 313 7 - "100.64.0.4" "python-requests/2.32.5" "ae90266f-83b1-40b9-a870-e8281f079974" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:41800 10.133.0.35:443 100.64.0.4:6748 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-06T22:41:22.004380Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:22.004430Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:22.007399Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:22.007504Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:22.007682Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:22.012967Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:22.013041Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-06T22:41:22.013056Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:22.033040Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:22.033077Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:22.045795Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:22.045930Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:22.048051Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.048087Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.054371Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.054503Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.054603Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.056660Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.056718Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:41:22.056728Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.057709Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:22.057762Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-06T22:41:22.057773Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:22.074416Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.074451Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.081104Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.081232Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.081476Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.081507Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.082952Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.082978Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.083180Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.084655Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.084781Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.084988Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.085522Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.160426Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.160473Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.163224Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.163333Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.163517Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.166640Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.166697Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:41:22.166705Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:22.183290Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:22.183327Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:22.187269Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:22.187372Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:22.191492Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:22.191545Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-06T22:41:22.191555Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-06T22:41:22.003Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 9 - "100.64.0.2" "python-requests/2.32.5" "3684d6f0-28f5-4524-83be-594f0541960d" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:41808 10.133.0.35:443 100.64.0.2:17084 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:41:22.046Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.133.0.40" "Go-http-client/1.1" "01c58fbe-311c-4da2-abf6-92e8d2de05d7" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.47:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:34702 10.133.0.35:443 10.133.0.40:42966 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-06T22:41:22.032Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 25 - "100.64.0.3" "python-requests/2.32.5" "e52e6549-190d-4086-81ee-f0ffafcf6fe8" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:41808 10.133.0.35:443 100.64.0.3:46934 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:41:22.073Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 123 379 12 - "100.64.0.2" "python-requests/2.32.5" "ebd05bce-e099-4543-8ba4-c165d9ff1ace" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.47:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:34702 10.133.0.35:443 100.64.0.2:14590 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-06-06T22:41:22.159Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 7 - "100.64.0.4" "python-requests/2.32.5" "939466fb-bc5f-4bc2-bf25-0c493dafb8c7" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:41800 10.133.0.35:443 100.64.0.4:43290 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:41:22.182Z] "DELETE /v1/api-keys/9dafceec-6926-44e9-8b7f-f4e38069adf1 HTTP/1.1" 200 - via_upstream - "-" 0 251 9 - "100.64.0.3" "python-requests/2.32.5" "221bfe61-d567-44ca-80df-3b11d3c2554c" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:41808 10.133.0.35:443 100.64.0.3:46950 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-06T22:41:25.219404Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.219459Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.223728Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.223835Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.225188Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.225257Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:41:25.225272Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.338319Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.338367Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.353007Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.353049Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.484173Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.484219Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.487042Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.487181Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.487439Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.491141Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.491216Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:41:25.491230Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.511790Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.511851Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.514614Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.514717Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.514935Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.517895Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.517951Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:41:25.517961Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.595453Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.595499Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.598313Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.598430Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.598628Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.601771Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.601856Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:41:25.601868Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.629199Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:25.629238Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:25.632099Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:25.632201Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:25.635738Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:25.635794Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-06T22:41:25.635819Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:25.653392Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.653428Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.656293Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.656397Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.658397Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.658449Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:41:25.658458Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.744510Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.744557Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.747361Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.747469Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.747657Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.750797Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.750871Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:41:25.750882Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.767661Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:25.767697Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:25.771516Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:25.771617Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:25.773515Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.773548Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.780169Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.780291Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.780395Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.781471Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.781522Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:41:25.781532Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.782419Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:25.782469Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-06T22:41:25.782480Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:25.861494Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.861538Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.864457Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.864564Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.864766Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.867916Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.867990Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:41:25.868005Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.885187Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.885228Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.889965Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.890085Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.891491Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.891546Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:41:25.891556Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.910682Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:25.910726Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:25.914293Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:25.914411Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:25.916492Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.916526Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.922834Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.922965Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.923062Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.924190Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.924247Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:41:25.924257Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:25.925234Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:25.925288Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-06T22:41:25.925299Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-06T22:41:25.218Z] "GET /v1/models HTTP/1.1" 500 - via_upstream - "-" 0 96 6 - "100.64.0.2" "python-requests/2.32.5" "b258fec2-b536-4322-825b-9888089d48a2" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:41800 10.133.0.35:443 100.64.0.2:14598 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:41:25.337Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 38 0 15 - "100.64.0.4" "python-requests/2.32.5" "54866ebc-d550-47d1-80ab-3a2e87531eb8" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.35:443 100.64.0.4:43292 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:41:25.483Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 34 318 8 - "100.64.0.3" "python-requests/2.32.5" "149f9bbc-440c-42aa-a513-5cb8d27cbe76" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:41800 10.133.0.35:443 100.64.0.3:32426 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:41:25.510Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 32 317 6 - "100.64.0.4" "python-requests/2.32.5" "1d8b2d36-f57a-45fc-be4e-0006fe64a24a" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:41800 10.133.0.35:443 100.64.0.4:6758 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:41:25.594Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 31 315 7 - "100.64.0.3" "python-requests/2.32.5" "764fbd87-63df-4329-b1ab-50dc97ef8cbb" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:41800 10.133.0.35:443 100.64.0.3:46964 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:41:25.628Z] "DELETE /v1/api-keys/159c17dc-9be5-492f-a738-2ecf154ec44b HTTP/1.1" 200 - via_upstream - "-" 0 253 7 - "100.64.0.2" "python-requests/2.32.5" "6429d096-37df-4738-b533-3689178c8a4d" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:41808 10.133.0.35:443 100.64.0.2:14610 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:41:25.652Z] "DELETE /v1/api-keys/159c17dc-9be5-492f-a738-2ecf154ec44b HTTP/1.1" 404 - via_upstream - "-" 0 29 5 - "100.64.0.2" "python-requests/2.32.5" "5d1faba7-17a2-4e07-8133-02120bc31c91" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:41800 10.133.0.35:443 100.64.0.2:17090 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:41:25.743Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 7 - "100.64.0.4" "python-requests/2.32.5" "fef3069b-7ff0-4164-8411-2df1fdc403ac" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:41800 10.133.0.35:443 100.64.0.4:43298 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:41:25.772Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.133.0.40" "Go-http-client/1.1" "57560840-df97-4d18-a91e-0b204844174c" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.47:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:34702 10.133.0.35:443 10.133.0.40:42966 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-06T22:41:25.766Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 15 - "100.64.0.3" "python-requests/2.32.5" "dc5f1cd1-0519-4d46-8ad7-70afab54f446" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:41808 10.133.0.35:443 100.64.0.3:46980 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:41:25.860Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 7 - "100.64.0.2" "python-requests/2.32.5" "3e854887-a17e-4680-9ef8-f3e85061427e" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:41800 10.133.0.35:443 100.64.0.2:14624 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:41:25.884Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 7 - "100.64.0.3" "python-requests/2.32.5" "81acd63b-2f76-41b3-ad8f-c3a8e8d33b3c" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:41800 10.133.0.35:443 100.64.0.3:32430 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:41:25.915Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.133.0.40" "Go-http-client/1.1" "0fb3b3b2-2b2e-4213-94ae-3378a69a3cbf" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.47:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:34702 10.133.0.35:443 10.133.0.40:42966 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-06T22:41:25.909Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 15 - "100.64.0.4" "python-requests/2.32.5" "609e70ee-4c2a-495e-b6d8-0bab8434b3c1" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:41808 10.133.0.35:443 100.64.0.4:43306 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-06T22:41:26.002623Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.002671Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.005384Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.005493Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.005683Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.008893Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.008958Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:41:26.008968Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.025323Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.025361Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.029093Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.029194Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.031166Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.031199Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.037935Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.038108Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.038220Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.039376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.039434Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:41:26.039443Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.040339Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.040389Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:41:26.040399Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.056182Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:26.056224Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:26.059797Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:26.059936Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:26.062070Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.062104Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.068783Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.068939Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.069040Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.070204Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.070262Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:41:26.070272Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.071215Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:26.071270Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-06T22:41:26.071280Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-06T22:41:26.148547Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.148592Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.151453Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.151563Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.151756Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.154767Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-06T22:41:26.154842Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-06T22:41:26.154853Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-06T22:41:26.001Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 7 - "100.64.0.4" "python-requests/2.32.5" "8c74517f-a97c-4e9d-8c7c-ca0dd4f3a680" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:41800 10.133.0.35:443 100.64.0.4:6772 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:41:26.030Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.133.0.40" "Go-http-client/1.1" "0c6d821a-fbe2-4d15-b871-315f116f5f3a" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.47:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:34702 10.133.0.35:443 10.133.0.40:42966 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-06T22:41:26.024Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 15 - "100.64.0.2" "python-requests/2.32.5" "9cc6d8eb-a37e-4e90-a483-1cba90dec993" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:41800 10.133.0.35:443 100.64.0.2:17092 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:41:26.060Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.133.0.40" "Go-http-client/1.1" "d4d47ea0-9efe-4ef8-8ed0-086d258517b1" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.47:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.35:34702 10.133.0.35:443 10.133.0.40:42966 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-06T22:41:26.055Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 15 - "100.64.0.3" "python-requests/2.32.5" "d6f8fd45-0fac-4bd2-8ef4-21d0dba99aab" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:41808 10.133.0.35:443 100.64.0.3:46992 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-06T22:41:26.147Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 36 321 7 - "100.64.0.2" "python-requests/2.32.5" "10178468-8631-416c-a5d7-25786ddd452e" "maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com" "10.133.0.40:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.35:41800 10.133.0.35:443 100.64.0.2:14636 maas.apps.8f21ea4f-2c23-4943-be76-968d4746bf91.prod.konfluxeaas.com opendatahub.maas-api-route.1