2026-05-27T14:34:54.081157Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:34:54.083321Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:34:54.083380Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T14:34:54.083391Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:34:54.084335Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:34:54.084384Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T14:34:54.084394Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:34:54.461960Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:34:54.462009Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:34:54.468354Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:34:54.468531Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:34:54.468820Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:34:54.472345Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:34:54.472409Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T14:34:54.472420Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-27T14:34:54.072Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 10 - "10.133.0.45" "Go-http-client/1.1" "cd87e0bf-932a-4823-ae2f-a8c52698dada" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:44108 10.132.0.29:443 10.133.0.45:35154 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-27T14:34:54.065Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 18 - "100.64.0.4" "python-requests/2.32.5" "0477acec-3fcf-49dc-a4c1-b20259f6e68b" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:43454 10.132.0.29:443 100.64.0.4:17980 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-27T14:34:54.460Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 33 317 11 - "100.64.0.2" "python-requests/2.32.5" "7456df60-7a08-46bb-ad18-34c95e522e22" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:43588 10.132.0.29:443 100.64.0.2:63632 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T14:35:02.512257Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:35:02.512304Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:35:02.516262Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:35:02.516393Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:35:02.518537Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:02.518576Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:02.524875Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:02.525038Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:02.525150Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:02.526575Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:02.526632Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T14:35:02.526642Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:02.527559Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:35:02.527610Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T14:35:02.527621Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-27T14:35:02.517Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.133.0.45" "Go-http-client/1.1" "0fa1794e-c4e3-465b-be54-61df12fca1dc" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:44108 10.132.0.29:443 10.133.0.45:35154 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-27T14:35:02.511Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 16 - "100.64.0.3" "python-requests/2.32.5" "de7eb572-e9b3-48ff-8d0b-e4bd9d60c8c3" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:43588 10.132.0.29:443 100.64.0.3:58526 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T14:35:03.248761Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:03.248810Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:03.254918Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:03.255070Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:03.255323Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:03.258760Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:03.258819Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T14:35:03.258830Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T14:35:03.247Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 322 11 - "100.64.0.4" "python-requests/2.32.5" "4d9aca00-4614-4490-b9d5-b41b7b6e15dc" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:43454 10.132.0.29:443 100.64.0.4:8692 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T14:35:11.288414Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:11.288466Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:11.292315Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:11.292464Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:11.294670Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:11.294711Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:11.294726Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:11.301266Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:11.301406Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:11.301531Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:11.301615Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:11.303362Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:11.303391Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:11.303444Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T14:35:11.303454Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:11.303467Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T14:35:11.303472Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:11.304551Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:11.304616Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T14:35:11.304627Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T14:35:11.293Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.133.0.45" "Go-http-client/1.1" "77b9f3e1-f0a8-497c-a735-b3e6be8527df" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:44108 10.132.0.29:443 10.133.0.45:35154 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-27T14:35:11.287Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 474 17 - "100.64.0.2" "python-requests/2.32.5" "45842019-56f5-4ba9-af96-430b41c9eca0" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:43454 10.132.0.29:443 100.64.0.2:33376 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T14:35:20.476341Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:20.476389Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:20.481927Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:20.482063Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:20.482289Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:20.486004Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:20.486081Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T14:35:20.486094Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T14:35:20.475Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 85 327 11 - "100.64.0.4" "python-requests/2.32.5" "3bf7422c-e413-921e-87bc-6074daa73052" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:43454 10.132.0.29:443 100.64.0.4:43672 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T14:35:36.750446Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:35:36.750518Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:35:36.754970Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:35:36.755144Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:35:36.757064Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:35:36.757124Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T14:35:36.757134Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-27T14:35:36.749Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 82 7 - "100.64.0.2" "python-requests/2.32.5" "7f3cb26e-a3b9-42df-baaa-a28977cb0b0b" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:43588 10.132.0.29:443 100.64.0.2:61634 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T14:35:53.899665Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:53.899719Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:53.906935Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:53.907093Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:53.908813Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:35:53.908871Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T14:35:53.908882Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T14:35:53.898Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 10 - "100.64.0.3" "python-requests/2.32.5" "bb046d2a-4d6d-48d7-b81d-f9f1b02af4de" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:43454 10.132.0.29:443 100.64.0.3:21734 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T14:36:11.092963Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:36:11.093012Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:36:11.101030Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:36:11.101179Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:36:11.103044Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:36:11.103105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T14:36:11.103115Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T14:36:11.091Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 82 11 - "100.64.0.3" "python-requests/2.32.5" "08dd108b-6c41-4761-a849-8ccc90e8e465" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:43454 10.132.0.29:443 100.64.0.3:12138 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T14:36:28.650265Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:36:28.650314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:36:28.656389Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:36:28.656557Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:36:28.658344Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:36:28.658405Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T14:36:28.658415Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T14:36:28.649Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 9 - "100.64.0.4" "python-requests/2.32.5" "8530e3b1-d7f2-4a35-b171-75f7c31f6751" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:43454 10.132.0.29:443 100.64.0.4:55604 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T14:36:40.810889Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:36:40.810971Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:36:40.825386Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:36:40.825517Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:36:40.825733Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:36:40.829436Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:36:40.829493Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T14:36:40.829504Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T14:36:40.809Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 45 329 19 - "100.64.0.2" "python-requests/2.32.5" "5d3d0482-3a1c-4391-a89f-c15413e5b4fd" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:43454 10.132.0.29:443 100.64.0.2:12658 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T14:36:48.869714Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:36:48.869774Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:36:48.884069Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:36:48.884232Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:36:48.895386Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:36:48.895430Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:36:48.902612Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:36:48.902757Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:36:48.902877Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:36:48.905598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:36:48.905664Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T14:36:48.905679Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:36:48.907415Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:36:48.907475Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T14:36:48.907491Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-27T14:36:48.894Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 11 - "10.133.0.45" "Go-http-client/1.1" "b14769cd-3966-4a86-beec-73149d76990e" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.47:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:45768 10.132.0.29:443 10.133.0.45:58142 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.2 [2026-05-27T14:36:48.868Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 471 38 - "100.64.0.4" "python-requests/2.32.5" "c0b9e901-da6e-4039-9e3a-06551bc79d59" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:35276 10.132.0.29:443 100.64.0.4:9032 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T14:37:01.111350Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:01.111399Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:01.120305Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:01.120432Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:01.120641Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:01.124317Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:01.124376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T14:37:01.124387Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T14:37:01.110Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 297 14 - "100.64.0.2" "python-requests/2.32.5" "b18c3e53-deb2-4c2d-9553-72ee8e1a880c" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:43454 10.132.0.29:443 100.64.0.2:26122 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T14:37:01.148689Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:01.148729Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:01.154603Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:01.154724Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:01.154945Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:01.158180Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:01.158235Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T14:37:01.158246Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-27T14:37:01.147Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 297 10 - "100.64.0.3" "python-requests/2.32.5" "f98f8317-b3f0-466e-8b59-37de581b614f" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:35276 10.132.0.29:443 100.64.0.3:60222 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T14:37:09.187258Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:09.187307Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:09.191264Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:09.191394Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:09.193598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:09.193633Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:09.200090Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:09.200215Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:09.200325Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:09.201745Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:09.201807Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T14:37:09.201818Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:09.202798Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:09.202853Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T14:37:09.202863Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:09.225723Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:09.225773Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:09.229199Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:09.229334Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:09.231444Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:09.231483Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:09.237644Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:09.237767Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:09.237881Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:09.241682Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:09.241742Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T14:37:09.241753Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:09.243538Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:09.243597Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T14:37:09.243608Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T14:37:09.192Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 9 - "10.133.0.45" "Go-http-client/1.1" "6bd05ea6-3909-471d-9e24-06ad47fe0e1f" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.47:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:45768 10.132.0.29:443 10.133.0.45:58142 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.2 [2026-05-27T14:37:09.186Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 468 16 - "100.64.0.4" "python-requests/2.32.5" "147b84c8-05ce-4b28-8690-0f96a9bbd0ad" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:35276 10.132.0.29:443 100.64.0.4:17894 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-27T14:37:09.230Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 11 - "10.133.0.45" "Go-http-client/1.1" "4984cef6-fb91-4553-b8a4-c6bc6f132d95" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.46:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:55264 10.132.0.29:443 10.133.0.45:58142 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.2 [2026-05-27T14:37:09.224Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 345 19 - "100.64.0.3" "python-requests/2.32.5" "8c03ebb3-3a14-4727-9450-423acd7d859b" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:43454 10.132.0.29:443 100.64.0.3:40742 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T14:37:25.838087Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.838133Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.845967Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.846097Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.848785Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.848822Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.859507Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.859624Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.861045Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.861103Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T14:37:25.861114Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.862614Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.862646Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.874880Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.875010Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.876267Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.876316Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T14:37:25.876327Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.877545Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.877577Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.887408Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.887516Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.891522Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.891575Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T14:37:25.891586Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.892582Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:25.892639Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T14:37:25.892651Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-27T14:37:25.847Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 13 - "10.133.0.45" "Go-http-client/1.1" "81f1da3f-1b3a-48db-b170-78c91ba92628" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.47:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:45768 10.132.0.29:443 10.133.0.45:58142 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.2 [2026-05-27T14:37:25.861Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 14 - "10.133.0.45" "Go-http-client/1.1" "45480cba-cd30-4d48-b768-272153281647" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.46:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:55264 10.132.0.29:443 10.133.0.45:58142 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.2 [2026-05-27T14:37:25.876Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 14 - "10.133.0.45" "Go-http-client/1.1" "cd229cff-befe-478a-bf9e-f72983eaf2e4" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:37224 10.132.0.29:443 10.133.0.45:58142 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-27T14:37:25.836Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 1361 55 - "100.64.0.2" "python-requests/2.32.5" "bbbe6cf7-6bdf-428d-9f9f-e225de668132" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:35276 10.132.0.29:443 100.64.0.2:52094 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T14:37:42.424125Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:42.424173Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:42.430867Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:42.431001Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:42.433203Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:42.433240Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:42.445823Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:42.445959Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:42.447215Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:42.447272Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T14:37:42.447283Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:42.448282Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:42.448335Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T14:37:42.448347Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:42.467865Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:42.467934Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:42.473927Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:42.474027Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:42.475834Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:42.475877Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:42.486138Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:42.486246Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:42.487399Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:42.487445Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T14:37:42.487456Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:37:42.488323Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:42.488372Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T14:37:42.488382Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T14:37:42.432Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 15 - "10.133.0.45" "Go-http-client/1.1" "6b416584-aea8-4af4-a955-a279db5d5241" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.47:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:45768 10.132.0.29:443 10.133.0.45:58142 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.2 [2026-05-27T14:37:42.422Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 470 25 - "100.64.0.3" "python-requests/2.32.5" "cae51b22-cda1-4534-a60d-19d96cd8bb0f" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:43454 10.132.0.29:443 100.64.0.3:57892 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-27T14:37:42.475Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 12 - "10.133.0.45" "Go-http-client/1.1" "2d0cd6be-9919-4e40-80bd-b8e708808615" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.46:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:55264 10.132.0.29:443 10.133.0.45:58142 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.2 [2026-05-27T14:37:42.466Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 347 21 - "100.64.0.4" "python-requests/2.32.5" "c15cb6a7-5cf0-4a5d-a228-73590ce4cc50" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:43454 10.132.0.29:443 100.64.0.4:2720 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T14:37:51.558433Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:51.558482Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:51.559567Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:37:51.559600Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T14:37:51.557Z] "GET /maas-api/v1/models HTTP/1.1" 401 - - "-" 0 0 2 - "100.64.0.4" "python-requests/2.32.5" "5efe82cb-ed56-40f2-ad5f-5bcd6b4c9df9" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.132.0.29:443 100.64.0.4:11592 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T14:38:08.209538Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.209598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.216730Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.216889Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.217169Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.222094Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.222163Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T14:38:08.222174Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.248536Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.248585Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.256470Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.256604Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.256946Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.256986Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.258581Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.258608Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.258853Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.260379Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.260524Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.260714Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.261324Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.280676Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:08.280724Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:08.287393Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:08.287521Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:08.287833Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:08.287878Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:08.288544Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:08.288571Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:08.313549Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.313600Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.320415Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.320551Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.320877Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.320940Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.321614Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.321642Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:08.341919Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:08.341965Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:08.345353Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:08.345488Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:08.348133Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:08.348168Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:08.354716Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:08.354839Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:08.354989Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:08.357543Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:08.357610Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T14:38:08.357621Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:08.358940Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:08.358992Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T14:38:08.359002Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-27T14:38:08.208Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 330 13 - "100.64.0.2" "python-requests/2.32.5" "dd5ad863-f86f-46fd-85ea-b9ae513a4ae6" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:43454 10.132.0.29:443 100.64.0.2:36048 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-27T14:38:08.247Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 285 13 - "100.64.0.2" "python-requests/2.32.5" "6fc97071-9676-46f5-b57c-0c548d31baec" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.132.0.38:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:43846 10.132.0.29:443 100.64.0.2:37140 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-05-27T14:38:08.280Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 8 - "100.64.0.3" "python-requests/2.32.5" "22b79204-a373-44ed-99ff-97a8e90f8f03" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.132.0.38:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:43860 10.132.0.29:443 100.64.0.3:45944 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-05-27T14:38:08.312Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 8 - "100.64.0.4" "python-requests/2.32.5" "462f5f70-a1f7-41c4-8049-449d40d4c0d4" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.132.0.38:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:43846 10.132.0.29:443 100.64.0.4:33384 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-05-27T14:38:08.347Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 10 - "10.133.0.45" "Go-http-client/1.1" "7920f3a9-3396-4ce3-a865-9a57dd2c6077" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.132.0.38:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:57596 10.132.0.29:443 10.133.0.45:58142 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.2 [2026-05-27T14:38:08.340Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 547 18 - "100.64.0.2" "python-requests/2.32.5" "cb5841b5-4e23-4f71-8fd2-0f7d3cb21ccc" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:35276 10.132.0.29:443 100.64.0.2:36064 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T14:38:17.213672Z info ads ADS: new connection for node:6 2026-05-27T14:38:17.218264Z info cache read certificate from file resource=file-root:system 2026-05-27T14:38:23.989698Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:23.989749Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:23.995587Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:23.995726Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:23.995982Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:23.999113Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:23.999173Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T14:38:23.999184Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T14:38:23.988Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 322 10 - "100.64.0.3" "python-requests/2.32.5" "e0ccf33e-ff77-4f92-b1de-be5407b10fa8" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:43454 10.132.0.29:443 100.64.0.3:1678 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T14:38:24.403771Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:24.403820Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:24.404771Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:24.404806Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:38:24.427394Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:24.427436Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:24.428361Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:24.428392Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:24.454442Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:24.454481Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:24.461016Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:24.461138Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:24.461427Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:24.461459Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:24.462966Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:24.462993Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:24.463231Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:24.464151Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:38:24.464278Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T14:38:24.464290Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T14:38:24.402Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 2 - "100.64.0.4" "python-requests/2.32.5" "503c2e50-9b57-40a9-9184-800d52da4cb0" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.132.0.29:443 100.64.0.4:41294 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-05-27T14:38:24.426Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 2 - "100.64.0.2" "python-requests/2.32.5" "ff68768d-325c-4227-8ba2-af883bf0bf2a" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.132.0.29:443 100.64.0.2:13366 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-05-27T14:38:24.453Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 404 - - "-" 83 98 10 - "100.64.0.3" "python-requests/2.32.5" "641e595d-599d-4214-bcab-939bb1aa5399" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.132.0.29:443 100.64.0.3:1680 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.e2e-external-model.0 2026-05-27T14:39:02.175522Z info ads ADS: "" 6 terminated 2026-05-27T14:39:06.133364Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.133415Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.140107Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.140259Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.140491Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.154056Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.154147Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T14:39:06.154160Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.196089Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.196136Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.201910Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.202052Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.204722Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.204768Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.211354Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.211515Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.211649Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.213220Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.213292Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T14:39:06.213306Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.214918Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.214993Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T14:39:06.215005Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T14:39:06.132Z] "POST /v1/api-keys HTTP/2" 201 - via_upstream - "-" 55 321 22 - "100.64.0.4" "curl/7.76.1" "1f6cbe50-a6fd-4c4d-8164-a25dbc5e9851" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:43454 10.132.0.29:443 100.64.0.4:29996 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-27T14:39:06.203Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.133.0.45" "Go-http-client/1.1" "a6f86e81-c3f9-4d4e-a6cb-309085012c3c" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:37224 10.132.0.29:443 10.133.0.45:58142 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-27T14:39:06.194Z] "GET /v1/models HTTP/2" 200 - via_upstream - "-" 0 590 20 - "100.64.0.3" "curl/7.76.1" "e7dd1d57-5116-441f-b37c-494a806cb736" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:43454 10.132.0.29:443 100.64.0.3:57742 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T14:39:06.262226Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.262277Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.272078Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.272244Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.272579Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.272623Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.274336Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.274371Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.274643Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.276418Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.276584Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.276778Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.277368Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.312775Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.312820Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.319420Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.319569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.319881Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.319932Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.320569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.320592Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.320817Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.322535Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.322689Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.322863Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.323314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.346942Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.346990Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.354208Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.354354Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.354671Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.354705Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.355329Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.355370Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.355642Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.357488Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.357651Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.357840Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.358346Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.391054Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.391098Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.399769Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.399950Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.400274Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.400306Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.400860Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.400888Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.432030Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.432074Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.439083Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.439227Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.439513Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.439546Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.440109Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.440134Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.468260Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.468315Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.476241Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.476391Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.476709Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.476742Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.477367Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.477394Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.503518Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.503564Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.510291Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.510435Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.510742Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.510773Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.511378Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.511405Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.538461Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.538506Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.545042Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.545181Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.545472Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.545505Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.546121Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.546147Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.573485Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.573539Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.580937Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.581084Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.581408Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.581446Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.582078Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.582105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.607714Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.607760Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.615026Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.615175Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.615560Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.615601Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.616208Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.616238Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.641638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.641683Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.648406Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.648552Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.648849Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.648881Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.649577Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.649606Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.677153Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.677199Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.678130Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.678163Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T14:39:06.703510Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.703568Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.709277Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.709418Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.714103Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T14:39:06.714187Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T14:39:06.714199Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-27T14:39:06.261Z] "POST /v1/chat/completions HTTP/2" 200 - via_upstream - "-" 100 399 15 - "100.64.0.2" "curl/7.76.1" "840e5a41-4d09-4488-b24d-df376c63fcff" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:37224 10.132.0.29:443 100.64.0.2:25110 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T14:39:06.312Z] "POST /v1/chat/completions HTTP/2" 200 - via_upstream - "-" 100 439 11 - "100.64.0.3" "curl/7.76.1" "6eb229f4-80ca-47c8-81be-4080f59798d2" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:44108 10.132.0.29:443 100.64.0.3:64574 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T14:39:06.346Z] "POST /v1/chat/completions HTTP/2" 200 - via_upstream - "-" 100 523 12 - "100.64.0.4" "curl/7.76.1" "2e929ba5-6fd2-9582-9a10-e35ab2284379" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:37224 10.132.0.29:443 100.64.0.4:9186 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T14:39:06.390Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 10 - "100.64.0.2" "curl/7.76.1" "08e038f9-3ab8-4704-a698-794b7a23084a" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:37224 10.132.0.29:443 100.64.0.2:53756 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T14:39:06.431Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 9 - "100.64.0.4" "curl/7.76.1" "56499aa3-bf27-4978-8b5a-8cc45a7ed965" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.29:443 100.64.0.4:30000 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T14:39:06.467Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 10 - "100.64.0.3" "curl/7.76.1" "6af82a63-b8c1-4d6b-ada1-0dba4d503617" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:43834 10.132.0.29:443 100.64.0.3:57756 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T14:39:06.502Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 8 - "100.64.0.2" "curl/7.76.1" "bdb32824-655e-477d-9734-eb13fb69c273" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.29:443 100.64.0.2:25118 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T14:39:06.537Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 8 - "100.64.0.3" "curl/7.76.1" "5e861e77-e142-4192-aa15-29f2dade6a9a" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:43836 10.132.0.29:443 100.64.0.3:64576 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T14:39:06.572Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 9 - "100.64.0.4" "curl/7.76.1" "154c1b2b-2e0d-4e0e-ad71-49d2932c2d69" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:44108 10.132.0.29:443 100.64.0.4:30010 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T14:39:06.607Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 9 - "100.64.0.4" "curl/7.76.1" "5a3cd209-eadd-4208-93cf-df3daf3ce6ad" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.29:443 100.64.0.4:9192 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T14:39:06.640Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 8 - "100.64.0.2" "curl/7.76.1" "81aa4304-0ed8-4e12-a630-78804a6f7b5c" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.134.0.28:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.29:43844 10.132.0.29:443 100.64.0.2:25128 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T14:39:06.676Z] "POST /llm/facebook-opt-125m-simulated/v1/chat/completions HTTP/2" 401 - - "-" 100 0 1 - "100.64.0.3" "curl/7.76.1" "b901b246-46d6-4077-a5f2-50b084d51b5a" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.132.0.29:443 100.64.0.3:64586 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T14:39:06.701Z] "DELETE /v1/api-keys/bd57721a-104a-42a3-819a-fed914dfe35d HTTP/2" 200 - via_upstream - "-" 0 334 12 - "100.64.0.4" "curl/7.76.1" "3758f7cc-cd70-43f6-814f-285d0d169e75" "maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com" "10.133.0.45:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.29:35276 10.132.0.29:443 100.64.0.4:30022 maas.apps.f3e10ad1-a280-4bf5-bfe4-53a1d45acd3a.prod.konfluxeaas.com opendatahub.maas-api-route.1