2026-05-27T20:00:17.810932Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:17.810991Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T20:00:17.811002Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-27T20:00:17.800Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 33 317 10 - "100.64.0.2" "python-requests/2.32.5" "c5936844-a728-4846-9c16-12f220dbf73a" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:38128 10.133.0.45:443 100.64.0.2:47078 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T20:00:25.839173Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:00:25.839222Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:00:25.843271Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:00:25.843408Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:00:25.845906Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:25.845943Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:25.855878Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:25.856038Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:25.856171Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:25.857761Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:25.857827Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T20:00:25.857840Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:25.859309Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:00:25.859364Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:00:25.859375Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:00:26.558806Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:00:26.559015Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:00:26.564694Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:00:26.564883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:00:26.565163Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:00:26.569103Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:00:26.569183Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:00:26.569196Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T20:00:25.845Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 12 - "10.134.0.38" "Go-http-client/1.1" "b10aaefe-c185-4f88-afcc-7418c1a67524" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:53264 10.133.0.45:443 10.134.0.38:54056 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-27T20:00:25.838Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 21 - "100.64.0.3" "python-requests/2.32.5" "c57b9725-2168-41ed-89b6-b784fac32c07" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:59060 10.133.0.45:443 100.64.0.3:56230 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-27T20:00:26.557Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 322 11 - "100.64.0.4" "python-requests/2.32.5" "b9e073f8-2775-4f9a-ac32-f612eeb02470" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:59060 10.133.0.45:443 100.64.0.4:45250 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T20:00:34.613057Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:34.613107Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:34.616813Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:34.616934Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:34.619259Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:34.619293Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:34.619307Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:34.625205Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:34.625333Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:34.625466Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:34.625543Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:34.626987Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:34.627011Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:34.627063Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T20:00:34.627081Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:34.627096Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T20:00:34.627102Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:34.628217Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:00:34.628268Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T20:00:34.628279Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-27T20:00:34.618Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.134.0.38" "Go-http-client/1.1" "6ac3b58f-56e1-4b55-a42b-7ed5ffa861b9" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:53264 10.133.0.45:443 10.134.0.38:54056 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-27T20:00:34.611Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 474 16 - "100.64.0.3" "python-requests/2.32.5" "f437dd96-9524-48c4-aef7-29d99f2a53a0" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:38128 10.133.0.45:443 100.64.0.3:18070 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T20:00:43.758860Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:00:43.758913Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:00:43.764471Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:00:43.764597Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:00:43.764808Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:00:43.768567Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:00:43.768627Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:00:43.768638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T20:00:43.757Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 85 327 10 - "100.64.0.2" "python-requests/2.32.5" "88622c23-b80a-465c-a08b-0e5eb249d5f5" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:59060 10.133.0.45:443 100.64.0.2:5212 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T20:01:00.027939Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:01:00.027988Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:01:00.032490Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:01:00.032640Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:01:00.034864Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:01:00.034934Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:01:00.034951Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T20:01:00.026Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 82 8 - "100.64.0.2" "python-requests/2.32.5" "42d59b7d-5687-46b4-be49-6044827fb14b" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:59060 10.133.0.45:443 100.64.0.2:47970 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T20:01:17.035113Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:01:17.035163Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:01:17.041104Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:01:17.041233Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:01:17.043415Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:01:17.043472Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T20:01:17.043482Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-27T20:01:17.033Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 9 - "100.64.0.4" "python-requests/2.32.5" "88c6048c-3988-4970-ae13-2ee2ec16a3c8" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:38128 10.133.0.45:443 100.64.0.4:28048 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T20:01:34.152541Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:01:34.152594Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:01:34.158794Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:01:34.158949Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:01:34.161211Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:01:34.161285Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T20:01:34.161297Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-27T20:01:34.151Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 82 9 - "100.64.0.3" "python-requests/2.32.5" "a04d48df-5cbb-4b5c-a6e1-5585ca48362a" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:38128 10.133.0.45:443 100.64.0.3:57206 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T20:01:51.515229Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:01:51.515278Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:01:51.520882Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:01:51.521015Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:01:51.523310Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:01:51.523368Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T20:01:51.523379Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-27T20:01:51.514Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 9 - "100.64.0.2" "python-requests/2.32.5" "23edfe92-aa3b-4559-9cda-fbb432271480" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:38128 10.133.0.45:443 100.64.0.2:22144 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T20:02:05.551666Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:05.551715Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:05.557281Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:05.557440Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:05.557660Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:05.561636Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:05.561704Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T20:02:05.561715Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-27T20:02:05.550Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 45 329 11 - "100.64.0.4" "python-requests/2.32.5" "fbd5815b-a615-495a-907a-e46190e3b99b" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:38128 10.133.0.45:443 100.64.0.4:44048 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T20:02:13.588263Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:13.588315Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:13.603194Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:13.603345Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:13.616291Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:13.616331Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:13.622425Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:13.622562Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:13.622677Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:13.624212Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:13.624271Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:02:13.624282Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:13.625387Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:13.625462Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:02:13.625473Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T20:02:13.615Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 8 - "10.134.0.38" "Go-http-client/1.1" "357ccea7-4bfe-43b2-b6da-fc059d1b9ddb" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.49:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:56924 10.133.0.45:443 10.134.0.38:58070 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.2 [2026-05-27T20:02:13.587Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 471 38 - "100.64.0.3" "python-requests/2.32.5" "87384d0b-f793-4fb7-a440-d48348d8bfa5" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:42990 10.133.0.45:443 100.64.0.3:63654 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T20:02:27.827485Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:27.827543Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:27.833752Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:27.833898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:27.834119Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:27.837826Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:27.837893Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:02:27.837909Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:27.857529Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:27.857574Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:27.862548Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:27.862674Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:27.862869Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:27.866117Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:27.866173Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T20:02:27.866184Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-27T20:02:27.826Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 297 11 - "100.64.0.2" "python-requests/2.32.5" "589a8d90-94a1-4b52-9026-5752db7a4e82" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:42990 10.133.0.45:443 100.64.0.2:3706 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-27T20:02:27.856Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 298 9 - "100.64.0.4" "python-requests/2.32.5" "c0065159-ae6c-4ca0-897b-3d81150cf6f2" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:38128 10.133.0.45:443 100.64.0.4:36834 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T20:02:35.897598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:35.897652Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:35.901295Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:35.901443Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:35.903936Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:35.903977Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:35.910463Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:35.910610Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:35.910746Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:35.912514Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:35.912586Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:02:35.912596Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:35.913830Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:35.913898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T20:02:35.913910Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:35.932354Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:35.932431Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:35.935732Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:35.935874Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:35.938244Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:35.938289Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:35.944875Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:35.945020Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:35.945158Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:35.948322Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:35.948428Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:02:35.948441Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:35.950544Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:02:35.950623Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T20:02:35.950635Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-27T20:02:35.902Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 9 - "10.134.0.38" "Go-http-client/1.1" "6798842e-c1f3-47f9-afe1-648feeb1397a" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.49:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:56924 10.133.0.45:443 10.134.0.38:58070 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.2 [2026-05-27T20:02:35.896Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 468 17 - "100.64.0.4" "python-requests/2.32.5" "29cea268-8dc1-4a09-b15a-db3e8dbc8ade" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:38128 10.133.0.45:443 100.64.0.4:9762 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-27T20:02:35.937Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 11 - "10.134.0.38" "Go-http-client/1.1" "104ac797-c3b9-4a34-bc50-7c5ae5afda0e" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.50:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:59082 10.133.0.45:443 10.134.0.38:58070 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.2 [2026-05-27T20:02:35.931Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 345 19 - "100.64.0.3" "python-requests/2.32.5" "188a79bd-0429-4283-be28-a0f346852236" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:38128 10.133.0.45:443 100.64.0.3:14190 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T20:02:52.531986Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.532035Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.537024Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.537151Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.539699Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.539741Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.548531Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.548662Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.550286Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.550346Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:02:52.550357Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.551809Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.551841Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.561195Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.561311Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.562808Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.562865Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:02:52.562876Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.564138Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.564170Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.572910Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.573022Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.574469Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.574519Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:02:52.574529Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.575646Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:02:52.575701Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:02:52.575713Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T20:02:52.538Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 11 - "10.134.0.38" "Go-http-client/1.1" "7f6978c5-890c-4017-bcbf-8a79b4f641ee" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.49:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:56924 10.133.0.45:443 10.134.0.38:58070 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.2 [2026-05-27T20:02:52.551Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 11 - "10.134.0.38" "Go-http-client/1.1" "2202d0f1-0dfa-428f-b22c-0454152840a3" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.50:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:59082 10.133.0.45:443 10.134.0.38:58070 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.2 [2026-05-27T20:02:52.563Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 11 - "10.134.0.38" "Go-http-client/1.1" "49f98d54-53e6-4c08-b0c1-1c716901b7a1" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:49704 10.133.0.45:443 10.134.0.38:58070 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-27T20:02:52.530Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 1361 44 - "100.64.0.2" "python-requests/2.32.5" "8a28791e-fed3-42a6-88b4-3c6b8b874bdf" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:42990 10.133.0.45:443 100.64.0.2:37656 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T20:03:09.051822Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:09.051868Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:09.057826Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:09.057972Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:09.060545Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:09.060580Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:09.070663Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:09.070796Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:09.072545Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:09.072604Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:03:09.072615Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:09.073860Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:09.073914Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T20:03:09.073925Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:09.091538Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:09.091574Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:09.096333Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:09.096461Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:09.099041Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:09.099082Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:09.108606Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:09.108727Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:09.110684Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:09.110739Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:03:09.110749Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:09.111988Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:09.112038Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:03:09.112048Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T20:03:09.059Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 12 - "10.134.0.38" "Go-http-client/1.1" "5340e641-293d-496c-81ca-686c57871291" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.49:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:56924 10.133.0.45:443 10.134.0.38:58070 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.2 [2026-05-27T20:03:09.050Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 470 23 - "100.64.0.4" "python-requests/2.32.5" "30694700-3302-44fa-99ab-df5014d748b6" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:38128 10.133.0.45:443 100.64.0.4:25612 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-27T20:03:09.098Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 12 - "10.134.0.38" "Go-http-client/1.1" "d317d6a9-3186-4d74-9578-705069e2fd16" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.50:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:59082 10.133.0.45:443 10.134.0.38:58070 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.2 [2026-05-27T20:03:09.090Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 347 21 - "100.64.0.3" "python-requests/2.32.5" "00ced275-58d4-4ec8-b204-64fc5790953c" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:42990 10.133.0.45:443 100.64.0.3:61836 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T20:03:18.146558Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:18.146605Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:18.147756Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:18.147790Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-27T20:03:18.145Z] "GET /maas-api/v1/models HTTP/1.1" 401 - - "-" 0 0 2 - "100.64.0.3" "python-requests/2.32.5" "60f14363-91c8-4a53-b085-6da6efefcb29" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.45:443 100.64.0.3:15996 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T20:03:34.762528Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.762579Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.767814Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.767955Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.768193Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.774135Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.774196Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:03:34.774207Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.793585Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.793631Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.800816Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.800951Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.801259Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.801295Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.802628Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.802656Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.802893Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.805067Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.805209Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.805414Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.806021Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.823610Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:34.823648Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:34.830460Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:34.830582Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:34.830876Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:34.830908Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:34.831486Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:34.831509Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:34.831728Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:34.835214Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:34.835358Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:34.835569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:34.836026Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:34.856084Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.856127Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.863355Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.863502Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.863797Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.863834Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.864375Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.864413Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.884092Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:34.884134Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:34.890552Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:34.890676Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:34.890951Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:34.890985Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:34.891476Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:34.891501Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:34.912370Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.912437Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.915435Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.915546Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.918511Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.918542Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.924268Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.924387Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.924503Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.927529Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.927588Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:03:34.927599Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.929290Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:34.929342Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:03:34.929353Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T20:03:34.761Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 330 12 - "100.64.0.2" "python-requests/2.32.5" "935fa28e-56f5-464f-92e7-a21b370ebdac" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:42990 10.133.0.45:443 100.64.0.2:32802 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-27T20:03:34.792Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 13 - "100.64.0.2" "python-requests/2.32.5" "a8951180-dc70-405e-8ed6-32c8c436640a" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:40696 10.133.0.45:443 100.64.0.2:4560 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-05-27T20:03:34.823Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 280 13 - "100.64.0.4" "python-requests/2.32.5" "8fe11a27-72e6-4a3d-996d-ddadfd1e27d3" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:58820 10.133.0.45:443 100.64.0.4:23868 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-05-27T20:03:34.855Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 9 - "100.64.0.4" "python-requests/2.32.5" "977f823f-8f63-45b5-a14a-cc021eb73dd9" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:40696 10.133.0.45:443 100.64.0.4:12048 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-05-27T20:03:34.883Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 8 - "100.64.0.3" "python-requests/2.32.5" "7a7a7759-9b46-4e6c-889c-7e7faff0596f" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:58820 10.133.0.45:443 100.64.0.3:30030 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-05-27T20:03:34.917Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.134.0.38" "Go-http-client/1.1" "0fa9c8be-bc43-4f7b-b258-c548ab44f077" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.39:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:58834 10.133.0.45:443 10.134.0.38:58070 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.2 [2026-05-27T20:03:34.911Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 547 18 - "100.64.0.3" "python-requests/2.32.5" "2fe005ae-98ad-4d50-9d3a-7c7e433db157" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:42990 10.133.0.45:443 100.64.0.3:41168 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-27T20:03:43.801118Z info ads ADS: new connection for node:6 2026-05-27T20:03:43.803291Z info cache read certificate from file resource=file-root:system 2026-05-27T20:03:50.498938Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:50.498995Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:50.504275Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:50.504431Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:50.504655Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:50.508514Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:50.508574Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:03:50.508585Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:50.865250Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:50.865296Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:50.866321Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:50.866359Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:03:50.884716Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:50.884756Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:50.885670Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:50.885704Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:50.904993Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:50.905032Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:50.911650Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:50.911782Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:50.912110Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:50.912146Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:50.913486Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:50.913512Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:50.913742Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:50.914745Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:03:50.914860Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:03:50.914870Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-27T20:03:50.497Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 321 10 - "100.64.0.2" "python-requests/2.32.5" "dfdaaf31-62f3-486c-ae3d-f9d572a4fc1b" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:42990 10.133.0.45:443 100.64.0.2:20290 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-27T20:03:50.864Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 2 - "100.64.0.4" "python-requests/2.32.5" "1bb1c950-d9af-46d2-9661-787533e1161e" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.133.0.45:443 100.64.0.4:39754 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-05-27T20:03:50.883Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 2 - "100.64.0.3" "python-requests/2.32.5" "43f20559-2d70-420e-9810-9a294755bcdc" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.133.0.45:443 100.64.0.3:1950 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-05-27T20:03:50.904Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 404 - - "-" 83 98 10 - "100.64.0.2" "python-requests/2.32.5" "61a940c4-51de-4a88-ac36-3f2a00b5e6dd" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.133.0.45:443 100.64.0.2:20292 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.e2e-external-model.0 2026-05-27T20:04:28.531436Z info ads ADS: "" 6 terminated with stream closed 2026-05-27T20:04:32.333677Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.333733Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.339674Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.339791Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.339984Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.355216Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.355298Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T20:04:32.355311Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.392545Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.392592Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.397455Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.397598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.400353Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.400418Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.407263Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.407446Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.407572Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.409245Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.409306Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-27T20:04:32.409316Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.410964Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.411042Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T20:04:32.411055Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.450892Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.450937Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.458095Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.458251Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.458608Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.458647Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.460062Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.460090Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.460339Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.462004Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.462156Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.462337Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.462878Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.496511Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.496555Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.503175Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.503319Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.503647Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.503681Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.504145Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.504168Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.504420Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.506096Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.506259Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.506477Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.506992Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.541472Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.541531Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.548567Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.548714Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.549013Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.549052Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.549578Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.549603Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.549841Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.551627Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.551792Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.551983Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.552464Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.574267Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.574314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.581500Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.581649Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.581951Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.581986Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.582504Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.582530Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.604965Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.605007Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.611931Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.612074Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.612345Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.612376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.612896Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.612922Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.638011Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.638052Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.645256Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.645417Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.645679Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.645710Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.646210Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.646234Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.677100Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.677142Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.684015Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.684158Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.684462Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.684495Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.684989Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.685016Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.709057Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.709098Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.716035Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.716179Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.716495Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.716529Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.717084Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.717110Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.743386Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.743460Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.750297Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.750462Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.750770Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.750805Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.751329Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.751354Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.775103Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.775146Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.782382Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.782545Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.782847Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.782880Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.783449Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.783474Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.804621Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.804661Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.811308Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.811470Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.811757Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.811789Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.812260Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.812284Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-27T20:04:32.834746Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.834788Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.835716Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.835750Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.857173Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.857219Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.862377Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.862534Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.866788Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-27T20:04:32.866868Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-27T20:04:32.866879Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-27T20:04:32.332Z] "POST /v1/api-keys HTTP/2" 201 - via_upstream - "-" 55 320 22 - "100.64.0.2" "curl/7.76.1" "dd9373d6-594f-4d85-995d-7762787d791a" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:40692 10.133.0.45:443 100.64.0.2:62876 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-27T20:04:32.399Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.134.0.38" "Go-http-client/1.1" "411ad63e-bcfc-4745-a5f2-ff8f337b711b" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:49704 10.133.0.45:443 10.134.0.38:58070 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-27T20:04:32.391Z] "GET /v1/models HTTP/2" 200 - via_upstream - "-" 0 590 19 - "100.64.0.4" "curl/7.76.1" "ca0366d4-baeb-4b8a-af0f-43741cc59c48" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:40692 10.133.0.45:443 100.64.0.4:20150 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-27T20:04:32.450Z] "POST /v1/chat/completions HTTP/2" 200 - via_upstream - "-" 100 490 12 - "100.64.0.3" "curl/7.76.1" "f5ca85f7-5b31-44f0-9045-353b9f613dde" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:53264 10.133.0.45:443 100.64.0.3:17552 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T20:04:32.495Z] "POST /v1/chat/completions HTTP/2" 200 - via_upstream - "-" 100 347 11 - "100.64.0.2" "curl/7.76.1" "c6162b05-03ce-4f36-a85b-3a2ceff9b90d" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:53264 10.133.0.45:443 100.64.0.2:40286 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T20:04:32.540Z] "POST /v1/chat/completions HTTP/2" 200 - via_upstream - "-" 100 449 11 - "100.64.0.4" "curl/7.76.1" "0f5dfbc6-feb8-4fa8-95e1-09b36b1482ce" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:49704 10.133.0.45:443 100.64.0.4:10802 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T20:04:32.573Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 9 - "100.64.0.3" "curl/7.76.1" "2b5d8d6e-a70e-470a-ab36-0801cbcbf11a" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:53264 10.133.0.45:443 100.64.0.3:42328 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T20:04:32.604Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 8 - "100.64.0.2" "curl/7.76.1" "484021f9-3672-4fb3-aeb7-2cb579779fcc" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:49704 10.133.0.45:443 100.64.0.2:62886 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T20:04:32.637Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 9 - "100.64.0.4" "curl/7.76.1" "844a351f-e47a-4458-8e0e-ce9f17d27f44" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.45:443 100.64.0.4:10806 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T20:04:32.676Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 8 - "100.64.0.3" "curl/7.76.1" "9b98dae0-abd2-47dc-b6bc-c3d514a92eee" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:34296 10.133.0.45:443 100.64.0.3:42334 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T20:04:32.708Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 9 - "100.64.0.4" "curl/7.76.1" "c9ad945e-48ae-410e-acd4-87034dfc3e0c" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.45:443 100.64.0.4:20154 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T20:04:32.742Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 8 - "100.64.0.2" "curl/7.76.1" "4493d80f-5fab-45df-939c-614d59f65d3a" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:34300 10.133.0.45:443 100.64.0.2:62898 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T20:04:32.774Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 9 - "100.64.0.4" "curl/7.76.1" "0003bfeb-baee-4006-b7c7-100b8f206084" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.45:443 100.64.0.4:10814 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T20:04:32.803Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 8 - "100.64.0.3" "curl/7.76.1" "33e2a79e-155d-4137-ac02-93ff3d79b55f" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.133.0.48:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:34316 10.133.0.45:443 100.64.0.3:42342 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T20:04:32.834Z] "POST /llm/facebook-opt-125m-simulated/v1/chat/completions HTTP/2" 401 - - "-" 100 0 1 - "100.64.0.2" "curl/7.76.1" "0827a223-b577-4f40-a67c-82e9636a960c" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.45:443 100.64.0.2:62914 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-27T20:04:32.856Z] "DELETE /v1/api-keys/1f07d55a-8e12-42a1-9da6-ec921b11fee2 HTTP/2" 200 - via_upstream - "-" 0 334 10 - "100.64.0.3" "curl/7.76.1" "60edaa56-f947-4990-a340-7fa979cddd91" "maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com" "10.134.0.38:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:40692 10.133.0.45:443 100.64.0.3:17568 maas.apps.0a1eafb3-85c4-4cc9-a8a5-594c440c0350.prod.konfluxeaas.com opendatahub.maas-api-route.1