2026-05-21T15:22:50.627511Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:22:50.627649Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:22:50.627884Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:22:50.631409Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:22:50.631469Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:22:50.631480Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-21T15:22:50.621Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 80 322 10 - "100.64.0.3" "python-requests/2.32.5" "df321ca1-1289-4c96-8be4-219028593dfc" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:33776 10.133.0.45:443 100.64.0.3:37896 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-21T15:22:58.661181Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:22:58.661231Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:22:58.665190Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:22:58.665314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:22:58.667436Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.667483Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.667506Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.675540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.675663Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.675817Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.675900Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.677127Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.677163Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.677175Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.678444Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.678479Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.678548Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:22:58.678566Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.678587Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:22:58.678645Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.683951Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.684149Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.684282Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.684398Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.685615Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.685638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.685676Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:22:58.685686Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.685701Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:22:58.685708Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:22:58.686833Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:22:58.686890Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:22:58.686901Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-21T15:22:58.666Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 12 - "10.134.0.30" "Go-http-client/1.1" "4eeb479c-e48f-446a-a3a4-9cc16a2b2212" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.133.0.58:8000" outbound|8000||premium-simulated-simulated-premium-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:41750 10.133.0.45:443 10.134.0.30:45894 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.premium-simulated-simulated-premium-kserve-route.2 [2026-05-21T15:22:58.676Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.134.0.30" "Go-http-client/1.1" "04d901dc-6d17-47d4-b580-d0b7ad0c7eb1" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.133.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:60438 10.133.0.45:443 10.134.0.30:43220 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-21T15:22:58.660Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 928 26 - "100.64.0.3" "python-requests/2.32.5" "01f4152a-9046-4a27-b6ff-b7771c8b9bf7" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:33776 10.133.0.45:443 100.64.0.3:48462 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-21T15:23:12.147150Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:23:12.147200Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:23:12.152826Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:23:12.152949Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:23:12.153189Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:23:12.157388Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:23:12.157447Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:23:12.157458Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-21T15:23:12.145Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 92 334 11 - "100.64.0.2" "python-requests/2.32.5" "915eed70-759d-439c-9485-9b535ea20888" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:33776 10.133.0.45:443 100.64.0.2:24014 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-21T15:23:20.187574Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:23:20.187623Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:23:20.191586Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:23:20.191714Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:23:20.193933Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:20.193971Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:20.194839Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:20.194870Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:20.201844Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:20.201967Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:20.202095Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:20.204236Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:20.204293Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:23:20.204304Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:20.223768Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:20.223884Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:20.223995Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:20.237759Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:20.237827Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:23:20.237838Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:20.240022Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:23:20.240108Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:23:20.240127Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-21T15:23:20.194Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 10 - "10.134.0.30" "Go-http-client/1.1" "31a21850-68de-4d38-8404-25261c3d6b78" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.33:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:53488 10.133.0.45:443 10.134.0.30:43220 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.2 [2026-05-21T15:23:20.193Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 44 - "10.134.0.30" "Go-http-client/1.1" "174ae067-af26-491f-9254-cbc57e352b61" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.31:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:38932 10.133.0.45:443 10.134.0.30:45894 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.2 [2026-05-21T15:23:20.186Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 813 53 - "100.64.0.4" "python-requests/2.32.5" "b119116e-9802-985f-8f57-9018dd6ad237" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:33776 10.133.0.45:443 100.64.0.4:35236 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-21T15:23:36.503436Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:23:36.503482Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:23:36.508627Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:23:36.508748Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:23:36.511118Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:36.511155Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:36.519549Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:36.519675Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:36.520964Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:36.521050Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:23:36.521062Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:36.522383Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:36.522414Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:36.529809Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:36.529917Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:36.531127Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:36.531175Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:23:36.531186Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:36.532428Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:36.532458Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:36.540583Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:36.540696Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:36.541777Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:36.541826Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:23:36.541837Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:36.542865Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:23:36.542932Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:23:36.542945Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-21T15:23:36.510Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 10 - "10.134.0.30" "Go-http-client/1.1" "12e76ce3-591a-476f-9b67-562b0b33b73d" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.33:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:53488 10.133.0.45:443 10.134.0.30:45894 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.2 [2026-05-21T15:23:36.521Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 9 - "10.134.0.30" "Go-http-client/1.1" "61b084fc-aace-47e6-a675-7dbb52f3e262" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.31:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:38932 10.133.0.45:443 10.134.0.30:45894 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.2 [2026-05-21T15:23:36.531Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 10 - "10.134.0.30" "Go-http-client/1.1" "267fe382-4248-4f78-a458-562b82fb4cea" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.133.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:60438 10.133.0.45:443 10.134.0.30:45894 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-21T15:23:36.502Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 1351 40 - "100.64.0.4" "python-requests/2.32.5" "00212ac1-cbd8-4a96-8e1f-0ff08d4a870f" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:33776 10.133.0.45:443 100.64.0.4:16308 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-21T15:23:54.119080Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:54.119131Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:54.124637Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:54.124771Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:54.128801Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:54.128838Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:54.128851Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:54.137860Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:54.137994Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:54.138102Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:54.139332Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:54.139356Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:54.139403Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:23:54.139413Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:54.139426Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:23:54.139433Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:54.140474Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:23:54.140528Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:23:54.140538Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-21T15:23:54.127Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 11 - "10.134.0.30" "Go-http-client/1.1" "1d28e9b4-cc7f-4be6-aef4-b6a5bc1fdf75" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.133.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:60438 10.133.0.45:443 10.134.0.30:45894 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-21T15:23:54.117Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 477 22 - "100.64.0.3" "python-requests/2.32.5" "a26f83f4-bd1c-4483-b508-537b6b548bb0" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:40518 10.133.0.45:443 100.64.0.3:6936 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-21T15:24:05.355375Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:05.355423Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:05.361390Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:05.361540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:05.361786Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:05.368730Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:05.368808Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:24:05.368823Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-21T15:24:05.354Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 84 326 14 - "100.64.0.2" "python-requests/2.32.5" "21921f41-fc56-4755-b74a-9a8d27adedb6" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:40518 10.133.0.45:443 100.64.0.2:58464 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-21T15:24:13.402959Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:13.403026Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:13.407061Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:13.407183Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:13.408791Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:13.408850Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:24:13.408861Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:14.015069Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:14.015116Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:14.020440Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:14.020573Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:14.020793Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:14.024229Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:14.024291Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:24:14.024302Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-21T15:24:13.401Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 27 7 - "100.64.0.4" "python-requests/2.32.5" "5b1679bd-d9cf-4b63-9c72-5d4981f7be7d" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:33776 10.133.0.45:443 100.64.0.4:39952 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-21T15:24:14.013Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 31 315 10 - "100.64.0.2" "python-requests/2.32.5" "d7f74a03-b0e8-40e9-b0fa-5f09a9215c7c" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:40518 10.133.0.45:443 100.64.0.2:6592 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-21T15:24:22.059384Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:22.059434Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:22.063771Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:22.063902Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:22.066006Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:22.066086Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:22.072509Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:22.072666Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:22.072786Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:22.074787Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:22.074862Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:24:22.074878Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:22.075973Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:22.076050Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:24:22.076063Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-21T15:24:22.065Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.134.0.30" "Go-http-client/1.1" "9e6bdc8d-fdd1-4ebf-bd60-26db4a2d2411" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.133.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:60438 10.133.0.45:443 10.134.0.30:45894 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-21T15:24:22.058Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 17 - "100.64.0.3" "python-requests/2.32.5" "1e8fba7e-41d9-49f0-a026-d0cc8cf84315" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:33776 10.133.0.45:443 100.64.0.3:20002 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-21T15:24:22.469774Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:22.469819Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:22.474976Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:22.475123Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:22.475334Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:22.478676Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:22.478734Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:24:22.478745Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-21T15:24:22.468Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 33 318 10 - "100.64.0.4" "python-requests/2.32.5" "7de1e64a-005b-4fa4-be7f-2a67f3a7387b" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:40518 10.133.0.45:443 100.64.0.4:12868 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-21T15:24:30.506079Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:30.506145Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:30.510475Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:30.510606Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:30.512703Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:30.512739Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:30.519263Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:30.519407Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:30.519510Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:30.520697Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:30.520754Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:24:30.520765Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:30.521855Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:30.521929Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:24:30.521945Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:31.279945Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:31.279995Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:31.285342Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:31.285471Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:31.285685Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:31.289103Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:31.289161Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:24:31.289171Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-21T15:24:30.511Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.134.0.30" "Go-http-client/1.1" "03266a5f-2ba9-4791-9377-a6446bd14c40" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.133.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:60438 10.133.0.45:443 10.134.0.30:45894 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-21T15:24:30.504Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 17 - "100.64.0.3" "python-requests/2.32.5" "5399de94-6f33-4829-bc19-f7144888f089" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:33776 10.133.0.45:443 100.64.0.3:21314 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-21T15:24:31.278Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 321 10 - "100.64.0.2" "python-requests/2.32.5" "ea64a071-15a6-4491-85b3-69cc754c428b" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:40518 10.133.0.45:443 100.64.0.2:24790 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-21T15:24:34.810Z] "GET / HTTP/1.1" 404 NR route_not_found - "-" 0 0 0 - "100.64.0.3" "Mozilla/5.0 (iPhone; CPU iPhone OS 13_2_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.3 Mobile/15E148 Safari/604.1" "9bb21125-8a7c-47b8-9a82-21f164cddc7e" "34.201.83.142" "-" - - 10.133.0.45:80 100.64.0.3:6076 - - 2026-05-21T15:24:39.328964Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:39.329029Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:39.333130Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:39.333240Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:39.335288Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:39.335323Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:39.335338Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:39.341148Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:39.341270Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:39.341372Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:39.341449Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:39.342830Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:39.342854Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:39.342898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:24:39.342907Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:39.342920Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:24:39.342925Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:39.343964Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:24:39.344033Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:24:39.344044Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-21T15:24:39.334Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.134.0.30" "Go-http-client/1.1" "61dc0967-676f-46f2-91f5-d8af8e2f2389" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.133.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:60438 10.133.0.45:443 10.134.0.30:45894 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-21T15:24:39.327Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 474 16 - "100.64.0.3" "python-requests/2.32.5" "7e94ac04-2c78-4d7e-a61d-17c7fedde9fb" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:33776 10.133.0.45:443 100.64.0.3:48410 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-21T15:24:48.538522Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:48.538569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:48.544966Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:48.545170Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:48.545461Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:48.550208Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:24:48.550277Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:24:48.550289Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-21T15:24:48.537Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 85 327 12 - "100.64.0.4" "python-requests/2.32.5" "9f14907e-7796-4f88-9c9a-ca7f8462540d" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:40518 10.133.0.45:443 100.64.0.4:25972 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-21T15:25:04.812042Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:25:04.812091Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:25:04.817296Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:25:04.817431Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:25:04.819076Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:25:04.819131Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:25:04.819141Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-21T15:25:04.810Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 82 8 - "100.64.0.2" "python-requests/2.32.5" "428f3d82-bd41-491b-8e4e-b5dacaebb642" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:33776 10.133.0.45:443 100.64.0.2:18760 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-21T15:25:21.882977Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:25:21.883045Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:25:21.888676Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:25:21.888799Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:25:21.890555Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:25:21.890613Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:25:21.890623Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-21T15:25:21.881Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 8 - "100.64.0.2" "python-requests/2.32.5" "12f80bbc-fb23-4e2c-8c57-dfcb434d57ca" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:33776 10.133.0.45:443 100.64.0.2:7068 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-21T15:25:39.092624Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:25:39.092673Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:25:39.098583Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:25:39.098718Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:25:39.100503Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:25:39.100559Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:25:39.100569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-21T15:25:39.091Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 82 9 - "100.64.0.4" "python-requests/2.32.5" "beea38f6-df30-495b-8361-759d8971ebd4" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:40518 10.133.0.45:443 100.64.0.4:8754 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-21T15:25:41.888Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 996 - "-" "-" "-" "-" "-" - - 10.133.0.45:443 100.64.0.3:38150 - - 2026-05-21T15:25:56.491164Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:25:56.491215Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:25:56.496478Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:25:56.496604Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:25:56.498409Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:25:56.498463Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:25:56.498474Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-21T15:25:56.490Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 8 - "100.64.0.2" "python-requests/2.32.5" "6210c3d1-7608-4bb5-90d9-18f89ce8112b" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:40518 10.133.0.45:443 100.64.0.2:19524 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-21T15:25:55.031Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 3000 - "-" "-" "-" "-" "-" - - 10.133.0.45:443 100.64.0.4:26048 - - 2026-05-21T15:26:08.482797Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:08.482858Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:08.487985Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:08.488159Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:08.488397Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:08.492185Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:08.492257Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:26:08.492273Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-21T15:26:08.481Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 45 329 10 - "100.64.0.3" "python-requests/2.32.5" "cc20f281-52c8-4327-8066-8eeece639e64" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:40518 10.133.0.45:443 100.64.0.3:36566 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-21T15:26:10.178Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 3000 - "-" "-" "-" "-" "-" - - 10.133.0.45:443 100.64.0.3:43748 - - 2026-05-21T15:26:16.517496Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:16.517557Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:16.530469Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:16.530618Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:16.540850Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:16.540894Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:16.547604Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:16.547732Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:16.547843Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:16.549306Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:16.549365Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:26:16.549376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:16.550485Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:16.550553Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:26:16.550570Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-21T15:26:16.539Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 9 - "10.134.0.30" "Go-http-client/1.1" "6f67dcc2-2ce2-4241-8a03-2bb45fbe51f7" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.33:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:53484 10.133.0.45:443 10.134.0.30:45392 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.2 [2026-05-21T15:26:16.516Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 471 34 - "100.64.0.4" "python-requests/2.32.5" "cb057ce0-fe8f-4f91-ae6e-098b8d8df94e" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:40518 10.133.0.45:443 100.64.0.4:41700 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-21T15:26:25.328Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 2999 - "-" "-" "-" "-" "-" - - 10.133.0.45:443 100.64.0.2:22624 - - 2026-05-21T15:26:30.800543Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:30.800591Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:30.806398Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:30.806527Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:30.806744Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:30.810831Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:30.810890Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:26:30.810901Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:30.833119Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:30.833157Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:30.838626Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:30.838741Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:30.838926Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:30.843924Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:30.843980Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:26:30.843991Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-21T15:26:30.799Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 298 11 - "100.64.0.2" "python-requests/2.32.5" "8ba5e9c3-1221-4a10-812b-051b850260e1" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:40518 10.133.0.45:443 100.64.0.2:37956 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-21T15:26:30.832Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 298 11 - "100.64.0.3" "python-requests/2.32.5" "b048fcc2-24b0-4210-86b2-3f152e6e5ba3" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:41350 10.133.0.45:443 100.64.0.3:5336 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-21T15:26:38.872156Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:38.872216Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:38.876104Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:38.876250Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:38.878613Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:38.878653Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:38.884773Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:38.884891Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:38.885001Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:38.886387Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:38.886445Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:26:38.886456Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:38.887507Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:38.887575Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:26:38.887591Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:26:38.909159Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:38.909198Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:38.912582Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:38.912685Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:38.914668Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:38.914699Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:38.920421Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:38.920540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:38.920635Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:38.924115Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:38.924170Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:26:38.924180Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:38.926025Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:38.926085Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:26:38.926097Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-21T15:26:38.877Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 8 - "10.134.0.30" "Go-http-client/1.1" "16c07e9b-f8e2-4fed-ae77-5aaa13069a50" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.33:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:53484 10.133.0.45:443 10.134.0.30:45392 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.2 [2026-05-21T15:26:38.870Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 468 16 - "100.64.0.4" "python-requests/2.32.5" "3af69bf4-5c37-4eee-85ce-4e712308d1e1" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:40518 10.133.0.45:443 100.64.0.4:10482 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-21T15:26:38.913Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 10 - "10.134.0.30" "Go-http-client/1.1" "5e366b9a-3a59-4605-993e-92934c86356c" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.31:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:33096 10.133.0.45:443 10.134.0.30:45392 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.2 [2026-05-21T15:26:38.908Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 345 17 - "100.64.0.4" "python-requests/2.32.5" "c65e19ed-036f-4dbc-a851-6803676178c7" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:41350 10.133.0.45:443 100.64.0.4:64478 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-21T15:26:46.549Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 0 - "-" "-" "-" "-" "-" - - 10.133.0.45:443 100.64.0.3:34372 - - 2026-05-21T15:26:55.601898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.601946Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.607334Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.607454Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.609659Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.609693Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.619050Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.619178Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.620621Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.620680Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:26:55.620691Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.622242Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.622272Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.634894Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.635090Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.636940Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.637039Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:26:55.637058Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.638991Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.639054Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.648826Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.648968Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.651891Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.651957Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:26:55.651974Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.653145Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:26:55.653213Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:26:55.653231Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-21T15:26:55.608Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 11 - "10.134.0.30" "Go-http-client/1.1" "816f5602-9da3-403b-a140-d5941433bfe5" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.33:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:53484 10.133.0.45:443 10.134.0.30:45392 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.2 [2026-05-21T15:26:55.621Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 15 - "10.134.0.30" "Go-http-client/1.1" "70ea1d5f-5396-47da-b13a-a71e7517f8fe" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.31:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:33096 10.133.0.45:443 10.134.0.30:45392 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.2 [2026-05-21T15:26:55.637Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 14 - "10.134.0.30" "Go-http-client/1.1" "b321e3e1-1cf8-4aee-bcc4-a706e71132b9" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.133.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:40832 10.133.0.45:443 10.134.0.30:45392 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-21T15:26:55.600Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 1361 52 - "100.64.0.4" "python-requests/2.32.5" "f3632a46-c115-4df2-b0a1-7c77ced3c1b2" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:41350 10.133.0.45:443 100.64.0.4:6706 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-21T15:26:58.702Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 2996 - "-" "-" "-" "-" "-" - - 10.133.0.45:443 100.64.0.2:1762 - - 2026-05-21T15:27:12.225421Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:12.225475Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:12.424428Z error envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1143 wasm log openshift-ingress.kuadrant-maas-default-gateway: gRPC status code is not OK thread=18 2026-05-21T15:27:12.424459Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:12.424487Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-21T15:27:12.224Z] "GET /v1/models HTTP/1.1" 500 - - "-" 0 23 200 - "100.64.0.3" "python-requests/2.32.5" "d310dcb8-9b6e-4d2e-a322-366e960fdc43" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:41350 10.133.0.45:443 100.64.0.3:29244 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-21T15:27:13.845Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 2999 - "-" "-" "-" "-" "-" - - 10.133.0.45:443 100.64.0.4:51128 - - 2026-05-21T15:27:21.548747Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:21.548796Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:21.549989Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:21.550051Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-21T15:27:21.547Z] "GET /maas-api/v1/models HTTP/1.1" 401 - - "-" 0 0 2 - "100.64.0.2" "python-requests/2.32.5" "ee3d9934-2a3f-46da-b32e-23ad1669cac2" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.45:443 100.64.0.2:26414 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-21T15:27:29.047Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 2997 - "-" "-" "-" "-" "-" - - 10.133.0.45:443 100.64.0.3:62810 - - 2026-05-21T15:27:38.162905Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.162953Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.169514Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.169684Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.169959Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.176180Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.176243Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:27:38.176254Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.197177Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.197215Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.205077Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.205258Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.205630Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.205674Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.206946Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.206976Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.207277Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.209120Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.209308Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.209560Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.210217Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.231805Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.231864Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.239236Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.239349Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.239614Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.239644Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.240188Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.240217Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.261192Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.261224Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.269885Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.270034Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.270321Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.270355Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.270936Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.270961Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.293320Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.293358Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.297164Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.297277Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.299941Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:38.299978Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:38.323580Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:38.323727Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:38.323865Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:38.325737Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:38.325818Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:27:38.325836Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:38.327813Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:38.327875Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:27:38.327887Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-21T15:27:38.161Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 331 14 - "100.64.0.3" "python-requests/2.32.5" "a917ba43-7d22-4ab9-b89b-3751cbf1450e" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:40518 10.133.0.45:443 100.64.0.3:46214 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-21T15:27:38.196Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 281 13 - "100.64.0.2" "python-requests/2.32.5" "74e7ddde-67b1-40c3-a8b6-71ed7198f65c" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.132.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:51258 10.133.0.45:443 100.64.0.2:53210 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-05-21T15:27:38.231Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 9 - "100.64.0.4" "python-requests/2.32.5" "2b141ca5-a355-47d0-a1ae-e23ae9a62932" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.132.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:51258 10.133.0.45:443 100.64.0.4:11852 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-05-21T15:27:38.260Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 10 - "100.64.0.2" "python-requests/2.32.5" "ef38003e-248a-4f33-afe8-6f3090740e64" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.132.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.45:443 100.64.0.2:34798 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-05-21T15:27:38.299Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 26 - "10.134.0.30" "Go-http-client/1.1" "8d53d818-9f8b-47c0-9494-f937ca4401b5" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.132.0.26:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.45:51268 10.133.0.45:443 10.134.0.30:45392 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.2 [2026-05-21T15:27:38.292Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 547 35 - "100.64.0.4" "python-requests/2.32.5" "c7f543e0-e829-4a68-9696-edb5613ea643" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:40518 10.133.0.45:443 100.64.0.4:30966 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-21T15:27:47.192431Z info ads ADS: new connection for node:6 2026-05-21T15:27:47.194911Z info cache read certificate from file resource=file-root:system [2026-05-21T15:27:44.193Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 2998 - "-" "-" "-" "-" "-" - - 10.133.0.45:443 100.64.0.3:31634 - - 2026-05-21T15:27:53.984796Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:53.984849Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:53.990412Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:53.990549Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:53.990779Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:53.995766Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:53.995826Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-21T15:27:53.995837Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:54.386100Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:54.386144Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:54.387236Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:54.387276Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:54.408239Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:54.408278Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:54.409257Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:54.409303Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-21T15:27:54.433808Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:54.433850Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:54.440470Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:54.440598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:54.440869Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:54.440901Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:54.442439Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:54.442464Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:54.442689Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:54.443562Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-21T15:27:54.443681Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-21T15:27:54.443693Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-21T15:27:53.983Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 322 12 - "100.64.0.3" "python-requests/2.32.5" "9642c292-11f6-4289-9d5a-012e990ed4e1" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "10.134.0.30:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.45:49262 10.133.0.45:443 100.64.0.3:20830 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-21T15:27:54.385Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 2 - "100.64.0.2" "python-requests/2.32.5" "157c11ea-ad2f-4758-ad7d-bfe127007463" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.133.0.45:443 100.64.0.2:16910 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-05-21T15:27:54.407Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 2 - "100.64.0.4" "python-requests/2.32.5" "f68f9a06-86d5-4867-911d-80d5bd3143ff" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.133.0.45:443 100.64.0.4:4066 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-05-21T15:27:54.432Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 404 - - "-" 83 98 11 - "100.64.0.2" "python-requests/2.32.5" "1dce80af-847b-492b-be98-59c0c9ec1fc3" "maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.133.0.45:443 100.64.0.2:33090 maas.apps.e6e4c48a-8289-4af5-bc74-b43ad44a1d02.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-05-21T15:28:05.267Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 2997 - "-" "-" "-" "-" "-" - - 10.133.0.45:443 100.64.0.3:64774 - - [2026-05-21T15:28:20.414Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 2997 - "-" "-" "-" "-" "-" - - 10.133.0.45:443 100.64.0.4:54258 - - 2026-05-21T15:28:32.101133Z info ads ADS: "" 6 terminated with stream closed [2026-05-21T15:28:35.559Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 2998 - "-" "-" "-" "-" "-" - - 10.133.0.45:443 100.64.0.2:49222 - - [2026-05-21T15:28:38.704Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 0 - "-" "-" "-" "-" "-" - - 10.133.0.45:443 100.64.0.3:24406 - -