2026-06-03T02:14:41.909659Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-03T02:14:41.909675Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-06-03T02:14:41.887Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 45 329 21 - "100.64.0.2" "python-requests/2.32.5" "aca634c3-cb95-4d5d-a4a1-1c47bc0deca6" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:53910 10.133.0.38:443 100.64.0.2:28306 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T02:14:49.949883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:14:49.949930Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:14:49.965035Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:14:49.965173Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:14:49.976346Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:14:49.976391Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:14:49.983222Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:14:49.983353Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:14:49.983478Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:14:49.986034Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:14:49.986100Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:14:49.986111Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:14:49.987512Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:14:49.987598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-03T02:14:49.987611Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-06-03T02:14:49.974Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 11 - "10.134.0.52" "Go-http-client/1.1" "77118232-a83f-4165-81e9-ab1cc89b13f8" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.133.0.41:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:48538 10.133.0.38:443 10.134.0.52:49428 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-03T02:14:49.947Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 471 39 - "100.64.0.2" "python-requests/2.32.5" "15827fc6-43db-40e8-8b1e-402a3597027d" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:53910 10.133.0.38:443 100.64.0.2:44588 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T02:15:06.432295Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:06.432345Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:06.446497Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:06.446701Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:06.446960Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:06.450633Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:06.450702Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:15:06.450713Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:06.473709Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:15:06.473754Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:15:06.483648Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:15:06.483811Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:15:06.484055Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:15:06.487637Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:15:06.487713Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-03T02:15:06.487730Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-06-03T02:15:06.430Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 296 20 - "100.64.0.4" "python-requests/2.32.5" "47db20e8-f1f5-435a-bebf-0cec99e6e95e" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:50422 10.133.0.38:443 100.64.0.4:63996 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:15:06.471Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 297 16 - "100.64.0.3" "python-requests/2.32.5" "30fd33d5-a47a-455a-9b5c-5331ef1aa473" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:53910 10.133.0.38:443 100.64.0.3:23956 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T02:15:14.530172Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:15:14.530223Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:15:14.534057Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:15:14.534195Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:15:14.537412Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:14.537474Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:14.543429Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:14.543591Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:14.543727Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:14.545398Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:14.545468Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:15:14.545480Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:14.546804Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:15:14.546880Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-03T02:15:14.546896Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:15:14.566351Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:14.566402Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:14.569460Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:14.569603Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:14.572381Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:14.572417Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:14.578299Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:14.578421Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:14.578577Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:14.580146Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:14.580206Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:15:14.580216Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:14.581275Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:14.581328Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:15:14.581338Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-06-03T02:15:14.535Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 9 - "10.134.0.52" "Go-http-client/1.1" "2f194252-f999-4c61-8e0e-33363f029174" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.133.0.41:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:48538 10.133.0.38:443 10.134.0.52:49428 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-03T02:15:14.528Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 468 18 - "100.64.0.4" "python-requests/2.32.5" "c85b5e4b-f85e-44e0-9578-db020444cb6d" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:53910 10.133.0.38:443 100.64.0.4:56034 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:15:14.570Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 9 - "10.134.0.52" "Go-http-client/1.1" "f0fbdacc-d300-4492-80f1-3d93f589f37f" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.55:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:51944 10.133.0.38:443 10.134.0.52:49428 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-03T02:15:14.564Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 345 16 - "100.64.0.2" "python-requests/2.32.5" "de14310c-f09b-48b1-ada1-1ef5e71946bd" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:50422 10.133.0.38:443 100.64.0.2:33398 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T02:15:33.222711Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:15:33.222760Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:15:33.236746Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:15:33.236890Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:15:33.240265Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:33.240308Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:33.249500Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:33.249665Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:33.251274Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:33.251347Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:15:33.251360Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:33.253599Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:33.253630Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:33.262101Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:33.262224Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:33.263921Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:33.263976Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:15:33.263987Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:33.266083Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:33.266117Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:33.274986Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:33.275120Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:33.276871Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:33.276929Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:15:33.276940Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:33.278163Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:15:33.278237Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-03T02:15:33.278249Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-06-03T02:15:33.238Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 12 - "10.134.0.52" "Go-http-client/1.1" "ce2d792a-29db-46e8-82f1-b72fe0bd7ac1" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.133.0.41:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:48538 10.133.0.38:443 10.134.0.52:49428 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-03T02:15:33.252Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 11 - "10.134.0.52" "Go-http-client/1.1" "1543367a-19b8-4da7-8e04-6740cce7670d" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.55:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:51944 10.133.0.38:443 10.134.0.52:49428 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-03T02:15:33.264Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 12 - "10.134.0.52" "Go-http-client/1.1" "6affe9fa-54e8-4993-94ec-3caacc97ac4d" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:48876 10.133.0.38:443 10.134.0.52:49428 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T02:15:33.220Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 1361 57 - "100.64.0.4" "python-requests/2.32.5" "2cb8e0c0-bf59-4e06-84f9-0c4ac4f05792" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:53910 10.133.0.38:443 100.64.0.4:64414 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T02:15:53.956756Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:53.956804Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:53.971720Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:53.971868Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:53.975267Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:53.975307Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:53.984392Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:53.984557Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:53.986239Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:53.986300Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:15:53.986311Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:53.987579Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:53.987634Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:15:53.987646Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:54.009100Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:54.009145Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:54.019399Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:54.019573Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:54.022754Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:54.022788Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:54.031321Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:54.031466Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:54.033320Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:54.033382Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:15:54.033393Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:54.034638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:15:54.034692Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:15:54.034703Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-06-03T02:15:53.973Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 12 - "10.134.0.52" "Go-http-client/1.1" "72f005b4-5c69-4e77-92fb-89991b3d1e46" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.133.0.41:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:48538 10.133.0.38:443 10.134.0.52:49428 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-03T02:15:53.954Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 470 33 - "100.64.0.3" "python-requests/2.32.5" "9271cb0c-1652-4211-8655-1ed064d080ee" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:50422 10.133.0.38:443 100.64.0.3:3368 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:15:54.021Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 12 - "10.134.0.52" "Go-http-client/1.1" "ce3bf464-0f22-463f-ba54-60233ab2fb02" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.55:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:51944 10.133.0.38:443 10.134.0.52:49428 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-03T02:15:54.007Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 347 27 - "100.64.0.2" "python-requests/2.32.5" "4dea874f-412a-4ab2-ab44-e67d608c3ee6" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:50422 10.133.0.38:443 100.64.0.2:15856 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T02:16:03.068081Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:03.068139Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:03.069801Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:03.069838Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-06-03T02:16:03.065Z] "GET /maas-api/v1/models HTTP/1.1" 401 - - "-" 0 0 4 - "100.64.0.3" "python-requests/2.32.5" "ade95cf2-7392-4aba-954a-caa6e83bebbc" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.38:443 100.64.0.3:6340 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:16:04.465Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 0 - "-" "-" "-" "-" "-" - - 10.133.0.38:443 100.64.0.4:12604 - - 2026-06-03T02:16:19.682323Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.682373Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.696872Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.697016Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.697237Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.700904Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.700976Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-03T02:16:19.700988Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.723214Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.723256Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.731513Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.731655Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.731950Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.731985Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.733177Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.733201Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.733435Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.735184Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.735326Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.735514Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.736112Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.756059Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.756102Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.762756Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.762878Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.763154Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.763186Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.763711Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.763737Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.788261Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.788308Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.794995Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.795127Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.795420Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.795456Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.796002Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.796030Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:19.818255Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:19.818311Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:19.821600Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:19.821753Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:19.825659Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:19.825705Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:19.832030Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:19.832180Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:19.832313Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:19.833939Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:19.834020Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:16:19.834036Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:19.835805Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:19.835879Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:16:19.835895Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-06-03T02:16:19.680Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 330 20 - "100.64.0.3" "python-requests/2.32.5" "24097e73-e8a9-4eff-a787-6bf03f697802" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:53910 10.133.0.38:443 100.64.0.3:30164 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:16:19.721Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 284 14 - "100.64.0.2" "python-requests/2.32.5" "1c3526f9-936c-4446-9c6f-ad3c80e9d7bd" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.133.0.42:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:55964 10.133.0.38:443 100.64.0.2:52734 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-03T02:16:19.754Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 9 - "100.64.0.2" "python-requests/2.32.5" "e13a4c90-5e0e-46f2-a6ac-43a83b993f06" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.133.0.42:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:55964 10.133.0.38:443 100.64.0.2:31176 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-03T02:16:19.786Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 9 - "100.64.0.4" "python-requests/2.32.5" "3385a510-ae67-4bc5-ae16-e00c88c710ce" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.133.0.42:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.38:443 100.64.0.4:12840 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-03T02:16:19.824Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.134.0.52" "Go-http-client/1.1" "75323b9b-3a52-4ebd-a2a4-b806f05f526e" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.133.0.42:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:34822 10.133.0.38:443 10.134.0.52:49428 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T02:16:19.816Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 547 19 - "100.64.0.3" "python-requests/2.32.5" "51bc9db7-9d7c-4b8d-a939-e59559c29e32" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:50422 10.133.0.38:443 100.64.0.3:30168 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T02:16:28.681407Z info ads ADS: new connection for node:6 2026-06-03T02:16:28.683889Z info cache read certificate from file resource=file-root:system 2026-06-03T02:16:37.572846Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:37.572893Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:37.586709Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:37.586844Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:37.587053Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:37.590573Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:37.590631Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:16:37.590642Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:37.954864Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:37.954910Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:37.956038Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:37.956077Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:37.977002Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:37.977050Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:37.978013Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:37.978051Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:16:38.000300Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:38.000354Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:38.007105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:38.007253Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:38.007577Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:38.007620Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:38.009092Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:38.009124Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:38.009374Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:38.010580Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:16:38.010716Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:16:38.010731Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-06-03T02:16:37.570Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 322 20 - "100.64.0.4" "python-requests/2.32.5" "abaefba6-ab2d-4e84-9de8-67291faed5c4" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:50422 10.133.0.38:443 100.64.0.4:48692 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:16:37.952Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 3 - "100.64.0.2" "python-requests/2.32.5" "755d2989-7cb8-41f0-a512-84e010e2b79f" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.133.0.38:443 100.64.0.2:56916 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-06-03T02:16:37.974Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 3 - "100.64.0.4" "python-requests/2.32.5" "569483db-bf9b-448e-a7cd-2af66a97c955" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.133.0.38:443 100.64.0.4:44256 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-06-03T02:16:37.998Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 404 - - "-" 83 98 12 - "100.64.0.3" "python-requests/2.32.5" "98be3e89-8100-4fc2-a3cf-a1dccafb454e" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.133.0.38:443 100.64.0.3:15978 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-06-03T02:17:13.687Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 27 - "-" "-" "-" "-" "-" - - 10.133.0.38:443 100.64.0.4:47564 - - [2026-06-03T02:17:13.686Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 30 - "-" "-" "-" "-" "-" - - 10.133.0.38:443 100.64.0.3:33840 - - [2026-06-03T02:17:13.687Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 30 - "-" "-" "-" "-" "-" - - 10.133.0.38:443 100.64.0.2:51618 - - [2026-06-03T02:17:13.700Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 58 - "-" "-" "-" "-" "-" - - 10.133.0.38:443 100.64.0.4:47578 - - [2026-06-03T02:17:13.692Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 67 - "-" "-" "-" "-" "-" - - 10.133.0.38:443 100.64.0.3:33850 - - [2026-06-03T02:17:13.700Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 64 - "-" "-" "-" "-" "-" - - 10.133.0.38:443 100.64.0.3:33864 - - [2026-06-03T02:17:13.700Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 64 - "-" "-" "-" "-" "-" - - 10.133.0.38:443 100.64.0.2:51626 - - 2026-06-03T02:17:15.626428Z info ads ADS: "" 6 terminated 2026-06-03T02:17:17.820440Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:17.820486Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:17.823322Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:17.823453Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:17.823721Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:17.836616Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:17.836696Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-03T02:17:17.836708Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:17.956193Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:17.956242Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:17.981871Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:17.981920Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.014722Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.014768Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.016343Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.016379Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.049108Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.049159Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.050565Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.050610Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.444510Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.444587Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.447468Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.447620Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.447819Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.451665Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.451728Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-03T02:17:18.451738Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.679589Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.679638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-06-03T02:17:17.818Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 18 - "100.64.0.2" "python-requests/2.32.5" "7a3f6c08-f7b1-4459-ab3f-40643b17d3d5" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:53910 10.133.0.38:443 100.64.0.2:39792 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:17:17.954Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 37 0 28 - "100.64.0.4" "python-requests/2.32.5" "1abadf4d-354e-4f59-b81c-41416adc5470" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.38:443 100.64.0.4:52238 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:17:18.012Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 35 0 3 - "100.64.0.3" "python-requests/2.32.5" "db06ec1e-0c38-46d5-8ede-36736daff0fc" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.38:443 100.64.0.3:13220 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:17:18.046Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 36 0 3 - "100.64.0.4" "python-requests/2.32.5" "e3edae49-26ab-4a8a-b0e4-a4114b19225d" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.38:443 100.64.0.4:39844 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:17:18.442Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 28 312 9 - "100.64.0.3" "python-requests/2.32.5" "c989df83-f512-4ed0-ad67-f10146bd2e83" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:53910 10.133.0.38:443 100.64.0.3:21964 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T02:17:18.682345Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.682466Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.682694Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.686304Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.686362Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-03T02:17:18.686373Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.706297Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.706342Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.711784Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.711921Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.715760Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.715799Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.721965Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.722113Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.722238Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.725128Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.725201Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:17:18.725212Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.727000Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.727064Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-03T02:17:18.727075Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:18.748129Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.748176Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.755316Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.755469Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.755783Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.755816Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.757240Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.757265Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.757504Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.759598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.759740Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.759930Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.760396Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.854737Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.854791Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.857423Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.857580Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.857797Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.861419Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.861498Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:17:18.861515Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.883048Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.883105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.885615Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.885760Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.889991Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:18.890072Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:17:18.890088Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-06-03T02:17:18.677Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 311 8 - "100.64.0.2" "python-requests/2.32.5" "171a752d-32ae-4f6c-9403-884459da5c15" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:53910 10.133.0.38:443 100.64.0.2:59506 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:17:18.714Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 11 - "10.134.0.52" "Go-http-client/1.1" "f4356bda-4d1a-4372-b152-86b082081889" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:48876 10.133.0.38:443 10.134.0.52:49428 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T02:17:18.704Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 22 - "100.64.0.4" "python-requests/2.32.5" "fa33d38e-bdb3-4238-a1f2-de805c493da3" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:53910 10.133.0.38:443 100.64.0.4:52250 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:17:18.746Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 123 327 13 - "100.64.0.3" "python-requests/2.32.5" "660ceb98-eea7-4308-9e17-a87079283f55" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:48876 10.133.0.38:443 100.64.0.3:13232 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-06-03T02:17:18.852Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 8 - "100.64.0.2" "python-requests/2.32.5" "9d3b7fb3-3a41-4d65-ae20-f33b4813de00" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:50422 10.133.0.38:443 100.64.0.2:59508 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:17:18.881Z] "DELETE /v1/api-keys/76bca992-95a9-41d8-8ff4-5ba8aa0280c6 HTTP/1.1" 200 - via_upstream - "-" 0 251 9 - "100.64.0.2" "python-requests/2.32.5" "d8bc4378-72c6-45ba-905a-1aaf3334ea3b" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:50422 10.133.0.38:443 100.64.0.2:39798 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T02:17:21.922191Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:21.922243Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:21.926107Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:21.926204Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:21.928234Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:21.928299Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-03T02:17:21.928309Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.072414Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.072469Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.086867Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.086914Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.255082Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.255151Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.257871Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.258018Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.258235Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.261936Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.262008Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-03T02:17:22.262019Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.285751Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.285801Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.288436Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.288584Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.288794Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.292277Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.292338Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:17:22.292354Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.385433Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.385483Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.388058Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.388179Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.388370Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.391994Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.392057Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:17:22.392067Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.413243Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.413297Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.415889Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.416011Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.420081Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.420150Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-03T02:17:22.420161Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.442397Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.442451Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.444881Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.445010Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.447577Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.447637Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-03T02:17:22.447646Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.532228Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.532278Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.535084Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.535218Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.535424Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.538954Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.539016Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-03T02:17:22.539027Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.558551Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.558596Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.561904Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.562030Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.564894Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.564931Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.571215Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.571355Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.571482Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.573425Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.573488Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:17:22.573498Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.574735Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.574790Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:17:22.574801Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.659673Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.659722Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.662814Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.663002Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.663213Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.666743Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-06-03T02:17:22.666805Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-06-03T02:17:22.666816Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-06-03T02:17:21.919Z] "GET /v1/models HTTP/1.1" 500 - via_upstream - "-" 0 96 8 - "100.64.0.4" "python-requests/2.32.5" "c3297c3a-4a71-4005-8d11-a7bc54de5d27" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:53910 10.133.0.38:443 100.64.0.4:52262 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:17:22.070Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 38 0 16 - "100.64.0.3" "python-requests/2.32.5" "b209ae01-44f4-408d-82db-083673b4a2f9" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.38:443 100.64.0.3:13242 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:17:22.252Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 34 318 9 - "100.64.0.2" "python-requests/2.32.5" "5271bf4a-3316-4064-b2da-1342d58601eb" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:53910 10.133.0.38:443 100.64.0.2:59514 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:17:22.283Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 32 317 8 - "100.64.0.4" "python-requests/2.32.5" "63c38edc-d2a0-47bd-8681-8cfcd8351049" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:50422 10.133.0.38:443 100.64.0.4:39846 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:17:22.383Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 31 316 8 - "100.64.0.3" "python-requests/2.32.5" "24389c45-2d2b-41d5-b2b5-fbb3e428dcb3" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:50422 10.133.0.38:443 100.64.0.3:21974 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:17:22.411Z] "DELETE /v1/api-keys/08c1c1eb-66c1-49ba-9518-ca92578f130b HTTP/1.1" 200 - via_upstream - "-" 0 253 8 - "100.64.0.2" "python-requests/2.32.5" "15dff168-ea53-48dc-8074-ffec916e5ff3" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:53910 10.133.0.38:443 100.64.0.2:39802 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:17:22.440Z] "DELETE /v1/api-keys/08c1c1eb-66c1-49ba-9518-ca92578f130b HTTP/1.1" 404 - via_upstream - "-" 0 29 7 - "100.64.0.4" "python-requests/2.32.5" "7fa4175d-04bd-416d-a181-b4b48f91cf0e" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:53910 10.133.0.38:443 100.64.0.4:39852 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:17:22.530Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 8 - "100.64.0.3" "python-requests/2.32.5" "98e6f82e-aaae-4bfa-b36a-15b2f0771cec" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:53910 10.133.0.38:443 100.64.0.3:21982 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:17:22.563Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 10 - "10.134.0.52" "Go-http-client/1.1" "3f3a004f-40bc-445f-8528-23b3c5e25e5c" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:48876 10.133.0.38:443 10.134.0.52:49428 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T02:17:22.556Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 18 - "100.64.0.4" "python-requests/2.32.5" "edbc787e-b9bc-4075-ba3f-febaa193074a" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:50422 10.133.0.38:443 100.64.0.4:52266 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:17:22.657Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 9 - "100.64.0.3" "python-requests/2.32.5" "075b0214-2e43-44fc-b84f-5fe3a8e64ff5" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:53910 10.133.0.38:443 100.64.0.3:13244 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T02:17:22.685882Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.685927Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.689809Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.689935Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.691813Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.691877Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:17:22.691887Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.712304Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.712350Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.715701Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.715833Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.718649Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.718687Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.725246Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.725389Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.725508Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.727467Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.727554Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:17:22.727567Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.728911Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.728969Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:17:22.728981Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.817453Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.817499Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.820086Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.820223Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.820434Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.824145Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.824216Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:17:22.824227Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.842426Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.842474Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.845770Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.845901Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.848699Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.848734Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.854704Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.854847Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.854958Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.856647Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.856710Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:17:22.856721Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.857899Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.857955Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:17:22.857967Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.876700Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.876753Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.879883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.880013Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.882951Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.882989Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.889006Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.889145Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.889265Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.891116Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.891183Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:17:22.891194Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.892358Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.892414Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:17:22.892425Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.977132Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.977178Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.979917Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.980042Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.980236Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.983745Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-06-03T02:17:22.983808Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-06-03T02:17:22.983819Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-06-03T02:17:22.683Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 7 - "100.64.0.2" "python-requests/2.32.5" "f8a37544-db5c-4885-8da0-0840ff8b6d2d" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:50422 10.133.0.38:443 100.64.0.2:59524 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:17:22.717Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 10 - "10.134.0.52" "Go-http-client/1.1" "8ae54e01-33b7-410b-a96b-94d15cbea757" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:48876 10.133.0.38:443 10.134.0.52:49428 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T02:17:22.710Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 18 - "100.64.0.4" "python-requests/2.32.5" "ab80aeb8-d8c5-46e5-9b6b-5615609de587" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:50422 10.133.0.38:443 100.64.0.4:52274 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:17:22.815Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 9 - "100.64.0.2" "python-requests/2.32.5" "85742890-733e-4b25-9e31-ee2c9d095a6f" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:50422 10.133.0.38:443 100.64.0.2:39806 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:17:22.847Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.134.0.52" "Go-http-client/1.1" "2479ee63-1dcc-4f73-8502-051d2d59cbbf" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:48876 10.133.0.38:443 10.134.0.52:49428 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T02:17:22.840Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 17 - "100.64.0.3" "python-requests/2.32.5" "3ee06250-2746-4fe1-8aa9-2d17e31a84d2" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:50422 10.133.0.38:443 100.64.0.3:13256 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:17:22.881Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.134.0.52" "Go-http-client/1.1" "d53163dd-99d8-481a-9753-4149b05a7eff" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.57:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.38:48876 10.133.0.38:443 10.134.0.52:49428 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T02:17:22.874Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 17 - "100.64.0.4" "python-requests/2.32.5" "293a68d1-632c-4ad6-ae67-ed6f49cb8d54" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:50422 10.133.0.38:443 100.64.0.4:39854 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T02:17:22.975Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 36 321 8 - "100.64.0.3" "python-requests/2.32.5" "e86d20e3-fe9f-44e7-863d-eb3edf763ada" "maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com" "10.134.0.52:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.38:50422 10.133.0.38:443 100.64.0.3:21986 maas.apps.d41abf13-c8a7-4414-93ce-034ecc5336ab.prod.konfluxeaas.com opendatahub.maas-api-route.1