[2026-06-03T20:15:02.666Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 20 - "100.64.0.3" "python-requests/2.32.5" "2d9cdceb-c558-402f-8275-a8210595037f" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:42506 10.132.0.41:443 100.64.0.3:54200 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:15:16.867921Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:15:16.867971Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:15:16.884578Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:15:16.884745Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:15:16.884995Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:15:16.889726Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:15:16.889791Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:15:16.889802Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T20:15:16.866Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 45 330 23 - "100.64.0.4" "python-requests/2.32.5" "a09fe239-02c4-438b-b43f-76e927af3de6" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:43476 10.132.0.41:443 100.64.0.4:18156 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:15:24.923511Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:15:24.923566Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:15:24.939293Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:15:24.939437Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:15:24.950257Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:24.950303Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:24.957788Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:24.957921Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:24.958044Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:24.960847Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:24.960920Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:15:24.960932Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:24.962321Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:15:24.962383Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:15:24.962395Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T20:15:24.949Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 11 - "10.134.0.39" "Go-http-client/1.1" "bc8c38e2-42be-4b4c-a272-d1560967f738" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.43:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.41:33988 10.132.0.41:443 10.134.0.39:35952 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-03T20:15:24.922Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 471 40 - "100.64.0.2" "python-requests/2.32.5" "7d51e09c-dee4-41b0-83d7-0a7080eb3b37" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:43476 10.132.0.41:443 100.64.0.2:40946 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:15:39.332587Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:15:39.332647Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:15:39.348704Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:15:39.348882Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:15:39.349150Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:15:39.354114Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:15:39.354195Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:15:39.354211Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:15:39.376494Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:39.376551Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:39.386880Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:39.387033Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:39.387290Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:39.391108Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:39.391177Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:15:39.391188Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T20:15:39.331Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 298 22 - "100.64.0.4" "python-requests/2.32.5" "c404b9f8-932f-41f9-9081-91cd0b07ef42" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:43476 10.132.0.41:443 100.64.0.4:51578 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:15:39.375Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 297 15 - "100.64.0.2" "python-requests/2.32.5" "5e672d35-3590-9601-a218-553729ab98c1" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:42506 10.132.0.41:443 100.64.0.2:26188 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:15:47.428486Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.428538Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.433213Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.433339Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.435700Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.435734Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.442746Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.442860Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.442963Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.444975Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.445038Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:15:47.445048Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.446352Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.446405Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:15:47.446416Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.466332Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.466377Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.470300Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.470420Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.472625Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.472663Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.479412Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.479530Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.479637Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.482757Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.482823Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:15:47.482833Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.484853Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:15:47.484907Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:15:47.484918Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T20:15:47.434Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 10 - "10.134.0.39" "Go-http-client/1.1" "9975d903-0869-4aa5-a81c-3416eacc3bd6" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.43:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.41:33988 10.132.0.41:443 10.134.0.39:35952 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-03T20:15:47.427Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 468 19 - "100.64.0.3" "python-requests/2.32.5" "03fe06ec-d9c3-4d09-a243-38aa209ec4d0" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:42506 10.132.0.41:443 100.64.0.3:63066 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:15:47.471Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 11 - "10.134.0.39" "Go-http-client/1.1" "d4d9896b-ebb5-48ce-9e65-6f28fe65396c" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.132.0.43:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.41:45364 10.132.0.41:443 10.134.0.39:35952 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-03T20:15:47.465Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 345 19 - "100.64.0.4" "python-requests/2.32.5" "508cff93-d31e-4ea5-83c0-4f5940f4dfba" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:42506 10.132.0.41:443 100.64.0.4:59288 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:16:06.406717Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:06.406770Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:06.424331Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:06.424484Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:06.427237Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:06.427280Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:06.436293Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:06.436454Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:06.438382Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:06.438451Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:16:06.438462Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:06.440032Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:06.440067Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:06.448659Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:06.448849Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:06.450507Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:06.450573Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:16:06.450584Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:06.452284Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:06.452328Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:06.461561Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:06.461814Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:06.468402Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:06.468510Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:16:06.468531Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:06.470201Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:06.470295Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:16:06.470313Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T20:16:06.426Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 12 - "10.134.0.39" "Go-http-client/1.1" "8d474501-3a6c-4543-bfaf-b0fbf3e17359" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.43:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.41:33988 10.132.0.41:443 10.134.0.39:35952 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-03T20:16:06.439Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 11 - "10.134.0.39" "Go-http-client/1.1" "8fce74a4-73f1-4e60-9bcc-7de6a4c419e9" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.132.0.43:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.41:45364 10.132.0.41:443 10.134.0.39:35952 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-03T20:16:06.451Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 17 - "10.134.0.39" "Go-http-client/1.1" "3e5af89e-e0ec-48d0-8ab7-6e2869d5c9da" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.44:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.41:54724 10.132.0.41:443 10.134.0.39:35952 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T20:16:06.405Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 1361 64 - "100.64.0.3" "python-requests/2.32.5" "b98ec3d8-fa0d-43b4-9217-62e45c716756" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:43476 10.132.0.41:443 100.64.0.3:37464 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:16:25.055634Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:25.055706Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:25.070540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:25.070705Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:25.073259Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:25.073303Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:25.082665Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:25.082814Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:25.084734Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:25.084799Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:16:25.084811Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:25.086233Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:25.086303Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:16:25.086320Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:25.106108Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:25.106152Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:25.117163Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:25.117283Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:25.119559Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:25.119593Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:25.128036Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:25.128148Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:25.129598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:25.129654Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:16:25.129665Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:25.130936Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:25.130996Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:16:25.131010Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T20:16:25.072Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 12 - "10.134.0.39" "Go-http-client/1.1" "8f14c504-1747-4b6e-a858-6c3c0f844cc7" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.43:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.41:33988 10.132.0.41:443 10.134.0.39:35952 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-06-03T20:16:25.054Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 470 31 - "100.64.0.2" "python-requests/2.32.5" "b85c8497-a0d2-4ee6-94a6-4bbb1d494409" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:43476 10.132.0.41:443 100.64.0.2:25658 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:16:25.118Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 10 - "10.134.0.39" "Go-http-client/1.1" "e4f21e48-0c06-43b8-9caf-51bb3a95c4da" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.132.0.43:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.41:45364 10.132.0.41:443 10.134.0.39:35952 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-06-03T20:16:25.105Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 347 25 - "100.64.0.2" "python-requests/2.32.5" "dd2d215a-a8e0-4f82-aaba-14fa4cae9600" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:43476 10.132.0.41:443 100.64.0.2:8424 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:16:34.140780Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:34.140836Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:34.142354Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:34.142388Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T20:16:34.139Z] "GET /maas-api/v1/models HTTP/1.1" 401 - - "-" 0 0 2 - "100.64.0.4" "python-requests/2.32.5" "3aa686fa-be85-4cd9-91a3-f383b38de27e" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.132.0.41:443 100.64.0.4:52656 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:16:50.782535Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.782586Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.798611Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.798792Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.799117Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.803762Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.803843Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:16:50.803857Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.825444Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.825489Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.833571Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.833714Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.834009Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.834045Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.835340Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.835372Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.835614Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.837374Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.837504Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.837696Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.838175Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.861013Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.861061Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.868131Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.868250Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.868518Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.868555Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.868966Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.868992Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:16:50.890431Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:50.890474Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:50.897791Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:50.897902Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:50.898148Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:50.898181Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:50.898561Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:50.898587Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:50.920957Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:50.920998Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:50.924549Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:50.924654Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:50.929258Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:50.929292Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:50.939825Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:50.939942Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:50.940047Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:50.943020Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:50.943082Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:16:50.943093Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:50.944851Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:16:50.944906Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:16:50.944917Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T20:16:50.781Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 331 22 - "100.64.0.3" "python-requests/2.32.5" "345c93b4-549f-4882-9630-682d4540dc6e" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:43476 10.132.0.41:443 100.64.0.3:17270 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:16:50.824Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 289 13 - "100.64.0.2" "python-requests/2.32.5" "2edcc24d-8fb0-444f-8016-2bd00a995b3f" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.41:34638 10.132.0.41:443 100.64.0.2:30058 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-03T20:16:50.860Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 8 - "100.64.0.4" "python-requests/2.32.5" "279fcd26-a31b-4b54-9771-1a3037678e30" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.41:34638 10.132.0.41:443 100.64.0.4:20322 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-03T20:16:50.889Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 8 - "100.64.0.3" "python-requests/2.32.5" "6e025cf6-37e6-41b6-b46a-75b86f2e7a60" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.41:44858 10.132.0.41:443 100.64.0.3:17286 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-06-03T20:16:50.928Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 14 - "10.134.0.39" "Go-http-client/1.1" "9fe488bc-4c16-4f1f-91eb-1822f197b9cc" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.132.0.45:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.41:43644 10.132.0.41:443 10.134.0.39:35952 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T20:16:50.919Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 547 25 - "100.64.0.3" "python-requests/2.32.5" "ab3f456b-4eaa-4f56-885b-508f26183d7f" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:55230 10.132.0.41:443 100.64.0.3:46052 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:16:59.797944Z info ads ADS: new connection for node:6 2026-06-03T20:16:59.800201Z info cache read certificate from file resource=file-root:system 2026-06-03T20:17:08.710527Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:08.710585Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:08.726418Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:08.726593Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:08.726902Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:08.730926Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:08.731003Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:17:08.731019Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:09.089038Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:09.089085Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:09.090340Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:09.090379Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:09.119022Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:09.119065Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:09.120245Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:09.120277Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:09.147448Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:09.147492Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:09.154039Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:09.154157Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:09.154429Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:09.154462Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:09.155703Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:09.155731Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:09.155980Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:09.157013Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:09.157142Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:17:09.157155Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T20:17:08.709Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 322 21 - "100.64.0.4" "python-requests/2.32.5" "47fc59da-8561-480c-b12a-b55483955395" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:55230 10.132.0.41:443 100.64.0.4:55328 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:09.087Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 2 - "100.64.0.2" "python-requests/2.32.5" "05207198-95fb-470f-8445-f44d93e99d31" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.132.0.41:443 100.64.0.2:21882 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-06-03T20:17:09.118Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 2 - "100.64.0.4" "python-requests/2.32.5" "89a94fc4-9a4b-4d7e-adf7-928152e8b120" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.132.0.41:443 100.64.0.4:32732 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-06-03T20:17:09.146Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 404 - - "-" 83 98 10 - "100.64.0.3" "python-requests/2.32.5" "80ac9f4e-f7e5-45f7-84c1-b4872b300199" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.132.0.41:443 100.64.0.3:58864 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com llm.e2e-external-model.0 2026-06-03T20:17:47.021785Z info ads ADS: "" 6 terminated with stream closed 2026-06-03T20:17:49.783247Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:49.783298Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:49.786361Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:49.786476Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:49.786671Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:49.802269Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:49.802330Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:17:49.802340Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:49.915809Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:49.915859Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:49.945202Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:49.945245Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:49.970998Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:49.971059Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:49.972745Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:49.972783Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:49.996912Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:49.996959Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:49.998450Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:49.998483Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:50.434591Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.434642Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.437723Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.437843Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.438050Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.442098Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.442157Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:17:50.442168Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T20:17:49.782Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 20 - "100.64.0.2" "python-requests/2.32.5" "a8ae782c-aa9c-4487-9931-4a8b4affa4b9" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:43476 10.132.0.41:443 100.64.0.2:11844 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:49.914Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 37 0 30 - "100.64.0.3" "python-requests/2.32.5" "c85015ab-5be8-495f-8278-187744152ccc" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.132.0.41:443 100.64.0.3:45856 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:49.969Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 35 0 3 - "100.64.0.2" "python-requests/2.32.5" "1762f3ac-653c-471b-b2c0-28bba597d099" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.132.0.41:443 100.64.0.2:29308 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:49.995Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 36 0 2 - "100.64.0.4" "python-requests/2.32.5" "aad1bce6-abcb-472a-b870-68ed1691c7ca" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.132.0.41:443 100.64.0.4:7216 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:50.433Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 28 312 8 - "100.64.0.4" "python-requests/2.32.5" "ef4c81dc-2e94-43a6-9603-a336fa9f3bbf" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:43476 10.132.0.41:443 100.64.0.4:30772 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:17:50.680316Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:50.680366Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:50.683404Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:50.683520Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:50.683747Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:50.687461Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:50.687520Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:17:50.687531Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:50.726564Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:50.726606Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:50.732807Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:50.732924Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:50.735726Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:50.735766Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:50.742585Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:50.742750Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:50.742856Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:50.745934Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:50.745993Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:17:50.746004Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:50.747859Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:50.747913Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:17:50.747924Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:50.765585Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.765622Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.772621Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.772785Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.773033Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.773069Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.774352Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.774382Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.774587Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.776435Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.776569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.776766Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.777097Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.862402Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.862451Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.865920Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.866041Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.866247Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.869940Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.869999Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:17:50.870009Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.891097Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.891136Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.894164Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.894273Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.899075Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:50.899134Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:17:50.899145Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T20:17:50.679Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 8 - "100.64.0.3" "python-requests/2.32.5" "b61d7d2d-9283-48d3-9bf8-44adb7ca3dbb" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:55230 10.132.0.41:443 100.64.0.3:18312 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:50.734Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 11 - "10.134.0.39" "Go-http-client/1.1" "04473229-a96b-4a10-a601-eecd83a7ee02" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.44:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.41:54724 10.132.0.41:443 10.134.0.39:35952 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T20:17:50.725Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 22 - "100.64.0.2" "python-requests/2.32.5" "212f3467-d2b1-4d62-9e2c-01e79fc647f0" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:55230 10.132.0.41:443 100.64.0.2:29320 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:50.765Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 123 337 12 - "100.64.0.4" "python-requests/2.32.5" "a62f174e-1f93-45e6-9515-fd84a9cc09e0" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.44:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.41:55504 10.132.0.41:443 100.64.0.4:7226 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-06-03T20:17:50.861Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 8 - "100.64.0.2" "python-requests/2.32.5" "e8303cb2-a319-40e2-8a3b-9ddbf5577cdf" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:43476 10.132.0.41:443 100.64.0.2:11856 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:50.890Z] "DELETE /v1/api-keys/392e2e7a-ce93-4f27-a8ed-47926f17866e HTTP/1.1" 200 - via_upstream - "-" 0 251 9 - "100.64.0.3" "python-requests/2.32.5" "4d1d77bc-b1ab-4722-a414-d9c65b240fe8" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:43476 10.132.0.41:443 100.64.0.3:45872 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:17:53.932731Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:53.932788Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:53.936957Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:53.937053Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:53.939003Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:53.939085Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:17:53.939101Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.065199Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.065252Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.081991Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.082036Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.253365Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.253414Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.256339Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.256449Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.256642Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.260221Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.260284Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:17:54.260295Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.280772Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.280820Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.283807Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.283932Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.284127Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.287807Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.287874Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:17:54.287890Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.396339Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.396396Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.399359Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.399510Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.399744Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.403380Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.403439Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:17:54.403450Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.437719Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.437761Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.440393Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.440498Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.444692Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.444749Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:17:54.444760Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.468200Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.468234Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.471168Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.471260Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.473868Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.473919Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:17:54.473929Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 [2026-06-03T20:17:53.931Z] "GET /v1/models HTTP/1.1" 500 - via_upstream - "-" 0 96 7 - "100.64.0.4" "python-requests/2.32.5" "9bde421d-1487-451f-9868-a75f40600ec2" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:55230 10.132.0.41:443 100.64.0.4:15586 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:54.064Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 38 0 18 - "100.64.0.3" "python-requests/2.32.5" "0e4f25c5-24b1-44b3-a629-86cff425c2bc" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.132.0.41:443 100.64.0.3:42462 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:54.252Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 34 318 8 - "100.64.0.2" "python-requests/2.32.5" "bc5b57ad-f169-40c7-906d-55dee92fdf73" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:43476 10.132.0.41:443 100.64.0.2:64472 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:54.279Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 32 317 8 - "100.64.0.3" "python-requests/2.32.5" "6014c055-7c62-49a9-903e-67ae001c60e5" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:55230 10.132.0.41:443 100.64.0.3:8188 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:54.395Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 31 316 8 - "100.64.0.2" "python-requests/2.32.5" "dc78a620-cbce-4287-b6a3-ebfdcbfd0afb" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:55230 10.132.0.41:443 100.64.0.2:39792 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:54.436Z] "DELETE /v1/api-keys/7d7b8d20-6f56-4364-9ce7-d1c45a26451d HTTP/1.1" 200 - via_upstream - "-" 0 253 8 - "100.64.0.4" "python-requests/2.32.5" "88aa0876-baf1-4573-9806-5b9136576c52" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:43476 10.132.0.41:443 100.64.0.4:18130 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:54.467Z] "DELETE /v1/api-keys/7d7b8d20-6f56-4364-9ce7-d1c45a26451d HTTP/1.1" 404 - via_upstream - "-" 0 29 6 - "100.64.0.3" "python-requests/2.32.5" "7ae8ccb8-a1c5-4477-aaea-9d6eb39bbbe5" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:43476 10.132.0.41:443 100.64.0.3:42478 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-06-03T20:17:54.575871Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.575933Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.578815Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.578943Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.579148Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.582734Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.582798Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:17:54.582813Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.607396Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.607441Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.611311Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.611444Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.613696Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.613731Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.620586Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.620740Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.620844Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.622626Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.622710Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:17:54.622722Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.624009Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.624075Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:17:54.624091Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.724614Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.724697Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.727813Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.727950Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.728177Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.731882Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.731955Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:17:54.731971Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.756784Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.756831Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.760656Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.760796Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.762668Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.762752Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:17:54.762768Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.788822Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.788869Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.792459Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.792581Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.794803Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.794837Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.801238Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.801372Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.801484Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.803455Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.803516Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:17:54.803527Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.804915Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.804982Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:17:54.804997Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.943406Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.943458Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.946396Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.946506Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.946716Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.950475Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.950532Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:17:54.950543Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.973309Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.973359Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.977199Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.977331Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.979479Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.979517Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.985896Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.986025Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.986119Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.987876Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.987936Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:17:54.987947Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:54.989248Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:54.989313Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=25 2026-06-03T20:17:54.989329Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=25 2026-06-03T20:17:55.019866Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:55.019914Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:55.023893Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:55.024017Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:55.026402Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:55.026436Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:55.032636Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:55.032791Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:55.032892Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:55.034765Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:55.034824Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:17:55.034834Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:55.036079Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:55.036131Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:17:55.036142Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:55.141587Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:55.141637Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:55.144530Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:55.144638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:55.144849Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:55.148614Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 2026-06-03T20:17:55.148672Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=24 2026-06-03T20:17:55.148712Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1152 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=24 [2026-06-03T20:17:54.574Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 8 - "100.64.0.4" "python-requests/2.32.5" "7953b0ba-2cca-45a8-b2d8-d86de0bfa7c6" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:43476 10.132.0.41:443 100.64.0.4:15588 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:54.612Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.134.0.39" "Go-http-client/1.1" "cf95d3b1-b3dc-4fc6-9a72-1f4bc2f9aaa4" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.44:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.41:54724 10.132.0.41:443 10.134.0.39:35952 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T20:17:54.606Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 17 - "100.64.0.2" "python-requests/2.32.5" "9004e3ee-dae7-4fdc-9384-b9cf6cc88d2c" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:43476 10.132.0.41:443 100.64.0.2:64478 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:54.723Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 8 - "100.64.0.2" "python-requests/2.32.5" "3d36f161-2610-4805-aead-e14f4e272bf6" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:43476 10.132.0.41:443 100.64.0.2:39806 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:54.755Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 7 - "100.64.0.4" "python-requests/2.32.5" "8ae15c1a-1b55-4233-aa3b-dfaad50a63b3" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:43476 10.132.0.41:443 100.64.0.4:18142 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:54.793Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.134.0.39" "Go-http-client/1.1" "72cacd83-5338-40fb-ba4c-afb14d689328" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.44:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.41:54724 10.132.0.41:443 10.134.0.39:35952 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T20:17:54.787Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 17 - "100.64.0.3" "python-requests/2.32.5" "47a2704c-763b-4611-89ac-f9cd320c7e53" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:43476 10.132.0.41:443 100.64.0.3:8194 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:54.812Z] "POST /GponForm/diag_Form?images/ HTTP/1.1" 400 DC downstream_remote_disconnect - "-" 0 0 0 - "-" "terrabot-owned-you" "-" "-" "-" - - 10.132.0.41:80 100.64.0.2:37726 - - [2026-06-03T20:17:54.942Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 8 - "100.64.0.3" "python-requests/2.32.5" "633bfeaf-617c-452f-8532-31a027368143" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:55230 10.132.0.41:443 100.64.0.3:42484 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:54.978Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.134.0.39" "Go-http-client/1.1" "62f6f8ea-ff4b-4009-9848-832ee03fc3f1" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.44:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.41:54724 10.132.0.41:443 10.134.0.39:35952 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T20:17:54.972Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 17 - "100.64.0.4" "python-requests/2.32.5" "e91a88db-54cd-4a53-a809-b7bd954dfe7a" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:43476 10.132.0.41:443 100.64.0.4:15602 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:55.025Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.134.0.39" "Go-http-client/1.1" "a13846f4-4994-4894-bded-4b87cb4240ee" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.44:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.41:54724 10.132.0.41:443 10.134.0.39:35952 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-06-03T20:17:55.018Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 17 - "100.64.0.2" "python-requests/2.32.5" "f2f663a2-6b15-4b2c-bca1-25e5fca2eeca" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:55230 10.132.0.41:443 100.64.0.2:39816 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-06-03T20:17:55.140Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 36 321 8 - "100.64.0.2" "python-requests/2.32.5" "9988e32a-9cac-4487-9924-6d27731355bd" "maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com" "10.134.0.39:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.41:55230 10.132.0.41:443 100.64.0.2:64484 maas.apps.63212ffb-4e4c-4aad-9a75-b5a33a5b1cad.prod.konfluxeaas.com opendatahub.maas-api-route.1