2026-05-23T22:35:46.735371Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:35:46.737830Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:46.737867Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:46.743984Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:46.744118Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:46.744247Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:46.746650Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:46.746709Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-23T22:35:46.746719Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:46.747836Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:35:46.747901Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:35:46.747918Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:35:47.077232Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:47.077275Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:47.082989Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:47.083096Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:47.083304Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:47.086659Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:47.086709Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-23T22:35:47.086720Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-23T22:35:46.737Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.132.0.29" "Go-http-client/1.1" "13f3acad-08ae-476f-bf2d-7e3fc597b708" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:49268 10.133.0.48:443 10.132.0.29:36620 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-23T22:35:46.730Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 17 - "100.64.0.4" "python-requests/2.32.5" "056e94b4-9145-4f6e-81e6-dd1a89ba4470" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:43678 10.133.0.48:443 100.64.0.4:2944 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-23T22:35:47.076Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 33 318 10 - "100.64.0.3" "python-requests/2.32.5" "c2881f14-f55b-4f0e-a079-dc46eb465e4c" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:49950 10.133.0.48:443 100.64.0.3:2068 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-23T22:35:55.114311Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:55.114356Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:55.118127Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:55.118259Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:55.120444Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:55.120474Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:55.126278Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:55.126416Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:55.126505Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:55.128099Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:55.128184Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-23T22:35:55.128197Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:55.129355Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:55.129403Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-23T22:35:55.129414Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-23T22:35:55.119Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.132.0.29" "Go-http-client/1.1" "f7b68b9f-cebf-4e6f-86c3-e7319616010a" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:49268 10.133.0.48:443 10.132.0.29:36620 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-23T22:35:55.113Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 16 - "100.64.0.2" "python-requests/2.32.5" "45686b32-f6da-4d52-abf9-b478f74d8879" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:49950 10.133.0.48:443 100.64.0.2:29446 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-23T22:35:55.798781Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:55.798825Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:55.804285Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:55.804393Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:55.804574Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:55.807998Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:35:55.808054Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-23T22:35:55.808065Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-23T22:35:55.797Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 322 10 - "100.64.0.2" "python-requests/2.32.5" "efd1553e-1c8f-4067-8017-5acbed9aa498" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:49950 10.133.0.48:443 100.64.0.2:23774 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-23T22:36:03.850108Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:36:03.850156Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:36:03.853669Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:36:03.853778Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:36:03.856260Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:36:03.856302Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:36:03.856322Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:36:03.862377Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:36:03.862530Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:36:03.862637Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:36:03.862725Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:36:03.864271Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:36:03.864305Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:36:03.864356Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-23T22:36:03.864370Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:36:03.864392Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-23T22:36:03.864404Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:36:03.865489Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:36:03.865540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:36:03.865552Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-05-23T22:36:03.855Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.132.0.29" "Go-http-client/1.1" "be496e46-a23e-478a-9485-ab26dde12893" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:49268 10.133.0.48:443 10.132.0.29:36620 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-23T22:36:03.848Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 474 16 - "100.64.0.4" "python-requests/2.32.5" "2d04405b-a9f8-4e5f-9099-74a9a098bf39" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:43678 10.133.0.48:443 100.64.0.4:59860 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-23T22:36:12.950318Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:36:12.950365Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:36:12.960428Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:36:12.960577Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:36:12.960827Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:36:12.964773Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:36:12.964830Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:36:12.964841Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-05-23T22:36:12.949Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 85 328 15 - "100.64.0.4" "python-requests/2.32.5" "b15533aa-c42b-443e-81d8-93248908e96d" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:43678 10.133.0.48:443 100.64.0.4:46458 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-23T22:36:29.224490Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:36:29.224536Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:36:29.228182Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:36:29.228296Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:36:29.230126Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:36:29.230203Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-23T22:36:29.230214Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-23T22:36:29.223Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 82 6 - "100.64.0.3" "python-requests/2.32.5" "9429ab84-2318-44a6-b889-eea621ef2c1c" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:49950 10.133.0.48:443 100.64.0.3:50528 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-23T22:36:46.240994Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:36:46.241040Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:36:46.246670Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:36:46.246786Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:36:46.248791Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:36:46.248845Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-23T22:36:46.248856Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-23T22:36:46.239Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 9 - "100.64.0.3" "python-requests/2.32.5" "e90174b7-348b-48a1-b38e-8b972286051e" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:49950 10.133.0.48:443 100.64.0.3:23722 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-23T22:37:03.386448Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:37:03.386494Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:37:03.392248Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:37:03.392363Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:37:03.394269Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:37:03.394323Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-23T22:37:03.394334Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-23T22:37:03.385Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 82 9 - "100.64.0.2" "python-requests/2.32.5" "4a2afb4c-96a7-4258-a965-66bd3f40d61d" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:49950 10.133.0.48:443 100.64.0.2:40310 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-23T22:37:20.711457Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:20.711503Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:20.717380Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:20.717490Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:20.720910Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:20.720966Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:37:20.720977Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-05-23T22:37:20.710Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 10 - "100.64.0.2" "python-requests/2.32.5" "90887920-89a4-4614-a757-bba23dad8e42" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:58174 10.133.0.48:443 100.64.0.2:6558 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-23T22:37:34.677215Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:37:34.677262Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:37:34.682880Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:37:34.682996Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:37:34.683215Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:37:34.686493Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:37:34.686550Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-23T22:37:34.686560Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-23T22:37:34.675Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 45 330 10 - "100.64.0.4" "python-requests/2.32.5" "3a833849-fa5e-4432-83bf-c04aee49f0d8" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:49950 10.133.0.48:443 100.64.0.4:33574 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-23T22:37:42.716134Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:42.716198Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:42.730110Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:42.730241Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:42.740680Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:42.740712Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:42.747350Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:42.747471Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:42.747563Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:42.750596Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:42.750648Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:37:42.750658Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:42.752581Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:42.752631Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:37:42.752642Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-05-23T22:37:42.739Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 10 - "10.132.0.29" "Go-http-client/1.1" "a40c4da8-52ed-4f1c-b508-448e06c9f9bb" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.30:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:39210 10.133.0.48:443 10.132.0.29:45642 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.2 [2026-05-23T22:37:42.714Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 471 37 - "100.64.0.3" "python-requests/2.32.5" "88869a33-7792-465e-a486-67f703f556c0" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:58174 10.133.0.48:443 100.64.0.3:55544 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-23T22:37:56.828239Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:56.828286Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:56.834184Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:56.834309Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:56.834507Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:56.838027Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:56.838082Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:37:56.838093Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:56.875850Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:56.875883Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:56.880639Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:56.880728Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:56.880874Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:56.883927Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:37:56.883972Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:37:56.883982Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-05-23T22:37:56.826Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 298 11 - "100.64.0.2" "python-requests/2.32.5" "caf2aab1-d0ef-4442-8a45-dd0d05faaf60" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:58174 10.133.0.48:443 100.64.0.2:8936 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-23T22:37:56.874Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 297 9 - "100.64.0.4" "python-requests/2.32.5" "3571e1b0-3917-4892-8241-6956620cfa28" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:58174 10.133.0.48:443 100.64.0.4:5218 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-23T22:38:04.913330Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:38:04.913377Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:38:04.917200Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:38:04.917304Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:38:04.919731Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:04.919766Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:04.925730Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:04.925841Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:04.925935Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:04.927571Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:04.927635Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:38:04.927646Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:04.928917Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:38:04.928971Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-23T22:38:04.928982Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:38:04.948323Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:04.948355Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:04.951505Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:04.951598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:04.953492Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:04.953523Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:04.959350Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:04.959460Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:04.959558Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:04.971539Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:04.971594Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:38:04.971606Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:04.973554Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:04.973600Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:38:04.973611Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-05-23T22:38:04.918Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 8 - "10.132.0.29" "Go-http-client/1.1" "1b5da8ca-0c86-469a-9b72-0f57bb0eac15" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.30:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:39210 10.133.0.48:443 10.132.0.29:45642 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.2 [2026-05-23T22:38:04.912Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 468 16 - "100.64.0.3" "python-requests/2.32.5" "0e20bbcd-8038-440c-9e5b-632edf137e96" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:49950 10.133.0.48:443 100.64.0.3:19738 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-23T22:38:04.952Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 18 - "10.132.0.29" "Go-http-client/1.1" "e8d41c94-2e51-4120-a90c-ccbcd8f4d501" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:47880 10.133.0.48:443 10.132.0.29:45642 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.2 [2026-05-23T22:38:04.947Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 345 26 - "100.64.0.4" "python-requests/2.32.5" "a6d9aa43-5496-4a02-8872-a61e1b04624e" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:58174 10.133.0.48:443 100.64.0.4:9072 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-23T22:38:21.349299Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:38:21.349348Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:38:21.354629Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:38:21.354744Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:38:21.357416Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:21.357453Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:21.366328Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:21.366445Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:21.368149Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:21.368233Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:38:21.368245Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:21.369669Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:21.369701Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-05-23T22:38:21.356Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 11 - "10.132.0.29" "Go-http-client/1.1" "ae2006a9-3f92-4cc4-9a4d-bed66f8abd7d" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.30:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:39210 10.133.0.48:443 10.132.0.29:45642 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.2 2026-05-23T22:38:21.377814Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:21.377914Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:21.379533Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:21.379588Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:38:21.379599Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:21.380825Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:21.380857Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:21.389040Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:21.389138Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:21.393058Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:21.393105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:38:21.393116Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:21.394301Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:38:21.394361Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-23T22:38:21.394374Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-23T22:38:21.368Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 10 - "10.132.0.29" "Go-http-client/1.1" "6c9a3e1c-c188-4131-98c7-0441650f9b72" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:47880 10.133.0.48:443 10.132.0.29:45642 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.2 [2026-05-23T22:38:21.380Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 12 - "10.132.0.29" "Go-http-client/1.1" "78cf7358-7ac5-43d3-a247-1476051beefa" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:58400 10.133.0.48:443 10.132.0.29:45642 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-23T22:38:21.348Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 1361 46 - "100.64.0.3" "python-requests/2.32.5" "79619477-3a2f-4844-aab7-6713fde23c91" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:49950 10.133.0.48:443 100.64.0.3:24942 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-23T22:38:37.754643Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:37.754691Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:37.759890Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:37.760001Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:37.762347Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:37.762380Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:37.771128Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:37.771262Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:37.772837Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:37.772890Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:38:37.772901Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:37.773976Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:37.774024Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:38:37.774035Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:37.789994Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:38:37.790029Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:38:37.794583Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:38:37.794684Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:38:37.796793Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:37.796824Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:37.804822Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:37.804920Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:37.806360Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:37.806406Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:38:37.806416Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:38:37.807568Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:38:37.807626Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-23T22:38:37.807638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-23T22:38:37.761Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 11 - "10.132.0.29" "Go-http-client/1.1" "a2078b9a-a04c-45b6-a859-6f1626e598d5" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.30:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:39210 10.133.0.48:443 10.132.0.29:45642 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.2 [2026-05-23T22:38:37.753Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 470 20 - "100.64.0.2" "python-requests/2.32.5" "bac67492-210f-4172-9f01-27e82c71725d" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:58174 10.133.0.48:443 100.64.0.2:1688 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-23T22:38:37.796Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 10 - "10.132.0.29" "Go-http-client/1.1" "deee0fdc-6481-41d1-8df5-8a2beac26a7c" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.32:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:47880 10.133.0.48:443 10.132.0.29:45642 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.2 [2026-05-23T22:38:37.789Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 347 18 - "100.64.0.4" "python-requests/2.32.5" "7b2406f6-0cce-450d-b936-230134119f5f" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:49950 10.133.0.48:443 100.64.0.4:21592 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-23T22:38:46.791119Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:38:46.791185Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:38:46.792534Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:38:46.792566Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-23T22:38:46.790Z] "GET /maas-api/v1/models HTTP/1.1" 401 - - "-" 0 0 2 - "100.64.0.2" "python-requests/2.32.5" "436c30e7-a05e-40d9-849e-ea69a08618aa" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.133.0.48:443 100.64.0.2:13800 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-23T22:39:03.388331Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.388387Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.394326Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.394454Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.394661Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.400107Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.400191Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:39:03.400207Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.426300Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.426337Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.433577Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.433702Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.433942Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.433982Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.435035Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.435067Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.435304Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.438888Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.439040Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.439253Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.439718Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-05-23T22:39:03.387Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 330 13 - "100.64.0.3" "python-requests/2.32.5" "d77a1185-0af1-4248-be3e-a670ac4b85c3" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:58174 10.133.0.48:443 100.64.0.3:4324 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-23T22:39:03.425Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 292 13 - "100.64.0.2" "python-requests/2.32.5" "8ace8083-c340-407d-877d-be6ce4be83a7" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:50910 10.133.0.48:443 100.64.0.2:27170 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 2026-05-23T22:39:03.477053Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:39:03.477088Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:39:03.484065Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:39:03.484198Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:39:03.484444Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:39:03.484474Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:39:03.484752Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:39:03.484774Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:39:03.501184Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:39:03.501214Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:39:03.507806Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:39:03.507912Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:39:03.508135Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:39:03.508188Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:39:03.508428Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:39:03.508448Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:39:03.530722Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.530761Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.533945Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.534047Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.536466Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.536510Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.542907Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.543034Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.543134Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.544494Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.544553Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:39:03.544567Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.545866Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:03.545925Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:39:03.545941Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-05-23T22:39:03.476Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 8 - "100.64.0.4" "python-requests/2.32.5" "e48ac96f-bf5b-42aa-98ce-a56b83ede7be" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:47384 10.133.0.48:443 100.64.0.4:15652 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-05-23T22:39:03.500Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 7 - "100.64.0.4" "python-requests/2.32.5" "f20be66c-cdf9-4c9e-b88d-ceeb024589c0" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.48:443 100.64.0.4:15186 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-05-23T22:39:03.535Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.132.0.29" "Go-http-client/1.1" "4e6ba811-b6f9-49ad-a9fc-49d394f013e7" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.134.0.34:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:50910 10.133.0.48:443 10.132.0.29:45642 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.2 [2026-05-23T22:39:03.529Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 547 16 - "100.64.0.3" "python-requests/2.32.5" "a4d987b1-5834-4b32-b008-02a5ce926e44" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:58174 10.133.0.48:443 100.64.0.3:4330 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-23T22:39:12.384352Z info ads ADS: new connection for node:6 2026-05-23T22:39:12.387749Z info cache read certificate from file resource=file-root:system 2026-05-23T22:39:19.112348Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:19.112397Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:19.118331Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:19.118441Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:19.118634Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:19.122259Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:19.122313Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:39:19.122324Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-05-23T22:39:19.111Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 321 11 - "100.64.0.2" "python-requests/2.32.5" "17a80a82-bd69-45ba-b0c7-6a07862bc566" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:58174 10.133.0.48:443 100.64.0.2:25948 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-23T22:39:19.483446Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:39:19.483485Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:39:19.484645Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:39:19.484679Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:39:19.502441Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:19.502474Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:19.503550Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:19.503587Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:19.523217Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:19.523249Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:19.529920Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:19.530028Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:19.530285Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:19.530314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:19.531234Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:19.531260Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:19.531469Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:19.532467Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:39:19.532572Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:39:19.532583Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-05-23T22:39:19.482Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 2 - "100.64.0.3" "python-requests/2.32.5" "da3f157d-ffb8-443d-915e-6aaadc0dbe44" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.133.0.48:443 100.64.0.3:23022 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-05-23T22:39:19.501Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 2 - "100.64.0.4" "python-requests/2.32.5" "61cd68e5-9578-4635-84fc-131d422f81dd" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.133.0.48:443 100.64.0.4:42898 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-05-23T22:39:19.522Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 404 - - "-" 83 98 10 - "100.64.0.3" "python-requests/2.32.5" "d0fcecde-fed6-468d-ba8f-ceb519e9b8cf" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.133.0.48:443 100.64.0.3:55718 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.e2e-external-model.0 2026-05-23T22:39:57.167721Z info ads ADS: "" 6 terminated with stream closed 2026-05-23T22:40:01.017391Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.017438Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.024657Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.024775Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.024961Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.038913Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.038980Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-23T22:40:01.038992Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.084894Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.084934Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.090039Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.090145Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.092704Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.092735Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.099112Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.099280Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.099375Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.100900Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.100953Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:40:01.100964Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.102431Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.102493Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-23T22:40:01.102505Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.149017Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.149052Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.155918Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.156056Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.156312Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.156341Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.157305Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.157331Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.157536Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.159670Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.159810Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.159982Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.160343Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.196284Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.196314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.202903Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.203033Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.203271Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.203300Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.203598Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.203620Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.203817Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.205608Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.205742Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.205910Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.206183Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.229741Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.229772Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.236464Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.236587Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.236799Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.236827Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.237066Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.237089Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.269469Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.269499Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.276322Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.276448Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.276663Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.276690Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.276937Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.276958Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.300000Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.300035Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.306715Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.306847Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.307075Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.307105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.307385Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.307414Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.339658Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.339687Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.345956Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.346079Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.346304Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.346331Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.346585Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.346605Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-23T22:40:01.402678Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.402709Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.408919Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.409045Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.409289Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.409318Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.409563Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.409584Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.442299Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.442329Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.448854Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.448976Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.449221Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.449254Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.449521Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.449541Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.493354Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.493384Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.499869Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.499990Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.500240Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.500268Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.500527Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.500549Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.530312Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.530343Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.537094Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.537236Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.537456Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.537483Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.537735Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.537757Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-23T22:40:01.016Z] "POST /v1/api-keys HTTP/2" 201 - via_upstream - "-" 55 320 22 - "100.64.0.2" "curl/7.76.1" "9e1f6e00-9410-45ea-bdae-7d6fa45d3d4a" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:37052 10.133.0.48:443 100.64.0.2:63416 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-23T22:40:01.091Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.132.0.29" "Go-http-client/1.1" "bba80d26-9e96-45e7-b136-72588f2ad48b" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:58400 10.133.0.48:443 10.132.0.29:45642 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.2 [2026-05-23T22:40:01.083Z] "GET /v1/models HTTP/2" 200 - via_upstream - "-" 0 590 18 - "100.64.0.4" "curl/7.76.1" "953626cf-abb0-43d7-987c-7578a25f3422" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:58174 10.133.0.48:443 100.64.0.4:64902 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-23T22:40:01.148Z] "POST /v1/chat/completions HTTP/2" 200 - via_upstream - "-" 100 514 11 - "100.64.0.3" "curl/7.76.1" "a8b3b7e1-b8c6-4377-8040-097701336fc5" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:49268 10.133.0.48:443 100.64.0.3:33388 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-23T22:40:01.195Z] "POST /v1/chat/completions HTTP/2" 200 - via_upstream - "-" 100 506 10 - "100.64.0.2" "curl/7.76.1" "0eb3a60a-00ad-4647-8cd3-fb1a486edf09" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:58400 10.133.0.48:443 100.64.0.2:48974 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-23T22:40:01.229Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 7 - "100.64.0.4" "curl/7.76.1" "670d5dda-0d51-47f3-a535-29185072e698" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:58400 10.133.0.48:443 100.64.0.4:10326 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-23T22:40:01.269Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 8 - "100.64.0.3" "curl/7.76.1" "35bb847e-c114-45ef-85ae-4cd7e2ac44cc" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.48:443 100.64.0.3:41108 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-23T22:40:01.299Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 8 - "100.64.0.2" "curl/7.76.1" "5daf7916-9e5a-4f94-8d2a-244816e64251" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:49268 10.133.0.48:443 100.64.0.2:49380 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-23T22:40:01.339Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 7 - "100.64.0.2" "curl/7.76.1" "b22e9d76-4fc0-43b7-8e16-e9188a4b6722" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:35206 10.133.0.48:443 100.64.0.2:48976 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-23T22:40:01.402Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 7 - "100.64.0.4" "curl/7.76.1" "9ca3f42f-aa10-44b0-8f1c-810cee54c66b" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.48:443 100.64.0.4:10340 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-23T22:40:01.441Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 7 - "100.64.0.3" "curl/7.76.1" "49135229-eb23-45ef-ad3b-1f20fdde13ca" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:35220 10.133.0.48:443 100.64.0.3:41124 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-23T22:40:01.492Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 7 - "100.64.0.2" "curl/7.76.1" "7dfb8606-4f32-42fa-b7ab-25f322bd4d3e" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.48:443 100.64.0.2:48990 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-23T22:40:01.529Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 8 - "100.64.0.4" "curl/7.76.1" "6d8a6378-b531-4ed5-9c79-46a754940c1a" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.133.0.48:35222 10.133.0.48:443 100.64.0.4:10352 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 2026-05-23T22:40:01.587204Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.587243Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.594011Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.594131Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.594365Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.594395Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.594635Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.594656Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.629712Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.629743Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.630768Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.630795Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.660099Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.660132Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.664843Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.664931Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.669277Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 2026-05-23T22:40:01.669335Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=17 2026-05-23T22:40:01.669347Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=17 [2026-05-23T22:40:01.586Z] "POST /v1/chat/completions HTTP/2" 429 - - "-" 100 18 8 - "100.64.0.3" "curl/7.76.1" "5960178d-f24f-434f-b200-73ef1d31f080" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.33:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.48:443 100.64.0.3:41130 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-23T22:40:01.629Z] "POST /llm/facebook-opt-125m-simulated/v1/chat/completions HTTP/2" 401 - - "-" 100 0 1 - "100.64.0.2" "curl/7.76.1" "07174534-d942-4c62-be54-70e0b14f80c1" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "-" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local - 10.133.0.48:443 100.64.0.2:49004 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-23T22:40:01.659Z] "DELETE /v1/api-keys/03560a0f-a625-4f65-aa66-5a4a92235ab9 HTTP/2" 200 - via_upstream - "-" 0 334 10 - "100.64.0.4" "curl/7.76.1" "e003cc94-fe5a-4f0d-9b4a-b979f8a3b8f0" "maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com" "10.132.0.29:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.133.0.48:37052 10.133.0.48:443 100.64.0.4:14504 maas.apps.8f1a0247-d951-4f2a-b528-bfe9c65fc73b.prod.konfluxeaas.com opendatahub.maas-api-route.1