2026-05-31T17:59:19.373134Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-31T17:59:19.353Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 19 - "100.64.0.4" "python-requests/2.32.5" "75acee2b-8452-45d2-99be-5365cb6c940f" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.4:32148 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-31T17:59:33.360483Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T17:59:33.360532Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T17:59:33.373474Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T17:59:33.373596Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T17:59:33.373801Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T17:59:33.378426Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T17:59:33.378484Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T17:59:33.378495Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-31T17:59:33.359Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 45 329 19 - "100.64.0.3" "python-requests/2.32.5" "d495f7c5-7d75-4faf-96ef-669d6727eec6" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.3:54998 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-31T17:59:41.408056Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T17:59:41.408112Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T17:59:41.422721Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T17:59:41.422863Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T17:59:41.434428Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T17:59:41.434463Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T17:59:41.441453Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T17:59:41.441580Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T17:59:41.441691Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T17:59:41.443430Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T17:59:41.443488Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T17:59:41.443499Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T17:59:41.444560Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T17:59:41.444609Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T17:59:41.444620Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-05-31T17:59:41.433Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 10 - "10.133.0.51" "Go-http-client/1.1" "39c0ef6c-74b1-419d-82ed-6c13efceb45a" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.52:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.25:43760 10.132.0.25:443 10.133.0.51:35110 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-05-31T17:59:41.406Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 471 37 - "100.64.0.2" "python-requests/2.32.5" "ce075392-8aad-429a-a2d3-933141b6e016" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:36470 10.132.0.25:443 100.64.0.2:50848 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-31T17:59:55.589377Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T17:59:55.589421Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T17:59:55.602542Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T17:59:55.602681Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T17:59:55.602907Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T17:59:55.609012Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T17:59:55.609074Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T17:59:55.609084Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T17:59:55.637834Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T17:59:55.637875Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T17:59:55.647343Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T17:59:55.647482Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T17:59:55.647693Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T17:59:55.651038Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T17:59:55.651105Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T17:59:55.651120Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-31T17:59:55.588Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 297 21 - "100.64.0.4" "python-requests/2.32.5" "414974fc-9127-4575-a26f-da9fbfce74d8" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:36470 10.132.0.25:443 100.64.0.4:10490 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T17:59:55.636Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 55 298 14 - "100.64.0.2" "python-requests/2.32.5" "e4364d36-00e9-4944-b127-15fd6ceee280" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.2:56464 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-31T18:00:03.701181Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:03.701236Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:03.705627Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:03.705808Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:03.708733Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:03.708771Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:03.715203Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:03.715374Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:03.715504Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:03.717358Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:03.717418Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:00:03.717428Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:03.718662Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:03.718718Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T18:00:03.718729Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:03.742110Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:03.742147Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:03.745606Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:03.745701Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:03.747841Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:03.747871Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:03.754163Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:03.754279Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:03.754393Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:03.756175Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:03.756224Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:00:03.756234Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:03.757281Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:03.757359Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:00:03.757371Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-05-31T18:00:03.707Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 10 - "10.133.0.51" "Go-http-client/1.1" "5ca888a5-4d5a-45c2-a64c-e70eca38ca3e" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.52:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.25:43760 10.132.0.25:443 10.133.0.51:35110 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-05-31T18:00:03.699Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 468 18 - "100.64.0.3" "python-requests/2.32.5" "e38f7327-f89e-4228-a7ef-b61ddcaaa250" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.3:11604 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:00:03.746Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 9 - "10.133.0.51" "Go-http-client/1.1" "40d94b74-a9fc-42cf-b5ce-204c6ae99c68" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.134.0.40:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.25:54566 10.132.0.25:443 10.133.0.51:35110 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-05-31T18:00:03.741Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 345 16 - "100.64.0.4" "python-requests/2.32.5" "7e54feac-4aee-4954-8082-443a9a628adf" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:36470 10.132.0.25:443 100.64.0.4:18220 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-31T18:00:22.289941Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:22.289989Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:22.305753Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:22.305878Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:22.308665Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:22.308707Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:22.318309Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:22.318469Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:22.320186Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:22.320259Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:00:22.320275Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:22.321750Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:22.321787Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:22.330682Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:22.330840Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:22.332601Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:22.332664Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:00:22.332679Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:22.334118Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:22.334149Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:22.343074Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:22.343172Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:22.344814Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:22.344862Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:00:22.344873Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:22.346203Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:22.346268Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T18:00:22.346281Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-31T18:00:22.307Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 12 - "10.133.0.51" "Go-http-client/1.1" "336067b1-ad36-44dd-8b0d-0e65f89e2e59" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.52:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.25:43760 10.132.0.25:443 10.133.0.51:35110 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-05-31T18:00:22.320Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 11 - "10.133.0.51" "Go-http-client/1.1" "9cd63ebe-9aec-4b1c-979e-aedd9294686a" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.134.0.40:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.25:54566 10.132.0.25:443 10.133.0.51:35110 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-05-31T18:00:22.333Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 11 - "10.133.0.51" "Go-http-client/1.1" "d0ee4ecf-cf2b-4744-88a0-0ce4ad7137e2" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.134.0.43:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.25:58256 10.132.0.25:443 10.133.0.51:35110 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-05-31T18:00:22.288Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 1361 57 - "100.64.0.3" "python-requests/2.32.5" "1617417c-5272-4dbb-ac1e-65bb347ed0a5" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.3:15774 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-31T18:00:40.847764Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:40.847823Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:40.863371Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:40.863497Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:40.866208Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:40.866243Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:40.877301Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:40.877458Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:40.879436Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:40.879494Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:00:40.879505Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:40.880765Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:40.880821Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T18:00:40.880832Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:40.904437Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:40.904472Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:40.914047Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:40.914152Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:40.916517Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:40.916555Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:40.925653Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:40.925758Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:40.927401Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:40.927451Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:00:40.927461Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:40.928620Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:00:40.928670Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T18:00:40.928681Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-31T18:00:40.865Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 162 14 - "10.133.0.51" "Go-http-client/1.1" "3170d029-08a7-4d5b-a6c3-33eebbf704b6" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.52:8000" outbound|8000||e2e-distinct-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.25:43760 10.132.0.25:443 10.133.0.51:35110 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com llm.e2e-distinct-simulated-kserve-route.3 [2026-05-31T18:00:40.845Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 470 35 - "100.64.0.2" "python-requests/2.32.5" "fd25cf2b-e23e-4bec-8bde-47c348a9e2a8" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.2:53006 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:00:40.915Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 166 11 - "10.133.0.51" "Go-http-client/1.1" "88959307-52c6-4c4a-9cd1-0315510f3642" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.134.0.40:8000" outbound|8000||e2e-distinct-2-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.25:54566 10.132.0.25:443 10.133.0.51:35110 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com llm.e2e-distinct-2-simulated-kserve-route.3 [2026-05-31T18:00:40.903Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 347 25 - "100.64.0.4" "python-requests/2.32.5" "1ba322ca-785e-48b2-8674-74987986f5b3" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.4:38520 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-31T18:00:49.922441Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:49.922492Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:49.924369Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:00:49.924402Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-05-31T18:00:49.921Z] "GET /maas-api/v1/models HTTP/1.1" 401 - - "-" 0 0 3 - "100.64.0.3" "python-requests/2.32.5" "bcea368b-e171-4485-ae2d-dbf6ca72e234" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.132.0.25:443 100.64.0.3:13608 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-31T18:01:06.542415Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.542470Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.557858Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.558003Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.558233Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.562799Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.562870Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T18:01:06.562886Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.585651Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.585690Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.593860Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.593988Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.594250Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.594292Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.595568Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.595599Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.595818Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.598359Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.598508Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.598703Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.599280Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.618257Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.618295Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.626160Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.626287Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.626540Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.626578Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.627021Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.627051Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:06.646200Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:06.646241Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:06.653379Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:06.653491Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:06.653739Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:06.653769Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:06.654259Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:06.654282Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:06.671952Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:06.671988Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:06.675475Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:06.675569Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:06.679927Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:06.679965Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:06.686386Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:06.686516Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:06.686622Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:06.689994Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:06.690054Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:01:06.690069Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:06.691417Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:06.691475Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:01:06.691491Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-05-31T18:01:06.541Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 88 330 21 - "100.64.0.2" "python-requests/2.32.5" "aa60c256-5a17-4166-9ff6-3e7c1076fc86" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.2:6914 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:01:06.585Z] "POST /v1/completions HTTP/1.1" 200 - via_upstream - "-" 66 295 14 - "100.64.0.4" "python-requests/2.32.5" "d52b189c-716a-402e-8ded-a89f6a33759d" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.134.0.42:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.25:57604 10.132.0.25:443 100.64.0.4:52660 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-05-31T18:01:06.617Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 9 - "100.64.0.3" "python-requests/2.32.5" "1ce4de92-8309-4d4a-90ac-4cced4ab188f" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.134.0.42:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.25:57604 10.132.0.25:443 100.64.0.3:29608 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-05-31T18:01:06.645Z] "POST /v1/completions HTTP/1.1" 429 - - "-" 66 18 8 - "100.64.0.2" "python-requests/2.32.5" "a793cfbf-522b-4e16-bdb6-fe1ed84b898c" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.134.0.42:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.25:57610 10.132.0.25:443 100.64.0.2:6920 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.0 [2026-05-31T18:01:06.679Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 10 - "10.133.0.51" "Go-http-client/1.1" "694091ff-67d8-455f-b7a1-ed7b7c6fe7dc" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.134.0.42:8000" outbound|8000||e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc.llm.svc.cluster.local 10.132.0.25:55168 10.132.0.25:443 10.133.0.51:35110 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com llm.e2e-unconfigured-facebook-opt-125m-simulated-kserve-route.3 [2026-05-31T18:01:06.670Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 547 20 - "100.64.0.4" "python-requests/2.32.5" "bca5d895-8f99-4983-a423-11956d3d3e62" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:45530 10.132.0.25:443 100.64.0.4:52664 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-31T18:01:15.523241Z info ads ADS: new connection for node:6 2026-05-31T18:01:15.526154Z info cache read certificate from file resource=file-root:system 2026-05-31T18:01:24.372044Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:24.372093Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:24.387681Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:24.387801Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:24.388011Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:24.391853Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:24.391910Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T18:01:24.391921Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:24.753308Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:24.753366Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:24.754665Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:24.754697Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:01:24.775900Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:24.775936Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:24.777024Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:24.777055Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:24.803722Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:24.803755Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:24.810565Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:24.810679Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:24.810916Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:24.810944Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:24.812146Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:24.812170Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:24.812412Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:24.813442Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:01:24.813553Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T18:01:24.813563Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-31T18:01:24.370Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 79 321 21 - "100.64.0.3" "python-requests/2.32.5" "b396f114-7a7c-4380-8710-48638d477b16" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.3:44928 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:01:24.752Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 2 - "100.64.0.2" "python-requests/2.32.5" "fb126e99-e5c5-4131-887d-db53452df4ee" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.132.0.25:443 100.64.0.2:2162 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-05-31T18:01:24.774Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 401 - - "-" 83 0 2 - "100.64.0.4" "python-requests/2.32.5" "1478c5e9-315e-486f-878d-3b4bee5d2f15" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.132.0.25:443 100.64.0.4:9158 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com llm.e2e-external-model.0 [2026-05-31T18:01:24.802Z] "POST /llm/e2e-external-model/v1/chat/completions HTTP/1.1" 404 - - "-" 83 98 10 - "100.64.0.3" "python-requests/2.32.5" "df23802d-23fc-4759-9765-4ec107fd4631" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "-" outbound|443||httpbin.org - 10.132.0.25:443 100.64.0.3:44940 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com llm.e2e-external-model.0 2026-05-31T18:02:02.445629Z info ads ADS: "" 6 terminated 2026-05-31T18:02:04.247091Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:04.247141Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:04.250304Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:04.250450Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:04.250648Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:04.263562Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:04.263620Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T18:02:04.263630Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:04.350630Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:04.350673Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:04.374142Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:04.374175Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:04.399638Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:04.399678Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:04.401283Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:04.401314Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:04.425634Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:04.425675Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:04.427158Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:04.427189Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:04.814577Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:04.814629Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:04.817667Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:04.817784Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:04.817977Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:04.821979Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:04.822035Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:02:04.822046Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-05-31T18:02:04.245Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 17 - "100.64.0.2" "python-requests/2.32.5" "848bac96-7097-4c28-b8dc-d5764eaafde0" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.2:62030 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:02:04.349Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 37 0 24 - "100.64.0.4" "python-requests/2.32.5" "0d884972-e794-4f54-b857-08694d1e82ad" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.132.0.25:443 100.64.0.4:9416 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:02:04.398Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 35 0 2 - "100.64.0.2" "python-requests/2.32.5" "8afd3bf9-4642-4e08-86cd-1711a498cb79" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.132.0.25:443 100.64.0.2:37880 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:02:04.424Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 36 0 2 - "100.64.0.3" "python-requests/2.32.5" "97854b6d-d072-4c61-a555-07853e27b0e6" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.132.0.25:443 100.64.0.3:55356 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:02:04.813Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 28 312 8 - "100.64.0.4" "python-requests/2.32.5" "bace5387-0b7a-42a6-beaa-b10ca4ee3726" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:45530 10.132.0.25:443 100.64.0.4:52594 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-31T18:02:05.028236Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:05.028282Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:05.031349Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:05.031466Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:05.031661Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:05.035420Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:05.035477Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:02:05.035485Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:05.056615Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.056653Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.061121Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.061238Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.064369Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:05.064398Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:05.070613Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:05.070748Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:05.070852Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:05.072625Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:05.072689Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:02:05.072703Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:05.074215Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.074274Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T18:02:05.074286Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.112929Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.112964Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.120093Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.120223Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.120481Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.120510Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.121944Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.121964Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.122157Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.124245Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.124394Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.124563Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.125037Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.214232Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.214282Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.217280Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.217411Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.217605Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.221194Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.221250Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T18:02:05.221261Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.237898Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.237931Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.240670Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.240765Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.245303Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:05.245375Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T18:02:05.245388Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-31T18:02:05.026Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 8 - "100.64.0.3" "python-requests/2.32.5" "477100ce-24c1-43bb-a7cd-ca4b89a4d281" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:45530 10.132.0.25:443 100.64.0.3:25838 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:02:05.063Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.133.0.51" "Go-http-client/1.1" "e82e7b40-24c8-4f65-8509-2cd8176a1685" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.134.0.43:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.25:58256 10.132.0.25:443 10.133.0.51:35110 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-05-31T18:02:05.055Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 18 - "100.64.0.2" "python-requests/2.32.5" "97227378-bf7a-45f9-8024-5b5a83b849a3" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.2:37888 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:02:05.112Z] "POST /v1/chat/completions HTTP/1.1" 200 - via_upstream - "-" 123 325 12 - "100.64.0.2" "python-requests/2.32.5" "57e6176a-36bd-4127-8aca-b43d3caf8ea9" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.134.0.43:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.25:52544 10.132.0.25:443 100.64.0.2:62034 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.1 [2026-05-31T18:02:05.212Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 313 8 - "100.64.0.3" "python-requests/2.32.5" "40428f83-acf8-41e1-8957-ad60322f277a" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.3:12274 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:02:05.236Z] "DELETE /v1/api-keys/93c32f67-6524-4d08-a198-cb396fb50315 HTTP/1.1" 200 - via_upstream - "-" 0 251 8 - "100.64.0.4" "python-requests/2.32.5" "2adaa2e0-31a4-4dbb-97f0-deee5bddb44f" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.4:9432 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-31T18:02:08.267511Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.267556Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.271581Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.271655Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.273432Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.273487Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T18:02:08.273497Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.397305Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.397376Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.410472Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.410506Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.555948Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.555993Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.559168Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.559292Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.559516Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.562949Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.563005Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T18:02:08.563016Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.585049Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.585085Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.588563Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.588667Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.588828Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.592458Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.592508Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:02:08.592519Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.678539Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.678579Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.681747Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.681847Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.682019Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.686153Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.686201Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T18:02:08.686212Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.704858Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.704892Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.707921Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.708018Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.712040Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.712086Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:02:08.712096Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.733696Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.733731Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.736660Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.736757Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.738976Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.739022Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T18:02:08.739032Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.825345Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.825386Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.828166Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.828266Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.828450Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.831818Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.831873Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T18:02:08.831884Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.852453Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.852486Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.856088Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.856192Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.858519Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.858549Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.864994Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.865131Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.865223Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.867001Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.867049Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:02:08.867059Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.868197Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.868243Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:02:08.868254Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.951937Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.951973Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.955114Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.955207Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.955393Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.958759Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.958806Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:02:08.958816Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:08.979019Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.979054Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.983220Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.983351Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.984986Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:08.985029Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T18:02:08.985039Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:09.004428Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.004460Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.007910Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.007997Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.009972Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.010002Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 [2026-05-31T18:02:08.266Z] "GET /v1/models HTTP/1.1" 500 - via_upstream - "-" 0 96 7 - "100.64.0.4" "python-requests/2.32.5" "9776d732-84d4-43db-810a-7f04495ab7ec" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.4:31120 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:02:08.396Z] "POST /maas-api/v1/api-keys HTTP/1.1" 401 - - "-" 38 0 14 - "100.64.0.2" "python-requests/2.32.5" "b8bc5a9c-2ca4-4477-95ab-b00c59e314aa" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "-" outbound|8443||maas-api.opendatahub.svc.cluster.local - 10.132.0.25:443 100.64.0.2:4338 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:02:08.554Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 34 318 8 - "100.64.0.3" "python-requests/2.32.5" "1076b948-f9b4-49c7-b0b0-0f766cb9c3c2" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.3:12284 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:02:08.583Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 32 317 8 - "100.64.0.4" "python-requests/2.32.5" "2b1e2eee-c472-47db-bd09-c58cf3e4af62" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:45530 10.132.0.25:443 100.64.0.4:31122 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:02:08.677Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 31 315 8 - "100.64.0.3" "python-requests/2.32.5" "81df96a8-1a97-4bf9-93ea-a00b6f155ecb" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.3:25844 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:02:08.703Z] "DELETE /v1/api-keys/9349f48a-2632-4c9f-ab15-8bee4b6eef5f HTTP/1.1" 200 - via_upstream - "-" 0 253 8 - "100.64.0.2" "python-requests/2.32.5" "6b20603a-4f55-4716-9abe-c80dfbdc0a64" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:45530 10.132.0.25:443 100.64.0.2:62040 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:02:08.732Z] "DELETE /v1/api-keys/9349f48a-2632-4c9f-ab15-8bee4b6eef5f HTTP/1.1" 404 - via_upstream - "-" 0 29 6 - "100.64.0.4" "python-requests/2.32.5" "877ee64a-a579-4720-b708-ff6a4b0d4aef" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.4:9436 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:02:08.824Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 7 - "100.64.0.3" "python-requests/2.32.5" "5e681cbf-339c-4a52-9b0a-89c3424265a5" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.3:25846 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:02:08.857Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.133.0.51" "Go-http-client/1.1" "32d99656-97f4-499f-ad8e-900faef78b15" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.134.0.43:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.25:58256 10.132.0.25:443 10.133.0.51:35110 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-05-31T18:02:08.851Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 16 - "100.64.0.2" "python-requests/2.32.5" "5d03a673-1535-4a62-aaca-750562542ecc" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:45530 10.132.0.25:443 100.64.0.2:4346 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:02:08.950Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 8 - "100.64.0.2" "python-requests/2.32.5" "223891ac-5cbf-4d49-a2a2-ce17b0257310" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:45530 10.132.0.25:443 100.64.0.2:62050 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:02:08.977Z] "GET /v1/models HTTP/1.1" 403 - via_upstream - "-" 0 89 7 - "100.64.0.3" "python-requests/2.32.5" "b5ae6275-2fe7-4cbf-8b84-d6ef4426588a" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.3:12286 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 2026-05-31T18:02:09.018178Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.018307Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.018415Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.020404Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.020451Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:02:09.020462Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.021805Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.021850Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:02:09.021861Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.115405Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.115446Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.118383Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.118484Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.118654Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.122029Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.122079Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:02:09.122087Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.144618Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:09.144651Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:09.148511Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:09.148614Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:09.150653Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.150684Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.156668Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.156797Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.156886Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.158466Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.158516Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:02:09.158526Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.159516Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:09.159564Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T18:02:09.159575Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:09.178468Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:09.178505Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:09.181647Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:09.181735Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:09.183668Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.183697Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.190160Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.190282Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.190387Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.191915Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.191960Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=19 2026-05-31T18:02:09.191970Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=19 2026-05-31T18:02:09.192973Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:09.193019Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T18:02:09.193030Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:09.280533Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:09.280576Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:09.283695Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:09.283805Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:09.283986Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:09.287408Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 2026-05-31T18:02:09.287458Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Missing json property: /usage/total_tokens thread=18 2026-05-31T18:02:09.287469Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1140 wasm log openshift-ingress.kuadrant-maas-default-gateway: Span buffer full (100), dropping oldest span thread=18 [2026-05-31T18:02:09.009Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 11 - "10.133.0.51" "Go-http-client/1.1" "7ee23423-59d9-4e73-8f67-e32419888119" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.134.0.43:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.25:58256 10.132.0.25:443 10.133.0.51:35110 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-05-31T18:02:09.003Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 18 - "100.64.0.4" "python-requests/2.32.5" "54d78ed3-9a54-4ee2-979f-fb275a967df6" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:45530 10.132.0.25:443 100.64.0.4:31130 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:02:09.114Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 29 314 7 - "100.64.0.2" "python-requests/2.32.5" "059c4fec-3ac3-434a-a4dc-250af6055e9c" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:45530 10.132.0.25:443 100.64.0.2:4358 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:02:09.149Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 8 - "10.133.0.51" "Go-http-client/1.1" "c1acc0de-21f4-4b8a-aa9a-c8699c14a2e7" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.134.0.43:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.25:58256 10.132.0.25:443 10.133.0.51:35110 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-05-31T18:02:09.143Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 16 - "100.64.0.4" "python-requests/2.32.5" "ab6b07d9-3aac-4cee-8ecb-2df6d86bdcba" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.4:9440 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:02:09.182Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 150 9 - "10.133.0.51" "Go-http-client/1.1" "1caf9028-e44d-42fa-a7a6-74e95fd1abff" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.134.0.43:8000" outbound|8000||facebook-opt-125m-simulated-kserve-workload-svc.llm.svc.cluster.local 10.132.0.25:58256 10.132.0.25:443 10.133.0.51:35110 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com llm.facebook-opt-125m-simulated-kserve-route.3 [2026-05-31T18:02:09.177Z] "GET /v1/models HTTP/1.1" 200 - via_upstream - "-" 0 590 15 - "100.64.0.3" "python-requests/2.32.5" "cd4e4e82-8353-44cb-a951-124f180caaba" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.3:25862 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1 [2026-05-31T18:02:09.279Z] "POST /v1/api-keys HTTP/1.1" 201 - via_upstream - "-" 36 321 8 - "100.64.0.3" "python-requests/2.32.5" "9b17acf5-7291-9a84-8399-f8b653dc07c5" "maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com" "10.133.0.51:8443" outbound|8443||maas-api.opendatahub.svc.cluster.local 10.132.0.25:51740 10.132.0.25:443 100.64.0.3:12298 maas.apps.8ddc7072-e4c4-4f40-a73e-418d40047252.prod.konfluxeaas.com opendatahub.maas-api-route.1